cortav  Check-in [7ba2577283]

Overview
Comment:continue iterating on groff renderer; add headings, basic formatting, beginnings of a footnote and link system, colors
Downloads: Tarball | ZIP archive | SQL archive
Timelines: family | ancestors | descendants | both | trunk
Files: files | file ages | folders
SHA3-256: 7ba2577283a82ca269a448ad4b2a40725ba066c32a5e68351cd3d3224a24c09e
User & Date: lexi on 2021-12-29 12:19:20
Other Links: manifest | tags
Context
2022-04-17
21:16
various updates fuck man idr what all i did check-in: c0bdfa46df user: lexi tags: trunk
2021-12-29
12:19
continue iterating on groff renderer; add headings, basic formatting, beginnings of a footnote and link system, colors check-in: 7ba2577283 user: lexi tags: trunk
2021-12-27
13:20
add beginnings of groff renderer, document more planned syntaxes check-in: 9215a9c850 user: lexi tags: trunk
Changes

Modified cli.lua from [ad6ab18d31] to [2aa6072a5d].

            1  +-- [ʞ] cli.lua
            2  +--  ~ lexi hale <lexi@hale.su>
            3  +--  🄯 AGPLv3
            4  +--  ? simple command line driver for the cortav library
     1      5   local ct = require 'cortav'
     2      6   local ss = require 'sirsem'
     3      7   
     4      8   local default_mode = {
     5      9   	['render:format'] = 'html';
     6     10   	['html:gen-styles'] = true;
           11  +	['groff:color'] = true;
     7     12   }
     8     13   
     9     14   local function
    10     15   main(input, output, log, mode, suggestions, vars, extrule)
    11     16   	local doc = ct.parse(input.stream, input.src, mode, function(c)
    12     17   		                     c.doc.ext = extrule
    13     18   	                     end)
................................................................................
    33     38   	local render_opts = ss.kmap(function(k,v)
    34     39   		return k:sub(2+#mode['render:format'])
    35     40   	end, ss.kfilter(mode, function(m)
    36     41   		return ss.str.begins(m, mode['render:format']..':')
    37     42   	end))
    38     43   
    39     44   	doc.vars = vars
    40         -	
    41         -	-- this is kind of gross but the context object belongs to the parser,
    42         -	-- not the renderer, so that's not a suitable place for this information
    43         -	doc.stage = {
    44         -		kind = 'render';
    45         -		format = mode['render:format'];
    46         -		mode = mode;
    47         -		suggestions = suggestions;
    48         -	}
    49     45   
    50         -	output:write(ct.render[mode['render:format']](doc, render_opts))
           46  +	output:write(ct.render[mode['render:format']](
           47  +		doc, render_opts, function(stage)
           48  +			stage.mode = mode
           49  +		end))
    51     50   	return 0
    52     51   end
    53     52   
    54     53   local inp,outp,log = io.stdin, io.stdout, io.stderr
    55     54   
    56     55   local function entry_cli()
    57     56   	local suggestions, vars, input = default_mode, {}, {

Modified cortav.ct from [4ed3bc7476] to [7b2c420807].

    74     74   * [*paragraphs] ([`.] [` ¶] [`❡]): a paragraph is a simple block of text. the period control sequence is only necessary if the paragraph text starts with text that would be interpreted as a control sequence otherwise
    75     75   * newlines [` \\]: inserts a line break into previous paragraph and attaches the following text. mostly useful for poetry or lyrics
    76     76   * [*section starts] [`#] [`§]: starts a new section. all sections have an associated depth, determined by the number of sequence repetitions (e.g. "###" indicates depth-three"). sections may have headers and IDs; both are optional. IDs, if present, are a sequence of raw-text immediately following the hash marks. if the line has one or more space character followed by styled-text, a header will be attached. the character immediately following the hashes can specify a particular type of section. e.g.:
    77     77   ** [`#] is a simple section break.
    78     78   ** [`#anchor] opens a new section with the ID [`anchor].
    79     79   ** [`# header] opens a new section with the title "header".
    80     80   ** [`#anchor header] opens a new section with both the ID [`anchor] and the title "header".
    81         -** [`#>conversation] opens a blockquote section named [`conversation] without a header.
    82         -** [`#&id mime] opens a new inline object [`id] of type [`mime]. useful for embedding SVGs. the ID and mime type must be specified.
    83     81   * [*nonprinting sections] ([`^]): sometimes, you'll want to create a namespace without actually adding a visible new section to the document. you can achieve this by creating a [!nonprinting section] and defining resources within it. nonprinting sections can also be used to store comments, notes, or other information that is useful to have in the source file without it becoming a part of the output
    84     82   * [*resource] ([`@]): defines a [!resource]. a resource is an file or object that exists outside of the document but which will be included in the document somehow. common examples of resources include images, videos, iframes, or headers/footers. see [>rsrc resources] for more information.
    85     83   * [*lists] ([`*] [`:]): these are like paragraph nodes, but list nodes that occur next to each other will be arranged so as to show they compose a sequence. depth is determined by the number of stars/colons. like headers, a list entry may have an ID that can be used to refer back to it; it is indicated in the same way. if colons are used, this indicates that the order of the items is signifiant. :-lists and *-lists may be intermixed; however, note than only the last character in the sequence actually controls the depth type.
    86     84   * [*directives] ([`%]): a directive issues a hint to the renderer in the form of an arbitrary string. directives are normally ignored if they are not supported, but you may cause a warning to be emitted where the directive is not supported with [`%!] or mark a directive critical with [`%!!] so that rendering will entirely fail if it cannot be parsed.
    87     85   * [*comments] ([`%%]): a comment is a line of text that is simply ignored by the renderer.
    88     86   * [*asides] ([`!]): indicates text that diverges from the narrative, and can be skipped without interrupting it. think of it like block-level parentheses. asides which follow one another are merged as paragraphs of the same aside, usually represented as a sort of box. if the first line of an aside contains a colon, the stretch of styled-text from the beginning to the aside to the colon will be treated as a "type heading," e.g. "Warning:"
    89     87   * [*code] ([`~~~]): a line beginning with ~~~ begins or terminates a block of code. code blocks are by default not parsed, but parsing can be activated by preceding the code block with an [`%[*expand]] directive. the opening line should look like one of the below
................................................................................
   126    124   * strikeout {obj ~|styled-text}: indicates that its text should be struck through or otherwise indicated for deletion
   127    125   * insertion {obj +|styled-text}: indicates that its text should be indicated as a new addition to the text body.
   128    126   ** consider using a macro definition [`\edit: [~[#1]][+[#2]]] to save typing if you are doing editing work
   129    127   * link \[>[!ref] [!styled-text]\]: produces a hyperlink or cross-reference denoted by [$ref], which may be either a URL specified with a reference or the name of an object like an image or section elsewhere in the document. the unicode characters [`→] and [`🔗] can also be used instead of [`>] to denote a link.
   130    128   * footnote {span ^|ref|[$styled-text]}: annotates the text with a defined footnote. in interactive output media [`\[^citations.qtheo Quantum Theosophy: A Neophyte's Catechism]] will insert a link with the next [`Quantum Theosophy: A Neophyte's Catechism] that, when clicked, causes a footnote to pop up on the screen. for static output media, the text will simply have a superscript integer after it denoting where the footnote is to be found.
   131    129   * superscript {obj '|[$styled-text]}
   132    130   * subscript {obj ,|[$styled-text]}
   133         -* raw {obj \\ |[$raw-text]}: causes all characters within to be interpreted literally, without expansion. the only special characters are square brackets, which must have a matching closing bracket
          131  +* raw {obj \\ |[$raw-text]}: causes all characters within to be interpreted literally, without expansion. the only special characters are square brackets, which must have a matching closing bracket, and backslashes.
   134    132   * raw literal \[$\\[!raw-text]\]: shorthand for [\[$[\…]]]
   135    133   * macro [`\{[!name] [!arguments]\}]: invokes a [>ex.mac macro], specified with a reference
   136    134   * argument {obj #|var}: in macros only, inserts the [$var]-th argument. otherwise, inserts a context variable provided by the renderer.
   137    135   * raw argument {obj ##|var}: like above, but does not evaluate [$var].
   138    136   * term {obj &|name}, {span &|name|[$expansion]}: quotes a defined term with a link to its definition, optionally with a custom expansion of the term (for instance, to expand the first use of an acronym)
   139    137   * inline image {obj &@|name}: shows a small image or other object inline. the unicode character [`🖼] can also be used instead of [`&@].
   140    138   * unicode codepoint {obj U+|hex-integer}: inserts an arbitrary UCS codepoint in the output, specified by [$hex-integer]. lowercase [`u] is also legal.
................................................................................
   196    194   <p>here is the resource in span context: <span class="res-smiley"></span></p>
   197    195   <p>and here it is in block context:</p>
   198    196   <div class=".res-smiley"></div>
   199    197   ~~~
   200    198   
   201    199   note that empty elements with CSS classes are used in the output, to avoid repeating long image definitions (especially base64 inline encoded ones!)
   202    200   
          201  +inline resources are defined a bit differently:
          202  +
          203  +~~~cortav
          204  +@smiling-man-business-card text/plain {
          205  +	THE SMILING MAN  | tel. 0-Ω00-666█
          206  +	if you can read this | email: nameless@smiles.gov
          207  +	it is already too late | address: right behind you
          208  +}
          209  +@smiling-man-business-card image/png;base64 {
          210  +	%% incomprehensible gibbering redacted
          211  +}
          212  +~~~
          213  +
          214  +for an inline resource, the identifier is followed by a MIME type and an opening bracket. the opening bracket may be any of the characters [`\{][`\[][`(][`<], and can optionally be followed by additional characters to help disambiguate the closing bracket. the closing bracket is determined by "flipping" the opening bracket, producing bracket pairs like the following:
          215  +* [`\{:][`:}]
          216  +* [`<!--] [`--!>]
          217  +* [`(*<][`>*)]
          218  +* [`<>][`<>] [!(disables nesting!)]
          219  +if the open and closing brackets are distinguishable, they will nest appropriately, meaning that [`{][`}] alone is very likely to be a safe choice to escape a syntactically correct C program (that doesn't abuse macros too badly). brackets are searched for during parsing; encoded resources are not decoded until a later stage, so a closing bracket character in a base64-encoded text file cannot break out of its escaping.
          220  +
          221  +as a convenience, if the first line of the resource definition begins with a single tab, one tab will be dropped from every following line in order to allow legible indentation. similarly, if an opening bracket is followed immediately by a newline, this newline is discarded.
          222  +
          223  +text within a resource definition body is not expanded unless the resource definition is preceded with an [`%[*expand]] directive. if an expand directive is found, the MIME type will be used to try and determine an appropriate type of formatting, potentially invoking a separate renderer. for example, [`text/html] will invoke the [`html] backend, and [`application/x-troff] will invoke the [`groff] backend. if no suitable renderer is available, expansions will generate only plain text.
          224  +
          225  +two suffixes are accepted: [`;base64] and [`;hex]. the former will decode the presented strings using the base64 algorithm to obtain the resources data; the second will ignore all characters but ASCII hexadecimal digits and derive the resource data byte-by-byte by reading in hexadecimal pairs. for instance, the following sections are equivalent:
          226  +
          227  +~~~
          228  +@propaganda text/plain {
          229  +	WORLDGOV SAYS
          230  +	“don't waste time with unproductive thoughts
          231  +	 your wages will be docked accordingly”
          232  +}
          233  +~~~
          234  +~~~
          235  +@propaganda text/plain;hex {
          236  +	574f 524c 4447 4f56 2053 4159 530a e280 9c64 6f6e 2774 2077 6173
          237  +	7465 2074 696d 6520 7769 7468 2075 6e70 726f 6475 6374 6976 6520
          238  +	7468 6f75 6768 7473 0a20 796f 7572 2077 6167 6573 2077 696c 6c20
          239  +	6265 2064 6f63 6b65 6420 6163 636f 7264 696e 676c 79e2 809d 0a
          240  +}
          241  +~~~
          242  +~~~
          243  +@propaganda text/plain;base64 {
          244  +	V09STERHT1YgU0FZUwrigJxkb24ndCB3YXN0ZSB0aW1lIHdpdGggdW5wcm9kdWN0aXZlIHRob3Vn
          245  +	aHRzCiB5b3VyIHdhZ2VzIHdpbGwgYmUgZG9ja2VkIGFjY29yZGluZ2x54oCdCg==
          246  +}
          247  +~~~
          248  +
          249  +inline resources can also be (ab)used for multiline macros:
          250  +~~~
          251  +@def text/x-cortav {
          252  +	* [*[#1]] [!([#2])
          253  +	*: [#3]
          254  +}
          255  +&def nuclear bunker|n|that which will not protect you from the Smiling Man
          256  +~~~
          257  +to make this usage simpler, resources with a type of [`text/x-cortav] can omit the MIME type field.
          258  +
   203    259   ### supported parameters
   204    260   * [`src] (all): specifies where to find the file, what it is, and how to embed it. each line of [`src] should consist of three whitespace-separated words: embed method, MIME type, and URI.
   205    261   ** embed methods
   206    262   *** [`local]: loads the resource at build time and embeds it into the output file. not all implementations may allow loading remote network resources at build time.
   207    263   *** [`remote]: only embeds a reference to the location of the resource. use this for e.g. live iframes, dynamic images, or images hosted by a CDN.
   208    264   *** [`auto]: embeds a reference in file formats where that's practical, and use a remote reference otherwise.
   209    265   ** MIME types: which file types are supported depends on the individual implementation and renderer backend; additionally, extensions can add support for extra types. MIME-types that have no available handler will, where possible, result in an attachment that can be extracted by the user, usually by clicking on a link. however, the following should be usable with all compliant implementations
................................................................................
   211    267   *** [`video/*] (interactive outputs only)
   212    268   *** [`image/svg+xml] is handled specially for HTML files, and may or may not be compatible with other renderer backends.
   213    269   *** [`font/*] can be used with the HTML backend to reference a web font
   214    270   *** [`font/woff2] can be used with the HTML backend to reference a web font
   215    271   *** [`text/plain] (will be inserted as a preformatted text block)
   216    272   *** [`text/css] (can be used when producing HTML files to link in an extra stylesheet, either by embedding it or referencing it from the header)
   217    273   *** [`text/x-cortav] (will be parsed and inserted as a formatted text block; context variables can be passed to the file with [`ctx.[$var]] parameters)
          274  +*** [`application/x-troff] can be used to supply sections of text written in raw [`groff] syntax. these are ignored by other renderers.
          275  +*** [`text/html] can be used to supply sections of text written in raw HTML.
   218    276   *** any MIME-type that matches the type of file being generated by the renderer can be used to include a block of data that will be passed directly to the renderer.
   219    277   ** URI types: additional URI types can be added by extensions or different implementations, but every compliant implementation must support these URIs.
   220    278   *** [`http], [`https]: accesses resources over HTTP. add a [`file] fallback if possible for the benefit of renderers/viewers that do not have internet access abilities.
   221    279   *** [`file]: references local files. absolute paths should begin [`file:/]; the slash should be omitted for relative paths. note that this doesn't have quite the same meaning as in HTML -- [`file] can (and usually should be) used with HTML outputs to refer to resources that reside on the same server. a cortav URI of [`file:/etc/passwd] will actually result in the link [`/etc/passwd], not [`file:///etc/passwd] when converted to HTML. generally, you only should use [`http] when you're referring to a resource that exists on a different domain.
   222    280   *** [`name]: a special URI used generally for referencing resources that are already installed on a target system and do not need to be embedded or linked, the name and type are enough for a renderer on another machine to locate the correct resource. this is useful mostly for [>fonts fonts], where it's more typical to refer to fonts that are installed on your system rather than providing paths to font files.
   223    281   *** [`gemini]: accesses resources over the gemini protocol. currently you should really only use this for [`local] resources unless you're using the gemtext renderer backend, since nothing but gemini browsers are liable to support this protocol.
   224    282   * [`desc]: supplies a narrative description of the resources, for use as an "alt-text" when the image cannot be loaded and for screenreaders.
   225         -* [`detail]: supplies extra narrative commentary that is displayed contextually, e.g. when the user hovers her mouse cursor over the embedded object.
          283  +* [`detail]: supplies extra narrative commentary that is displayed contextually, e.g. when the user hovers her mouse cursor over the embedded object. also used for [`desc] if [`desc] is not supplied.
   226    284   
   227         -note that in certain cases, full MIME types do not need to be used. say you're defining a font with the [`name] URI -- you can't necessary know what file type the system fonts on another computer are going to be. in this case, you can just write [`font] instead of [`font/ttf] or [`font/woff2] or similar. all cortav needs to know in this case is what abstract kind of object you're referencing.
          285  +note that in certain cases, full MIME types do not need to be used. say you're defining a font with the [`name] URI -- you can't necessary know what file type the system fonts on another computer are going to be. in this case, you can just write [`font] instead of [`font/ttf] or [`font/woff2] or similar. all cortav needs to know in this case is what abstract kind of object you're referencing. [`groff] fonts (referenced with the [`dit] URI) don't have a specific MIME type either.
   228    286   
   229    287   
   230    288   ##ctxvar context variables
   231    289   context variables are provided so that cortav renderers can process templates. certain context variables are provided for by the standard. you can test for the presence of a context variable with the directive [`%[*when] ctx [$var]]. context variables are accessed with the [` \[#[$name]\]] span.
   232    290   
   233    291   * {def cortav.file} the name of the file currently being rendered
   234    292   * {def cortav.path} the absolute path of the file currently being rendered
................................................................................
   282    340   	font-family: "fontdef-sans";
   283    341   	src: local("Alegreya Sans"),
   284    342   		local("Open Sans"),
   285    343   		local("sans-serif");
   286    344   }
   287    345   ~~~
   288    346   
   289         -there are two things that aren't super clear from the CSS, however. notice how we used [`auto] on a couple of those specs? this means it's up to the renderer to decide whether to link or embed the font. for html, a font specified by name can't really be embedded, but for some file formats, it can be. [`auto] lets us produce valid HTML while still taking advantage of font embedding in other formats.
          347  +there are two things that aren't super clear from the CSS, however. notice how we used [`auto] on a couple of those specs? this means it's up to the renderer to decide whether to link or embed the font. in HTML, a font specified by name can't really be embedded, but for some file formats, it can be. [`auto] lets us produce valid HTML while still taking advantage of font embedding in other formats.
   290    348   
   291    349   now that we have our font families defined, we can use their identifiers with the [`%[*font]] directive to control the font stack. the first thing we need to do is push a new font context. there's two ways we can do this:
   292    350   	fnd: [`%[*font] [#1]]
   293    351   * {fnd dup} will create a copy of the current font context, allowing us to make some changes and then revert later with the {fnd pop} command. this isn't useful in our case, however, because right now the stack is empty; there's nothing to duplicate.
   294    352   * {fnd new} will create a brand new empty context for us to work with and push it to the stack. this can also be used to temporarily revert to the system default fonts, and then switch back with {fnd pop}.
   295    353   * {fnd set} changes one or more entries in the current font context. it can take a space-separated list of arguments in the form [`[$entry]=[$font-id]]. the supported entries are:
   296    354   ** [`body]: the fallback font. if only this is set in a given font context, it will be used for everything
................................................................................
   309    367   ~~~cortav
   310    368   %% let's pretend we've also defined the fonts 'title', 'cursive', and 'thin'
   311    369   
   312    370   %font new
   313    371   %font set body=sans header=serif
   314    372   %font dup
   315    373   %font header=title
   316         -# lorem ipsum dolor
          374  +# WorldGov  announcement
   317    375   %font pop
   318    376   
   319    377   %% we've now set up a default font context, created a new context for the title of the
   320    378   %% document, and then popped it back off after the title was inserted so that our
   321    379   %% first font context is active again. everything after that last '%font pop' will
   322    380   %% be printed in sans, except for headers, which will be printed in 'serif'
   323    381   
   324         -lorem ipsum dolor sit amet, sed consectetur apiscing elit…
          382  +WorldGov would like to congratulate 2274's Employee of the Year, [*The Smiling Man]! The Smiling Man had a few words of encouragement for the weary proles of the world when he graciously accepted his award at this year's ceremonial bloodletting:
   325    383   
   326    384   %font dup
   327    385   %font set body=cursive
   328         -> sed do eiusmod tempor incididunt ut labore et dolore magna aliqua.
   329         -> Ut enim ad minim veniam, quis nostrud exercitation ullamco laboris nisi ut
          386  +> It is very important for you to understand that your dreams are the intellectual property of the WorldGov organization.
          387  +> Laborers who fail more than one duplicity check per workcycle will receive extra Pit Time.
   330    388   %font pop
   331    389   
   332    390   %% above we created a blockquote whose text is printed in a cursive font; afterwards,
   333         -%% we simply remove this new context—
          391  +%% we simply remove this new context, and everything is back the way it was at "WorldGov would like"
   334    392   
   335         -and everything is back the way it was at "lorem ipsum"
          393  +In addition to his 227th consecutive Employee of the Year Award, The Smiling Man has been nominated for a WorldGov Lifetime Achievement Award by the Hyperion Entity in recognition of his exceptional leadership in the Department Which Has No Name. Chief Ritual Officer Mr. Winthrop had this to say:
   336    394   
   337    395   %% the font mechanism is at its most powerful when used with multiline macros:
   338    396   
   339    397   	cursive-quote: %font dup
   340    398   		%font set body=cursive
   341    399   		> [#1]
   342    400   		%font pop
   343    401   
   344    402   %% now, whenever we want a block with a cursive body, we can simply invoke
   345    403   
   346         -&$cursive-quote Duis aute irure dolor in reprehenderit in voluptate velit esse cillum dolore eu fugiat nulla pariatur. Excepteur sint occaecat cupidatat non proident
          404  +&$cursive-quote A sea of blood yet lies between us and the Destination. It won't impede me. And I'm so very proud to say that, apparently, it won't impede the Smiling Man either, if the Svalbard contract was any indication! [pause for laughter]
   347    405   
   348    406   %% without affecting the overall font context. in fact, since 'cursive-quote' creates
   349    407   %% its context using 'dup', it would import all font specifications besides 'body'
   350    408   %% from the environment it is invoked in
   351    409   ~~~
   352    410   
   353    411   you may have noticed the rather odd bit at the end of our font definition, with the [`dit] URI. the reasons for this are tragic. groff, while delightful, has a thoroughly antiquated understanding of fonts, and doesn't support normal font formats like truetype. groff ships with a limited number of fonts in its own format, identified by obscurantist letter code ([`HBI] is "Helvetica Bold Italic", for instance) and lacking normal metadata. for this reason, you'll have to tell cortav how you want your fonts translated.
................................................................................
   686    744   + encoding-data-ucs-url | where to download UnicodeData.txt from, if encoding-data-ucs is not changed. defaults to the unicode consortium website
   687    745   
   688    746   #### deterministic builds
   689    747   some operating systems, like NixOS, require packages that can be built in reproducible ways. this implies that all data, all [!state] that goes into producing a package needs to be accounted for before the build proper begins. the [`cortav] build process needs to be slightly altered to support such a build process.
   690    748   
   691    749   while the cortav specification itself does not concern itself with matters like whether a particular character is a numeral or a letter, optimal typesetting in some cases requires such information. this is the case for the equation span- and block-types, which need to be able to distinguish between literals, variables, and mathematical symbols in [^alas-math the equations they format]. the ASCII charset is small enough that exhaustive character class information can be manually hardcoded into a cortav implementation, the various encodings of Unicode most certainly are not.
   692    750   
   693         -	alas-math: sadly, i was not at any point consulted by any of the generations of mathematicians stretching back into antiquity who devised their notations without any regard for machine-readability. [!for shame!]
          751  +	alas-math: sadly, i was not at any point consulted by any of the generations of mathematicians stretching back into antiquity, who as a direct consequence devised their notations without [*any] regard for machine-readability. [!for shame!]
   694    752   
   695    753   for this reason, the reference implementation of cortav embeds the file [`UnicodeData.txt], a database maintained by the Unicode Consortium. this is a rather large file that updates for each new Unicode version, so it is downloaded as part of the build process. to build on NixOS, you'll need to either disable the features that rely on this database (not recommended), or download the database yourself and tell the build script where to find it. this is the approach the official nix expression will take when i can be bothered to write it. see the examples below for how to conduct a deterministic build
   696    754   
   697    755   ~~~ deterministic build with unicode database [sh] ~~~
   698    756   /src $ mkdir cortav && cd cortav
   699    757   /src/cortav $ fossil clone https://c.hale.su/cortav .fossil && fossil open .fossil
   700    758   /src/cortav $ curl https://www.unicode.org/Public/UCD/latest/ucd/UnicodeData.txt > /tmp/unicode.txt
................................................................................
   703    761   ~~~ [sh] deterministic build [!without] unicode database ~~~
   704    762   /src $ mkdir cortav && cd cortav
   705    763   /src/cortav $ fossil clone https://c.hale.su/cortav .fossil && fossil open .fossil
   706    764   /src/cortav $ make build/cortav encoding-data=
   707    765   ~~~
   708    766   
   709    767   ! while most of the data used is taken directly from UnicodeData.txt, the database generated by [`tools/ucs.lua] splices in some extra character information before generating a database. this is partly because certain characters may not be classified in a useful way and need to be manually overwritten. however, the reference implementation also seeks to provide accurate data for certain character sets that are not part of unicode proper and can be expressed in UTF only through its private use areas.
   710         -! currently, only the [>corran Corran] script is currently supported in this fashion, but i intend to add [>tengwar Tengwar] as well. if there is a con-script or any other informally encoded script you would like supported by the reference implementation, please open an issue.
          768  +! currently, only the [>corran Corran] script is supported in this fashion, but i intend to add [>tengwar Tengwar] as well. if there is a con-script or any other informally encoded script you would like supported by the reference implementation, please open an issue.
   711    769   
   712    770   [*do note] that no cortav implementation needs to concern itself with character class data. this functionality is provided in the reference implementation strictly as an (optional) extension to the spec to improve usability, not as a normative requirement.
   713    771   
   714    772   	corran: http://ʞ.cc/fic/spirals/society
   715    773   	tengwar: https://en.wikipedia.org/wiki/Tengwar
   716    774   
   717    775   ###refimpl-switches switches
................................................................................
   782    840   * [`@[*fg]]: resolves to a color expression denoting the selected foreground color. equivalent to [`[*tone](1)]
   783    841   * [`@[*bg]]: resolves to a color expression denoting the selected background color. equivalent to [`[*tone](0)]
   784    842   * [`@[*tone]\[/[$alpha]\]([$fac] \[[$shift] \[[$saturate]\]\] )]: resolves to a color expression. [$fac] is a floating-point value scaling from the background color to the foreground color. [$shift] is a value in degrees controlling how far the hue will shift relative to the accent. [$saturate] is a floating-point value controlling how satured the color is.
   785    843   
   786    844   ###refimpl-rend-groff groff
   787    845   the [`groff] backend produces a text file suitable for supplying to a [`groff] compiler. [`groff] is the GNU implementation of a venerable typesetting system from the early days of UNIX
   788    846   
   789         -as a convenience, the groff backend supports two modes of operation: it can write a [`groff] file directly to disk, or it can automatically launch a [`groff] process with the appropriate command line options and environment variables. this second mode is recommended unless you're rendering very large files to multiple formats, as [`groff] invocation is nontrivial and it's best to let the renderer handle that for you.
          847  +you can produce a final output directly by piping from the [`cortav] driver into [`groff]. if your document uses an encoding other than ASCII, you'll need to notify [`groff] of this with the [`-K] flag. for example, to render a UTF8 cortav file to PDF:
          848  +
          849  +~~~
          850  +$ cortav input.ct -m render:format groff | groff -Tpdf -Kutf8 > output.pdf
          851  +~~~
          852  +
          853  +in the future, it is planned to enable the driver to operate groff automatically and directly produce the desired output format when the binary wrapper is in use. doing so securely and hygienically is not possible in pure lua, however.
   790    854   
   791    855   ####refimpl-rend-groff-modes modes
   792    856   [`groff] supports the following modes:
   793    857   
   794    858   * string [`groff:annotate] controls how footnotes will be handled.
   795    859   ** [`footnote] places footnotes at the end of the page they are referenced on. if the same footnote is used on multiple pages, it will be duplicated on each.
   796    860   ** [`secnote] places footnotes at the end of each section. footnotes used in multiple sections will be duplicated for each
   797    861   ** [`endnote] places all footnotes at the end of the rendered document.
   798         -* string [`groff:dev] names an output device (such as [`dvi] or [`pdf]). if this mode is present, [`groff] will be automatically invoked
   799    862   * string [`groff:title-page] takes an identifier that names a section. this section will be treated as the title page for the document.
          863  +* string [`groff:title] sets a specific title to be used in headers instead of relying on header heuristics
   800    864   
   801    865   ### directives
   802    866   * [`%[*pragma] title-page [$id]] sets the title page to section [$id]. this causes it to be specially formatted, with a large, centered title and subtitle.
   803    867   
   804    868   ### quirks
   805    869   if the [`toc] extension is active but [`%[*toc]] directive is provided, the table of contents will be given its own section at the start of the document (after the title page, if any).
   806    870   

Modified cortav.lua from [5feb0b86b1] to [bf3719fe9a].

    83     83   		return string.format("mode “%s” "..msg, ...)
    84     84   	end);
    85     85   	unimpl = ss.exnkind 'feature not implemented';
    86     86   	ext = ss.exnkind 'extension error';
    87     87   	enc = ss.exnkind('encoding error', function(msg, ...)
    88     88   		return string.format('[%s]' .. msg, ...)
    89     89   	end);
           90  +	rdr = ss.exnkind('could not render', function(msg, ...)
           91  +		return string.format('(backend %s)'..msg, ...)
           92  +	end);
    90     93   }
    91     94   
    92     95   ct.ctx = declare {
    93     96   	mk = function(src) return {src = src} end;
    94     97   	ident = 'context';
    95     98   	cast = {
    96     99   		string = function(me)
................................................................................
   141    144   		depth = 0;
   142    145   		kind = 'ordinary';
   143    146   	} end;
   144    147   	construct = function(self, id, depth)
   145    148   		self.id = id
   146    149   		self.depth = depth
   147    150   	end;
          151  +	fns = {
          152  +		visible = function(self)
          153  +			if self.kind == 'nonprinting' then return false end
          154  +			local invisibles = {
          155  +				['break'] = true;
          156  +				reference = true;
          157  +				resource = true;
          158  +				directive = true;
          159  +			}
          160  +			for k,b in pairs(self.blocks) do
          161  +				if not (invisibles[b.kind] or b.invisible) then return true end
          162  +				-- extensions that add invisible nodes to the AST must
          163  +				-- mark them as such for rendering to work properly!
          164  +			end
          165  +			return false
          166  +		end;
          167  +	}
   148    168   }
   149    169   
   150    170   ct.doc = declare {
   151    171   	ident = 'doc';
   152    172   	fns = {
   153    173   		mksec = function(self, id, depth)
   154    174   			local o = ct.sec(id, depth)
................................................................................
   749    769   				if ss.str.begins(substr, i.seq) then
   750    770   					found = true
   751    771   					table.insert(spans, i.parse(substr:sub(1+#i.seq), ctx))
   752    772   					break
   753    773   				end
   754    774   			end
   755    775   			if not found then
   756         -				ctx:fail('no recognized control sequence in [%s]', substr)
          776  +				buf = buf .. c
   757    777   			end
   758    778   		elseif c == '\n' then
   759    779   			flush()
   760    780   			table.insert(spans,{kind='line-break',origin=ctx:clone()})
   761    781   		else
   762    782   			buf = buf .. c
   763    783   		end
................................................................................
   986   1006   		if (not last) or (last.kind ~= 'reference') then
   987   1007   			c:fail('reference continuations must immediately follow a reference')
   988   1008   		end
   989   1009   		local str = l:match '^\t\t(.-)%s*$'
   990   1010   		last.val = last.val .. '\n' .. str
   991   1011   		c.sec.refs[last.key] = last.val
   992   1012   	end};
   993         -	{seq = '\t', fn = blockwrap(function(l,c,j,d)
         1013  +	{seq = '\t', pred = function(l)
         1014  +		return (l:match '\t+([^:]+):%s*(.*)$')
         1015  +	end; fn = blockwrap(function(l,c,j,d)
   994   1016   		local ref, val = l:match '\t+([^:]+):%s*(.*)$'
   995   1017   		local last = d[#d]
   996   1018   		local rsrc
   997   1019   		if last and last.kind == 'resource' then
   998   1020   			last.props[ref] = val
   999   1021   			rsrc = last
  1000   1022   		elseif last and last.kind == 'reference' and last.rsrc then
................................................................................
  1185   1207   			end
  1186   1208   		end
  1187   1209   	end
  1188   1210   	job:hook('line_end',ctx,l)
  1189   1211   end
  1190   1212   
  1191   1213   function ct.parse(file, src, mode, setup)
  1192         -
         1214  +	-- this object is threaded down through the parse tree
         1215  +	-- and copied to store information like the origin of the
         1216  +	-- element in the source code
  1193   1217   	local ctx = ct.ctx.mk(src)
  1194   1218   	ctx.line = 0
  1195   1219   	ctx.doc = ct.doc.mk()
  1196   1220   	ctx.doc.src = src
  1197   1221   	ctx.sec = ctx.doc:mksec() -- toplevel section
  1198   1222   	ctx.sec.origin = ctx:clone()
  1199   1223   	ctx.lang = mode['meta:lang']
................................................................................
  1267   1291   			end
  1268   1292   		end
  1269   1293   	end
  1270   1294   	ctx.doc.stage = nil
  1271   1295   	ctx.doc.docjob:hook('meddle_ast')
  1272   1296   	return ctx.doc
  1273   1297   end
         1298  +
         1299  +function ct.expand_var(v)
         1300  +	local val
         1301  +	if v.pos then
         1302  +		if not v.origin.invocation then
         1303  +			v.origin:fail 'positional arguments can only be used in a macro invocation'
         1304  +		elseif not v.origin.invocation.args[v.pos] then
         1305  +			v.origin.invocation.origin:fail('macro invocation %s missing positional argument #%u', v.origin.invocation.macro, v.pos)
         1306  +		end
         1307  +		val = v.origin.invocation.args[v.pos]
         1308  +	else
         1309  +		val = v.origin.doc:context_var(v.var, v.origin)
         1310  +	end
         1311  +	if v.raw then
         1312  +		return val, true
         1313  +	else
         1314  +		return ct.parse_span(val, v.origin), false
         1315  +	end
         1316  +end

Modified render/groff.lua from [a43bfa19e3] to [ab8e0fd21b].

    18     18   end
    19     19   local lines = function(...)
    20     20   	local s = ss.strac()
    21     21   	for _, v in pairs{...} do s(v) end
    22     22   	return s
    23     23   end
    24     24   
    25         -function ct.render.groff(doc, opts)
           25  +local function gsan(str)
           26  +	local tocodepoint = function(ch)
           27  +		return string.format('\\[u%04X]', utf8.codepoint(ch))
           28  +	end
           29  +	str = str:gsub('(["\'\\])',tocodepoint)
           30  +	return str
           31  +end
           32  +
           33  +local gtxt = ss.declare {
           34  +	ident = 'groff-text';
           35  +	mk = function() return {
           36  +		lines = {};
           37  +	} end;
           38  +	fns = {
           39  +		raw = function(me, text)
           40  +			if me.linbuf == nil then
           41  +				me.linbuf = ss.strac()
           42  +			end
           43  +			me.linbuf(text)
           44  +		end;
           45  +		txt = function(me, str, ...)
           46  +			if str == nil then return end
           47  +			me:raw(gsan(str))
           48  +			-- WARN this will cause problems if str is ever allowed to
           49  +			-- include a line break. we can sanitize by converting
           50  +			-- every line break into a new entry in the table, but i
           51  +			-- don't think it should be possible for a \n to reach us
           52  +			-- at this point, so i'm omitting the safety check as it
           53  +			-- would involve an excessive hit to performance
           54  +			me:txt(...)
           55  +		end;
           56  +		brk = function(me)
           57  +			me:flush()
           58  +			table.insert(me.lines, '')
           59  +		end;
           60  +		line = function(me, ...)
           61  +			me:flush()
           62  +			me:txt(...)
           63  +		end;
           64  +		req = function(me, r)
           65  +			me:flush()
           66  +			table.insert(me.lines, '.'..r)
           67  +		end;
           68  +		esc = function(me, e)
           69  +			me:raw('\\' .. e)
           70  +		end;
           71  +		flush = function(me)
           72  +			if me.linbuf ~= nil then
           73  +				local line = me.linbuf:compile()
           74  +				local first = line:sub(1,1)
           75  +				-- make sure our lines aren't accidentally interpreted
           76  +				-- as groff requests. groff is kinda hostile to script
           77  +				-- generation, huh?
           78  +				if first == '.' or first == "'" then
           79  +					line = '\\&' ..line
           80  +				end
           81  +				table.insert(me.lines, line)
           82  +				me.linbuf = nil
           83  +			end
           84  +		end;
           85  +		compile = function(me)
           86  +			me:flush()
           87  +			return table.concat(me.lines, '\n')
           88  +		end;
           89  +	}
           90  +}
           91  +
           92  +local function mkColorDef(name, color)
           93  +	return '.defcolor '..name..' rgb ' ..
           94  +		table.concat({color:rgb_t()}, ' ', 1, 3)
           95  +end
           96  +
           97  +local function addAccentTones(rs,hue,spread)
           98  +	local base = ss.color(hue, 1, .5)
           99  +	local right = spread > 0 and ss.color(hue + spread, 1, .5)
          100  +		or ss.color(hue, 0.4, 0.6)
          101  +	local left = spread > 0 and ss.color(hue - spread, 1, .5)
          102  +		or ss.color(hue, 1, 0.3)
          103  +
          104  +	local steps = 6
          105  +	for i=-3,3 do
          106  +		local nc, nm
          107  +		local o if i > 0
          108  +			then o = right nm = 'R'
          109  +			else o = left  nm = 'L'
          110  +		end
          111  +		nc = base + o:alt('alpha', math.abs(i) / 3)
          112  +		rs.addColor('accent'..nm..tostring(math.abs(i)),nc)
          113  +	end
          114  +end
          115  +local function mkrc()
          116  +	return {
          117  +		clone = function(self, origin)
          118  +			return {
          119  +				origin = origin;
          120  +				clone = self.clone;
          121  +				prop = ss.clone(self.prop);
          122  +				mk = self.mk;
          123  +				add = self.add;
          124  +				block = self.block;
          125  +				blocks = self.blocks;
          126  +				span = self.span;
          127  +				spans = self.spans;
          128  +			}
          129  +		end;
          130  +		blocks = {};
          131  +		prop = {};
          132  +		block = function(self)
          133  +			local sub = self:clone()
          134  +			sub.spans = {}
          135  +			sub.blocks = nil
          136  +			sub.span = function(me, ln)
          137  +				local p = ss.clone(me.prop)
          138  +				p.txt = ln
          139  +				p.block = sub
          140  +				p.origin = me.origin
          141  +				table.insert(me.spans, p)
          142  +				return p
          143  +			end;
          144  +			table.insert(self.blocks, sub)
          145  +			return sub
          146  +		end;
          147  +	}
          148  +end
          149  +
          150  +function ct.render.groff(doc, opts, setup)
    26    151   	-- rs contains state specific to this render job
    27    152   	-- that modules will need access to
          153  +	local fail = function(msg, ...)
          154  +		ct.exns.rdr(msg, 'groff', ...):throw()
          155  +	end
    28    156   	local rs = {};
    29    157   	rs.macsets = {
    30    158   		strike = {
    31    159   			'.de ST';
    32    160   			[[.nr ww \w'\\$1']];
    33         -			[[\Z@\v'-.25m'\l'\\n[ww]u'@\\$1']];
          161  +			[[\Z@\v'-.25m'\l'\\n[ww]u'@\\$1]];
          162  +			'..';
          163  +		};
          164  +		color = {'.color'};
          165  +		insert = {};
          166  +		footnote = {
          167  +			'.de footnote-blank';
          168  +			'.  sp 0.25m';
          169  +			'..';
          170  +			'.ev footnote-env';
          171  +			'.  ps 8p';
          172  +			'.  in 0.5c';
          173  +			'.  blm footnote-blank';
          174  +			'.ev';
          175  +			'.de footnote-print';
          176  +-- 			'.  sp |\\\\n[.p]u-\\\\n[footnote-pos]u';
          177  +			'.  sp 0.5c';
          178  +			'.  ev footnote-env';
          179  +			'.    fn';
          180  +			'.  ev';
          181  +			'.  rm fn';
          182  +			'.  nr footnote-pos 0';
          183  +			-- move the trap past the bottom of the page so it's not
          184  +			-- invoked again until more footnotes have been assembled
          185  +			'.  ch footnote-print |\\\\n[.p]u+10';
          186  +			'.  bp';
          187  +			'..';
          188  +			'.wh |\\n[.p]u footnote-print';
          189  +		};
          190  +		root = {
          191  +		-- these are macros included in all documents
          192  +		-- page offset is hideously broken and unusable; we
          193  +		-- zero it out so we can use .in to control indents
          194  +		-- instead. note that the upshot of this is we need
          195  +		-- to manually specify the indent in every other
          196  +		-- environment from now on, .evc doesn't seem to cut it
          197  +		-- set up the page title environment & trap
          198  +			"'in 2c";
          199  +			"'ll 18c";
          200  +			"'po 0";
          201  +			"'ps 13p";
          202  +			"'vs 15p";
          203  +			".ev pgti";
          204  +			".  evc 0";
          205  +			".  fam H";
          206  +			".  ps 10pt";
          207  +			".ev";
          208  +			'.de ph';
          209  +			'.  sp 0.6c';
          210  +			'.  ev pgti';
          211  +			'.  po 1c';
          212  +			'.  lt 19c';
          213  +			".  tl '\\\\*[doctitle]'\\fB\\\\*[title]\\f[]'%'";
          214  +			'.  po 0';
          215  +			".  br";
          216  +			'.  ev';
          217  +			'.  sp 1.2c';
          218  +			'..';
          219  +			'.wh 0 ph';
          220  +			'.de np';
          221  +			'.  sp 0.2c';
    34    222   			'..';
          223  +			'.blm np'
          224  +
    35    225   		};
    36    226   	}
    37    227   	rs.macsNeeded = {
    38    228   		order = {};
          229  +		map = {};
    39    230   		count = 0;
          231  +		deps = {
          232  +			insert = {'color'};
          233  +			strike = {'color'};
          234  +		};
    40    235   	}
          236  +	rs.linkctr = 0
          237  +
    41    238   	function rs.macAdd(id)
    42         -		if rs.macsets[id] then
    43         -			rs.macsNeeded.count = macsNeeded.count + 1
          239  +		if rs.macsets[id] and not rs.macsNeeded.map[id] then
          240  +			rs.macsNeeded.count = rs.macsNeeded.count + 1
    44    241   			rs.macsNeeded.order[rs.macsNeeded.count] = id
          242  +			rs.macsNeeded.map[id] = true
          243  +			if not rs.macsNeeded.deps[id] then
          244  +				return true
          245  +			end
          246  +
          247  +			for k,v in pairs(rs.macsNeeded.deps[id]) do
          248  +				if not rs.macsNeeded.map[v] then
          249  +					rs.macAdd(v)
          250  +				end
          251  +			end
          252  +
    45    253   			return true
    46    254   		else return false end
    47    255   	end
          256  +
          257  +	rs.macAdd 'root'
          258  +
          259  +	rs.colors = {}
          260  +	rs.addColor = function(name,color)
          261  +		if not ss.color.is(color) then
          262  +			ss.bug('%s is not a color value', color):throw()
          263  +		end
          264  +		rs.colors[name] = color
          265  +	end
          266  +
          267  +	if opts.accent then
          268  +		addAccentTones(rs, tonumber(opts.accent), tonumber(opts['hue-spread']) or 0)
          269  +		rs.addColor('new', rs.colors.accentR3)
          270  +		rs.addColor('del', rs.colors.accentL3)
          271  +	else
          272  +		rs.addColor('new', ss.color(80, 1, .3))
          273  +		rs.addColor('del', ss.color(0, 1, .3))
          274  +	end
          275  +
          276  +	doc.stage = {
          277  +		type = 'render';
          278  +		format = 'groff';
          279  +		groff_render_state = rs;
          280  +	}
          281  +
          282  +	setup(doc.stage)
    48    283   	local job = doc:job('render_groff',nil,rs)
          284  +
          285  +	local function collect(rc, spans, b, s)
          286  +		local rcc = rc:clone()
          287  +		rcc.spans = {}
          288  +		rs.renderSpans(rcc, spans, b, s)
          289  +		return rcc.spans
          290  +	end
          291  +	local function collectText(...)
          292  +		local text = collect(...)
          293  +		local s = ss.strac()
          294  +		for i, l in ipairs(text) do
          295  +			s(l.txt)
          296  +		end
          297  +		return s
          298  +	end
          299  +
    49    300   
    50    301   	-- the way this module works is we build up a table for each block
    51    302   	-- of individual strings paired with attributes that say how they
    52    303   	-- should be rendered. we then iterate over the table, applying
    53    304   	-- formats as need be, and inserting blanks after each block
          305  +
          306  +
    54    307   
    55    308   	local spanRenderers = {}
    56    309   	function spanRenderers.format(rc, s, b, sec)
    57    310   		local rcc = rc:clone()
    58    311   		if s.style == 'strong' then
    59    312   			rcc.prop.bold = true
    60    313   		elseif s.style == 'emph' then
    61    314   			rcc.prop.emph = true
    62    315   		elseif s.style == 'strike' then
    63    316   			rcc.prop.strike = true
    64    317   			rs.macAdd 'strike'
          318  +			rcc.prop.color = 'del'
    65    319   		elseif s.style == 'insert' then
          320  +			rs.macAdd 'insert'
          321  +			rcc.prop.color = 'new'
    66    322   		end
    67    323   		rs.renderSpans(rcc, s.spans, b, sec)
    68    324   	end;
          325  +
          326  +	function spanRenderers.link(rc, l, b, sec)
          327  +		rs.renderSpans(rc, l.spans, b, sec)
          328  +		rs.linkctr = rs.linkctr + 1
          329  +		rs.macAdd 'footnote'
          330  +		local p = rc:span(string.format('[%u]', rs.linkctr))
          331  +		if type(l.ref) == 'string' then
          332  +			local t = ''
          333  +			if b.origin.doc.sections[l.ref] then
          334  +				local hn = b.origin.doc.sections[l.ref].heading_node
          335  +				if hn then
          336  +					t = collectText(rc, hn.spans, b, sec):compile()
          337  +				end
          338  +			else
          339  +				local obj = l.origin:ref(l.ref)
          340  +				if type(obj) == 'string' then
          341  +					t = l.origin:ref(l.ref)
          342  +				end
          343  +			end
          344  +			p.div = { fn = tostring(rs.linkctr) .. ') ' .. t }
          345  +		end
          346  +	end;
          347  +
          348  +	function spanRenderers.raw(rc, s, b, sec)
          349  +		rs.renderSpans(rc, s.spans, b, sec)
          350  +	end;
          351  +
          352  +	function spanRenderers.var(rc,v,b,s)
          353  +		local t, raw = ct.expand_var(v)
          354  +		if raw then rc:span(t) else
          355  +			rs.renderSpans(rc,t,b,s)
          356  +		end
          357  +	end
          358  +	function spanRenderers.macro(rc, m,b,s)
          359  +		local macroname = collectText(rc,
          360  +			ct.parse_span(m.macro, b.origin),
          361  +			b, s):compile()
          362  +
          363  +		local r = b.origin:ref(macroname)
          364  +		if type(r) ~= 'string' then
          365  +			b.origin:fail('%s is an object, not a reference', t.ref)
          366  +		end
          367  +		local mctx = b.origin:clone()
          368  +		      mctx.invocation = m
          369  +		rs.renderSpans(rc, ct.parse_span(r, mctx))
          370  +	end
    69    371   
    70    372   	function rs.renderSpans(rc, sp, b, sec)
          373  +		rc = rc or mkrc(b.origin)
    71    374   		for i, v in ipairs(sp) do
    72    375   			if type(v) == 'string' then
    73         -				rc:add(v)
          376  +				rc:span(v)
    74    377   			elseif spanRenderers[v.kind] then
    75    378   				spanRenderers[v.kind](rc, v, b, sec)
    76    379   			end
    77    380   		end
    78    381   	end
    79    382   
    80    383   	local blockRenderers = {}
          384  +	function	blockRenderers.label(rc, b, sec)
          385  +		if ct.sec.is(b.captions) then
          386  +			local sizes = {36,24,12,8,4,2}
          387  +			local margins = {0,5,2,1,0.5}
          388  +			local dedents = {2.5,1.3,0.8,0.4}
          389  +			rc.prop.dsz = sizes[b.captions.depth] or 10
          390  +			rc.prop.underline = b.captions.depth < 4
          391  +			rc.prop.bold = b.captions.depth > 3
          392  +			rc.prop.margin = {
          393  +				top = margins[b.captions.depth] or 0;
          394  +				bottom = 0.1;
          395  +			}
          396  +			rc.prop.indent = -(dedents[b.captions.depth] or 0)
          397  +			rc.prop.underline = true
          398  +			rc.prop.chtitle = collectText(rc, b.spans, b.spec):compile()
          399  +			if b.captions.depth == 1 then
          400  +				rc.prop.breakBefore = true
          401  +			end
          402  +			rs.renderSpans(rc, b.spans, b, sec)
          403  +		else
          404  +			ss.bug 'tried to render label for an unknown object type':throw()
          405  +		end
          406  +	end
    81    407   	function	blockRenderers.paragraph(rc, b, sec)
    82    408   		rs.renderSpans(rc, b.spans, b, sec)
    83    409   	end
    84         -	function rs.renderBlock(b, sec)
    85         -		local rc = {
    86         -			clone = function(self)
    87         -				return {
    88         -					clone = self.clone;
    89         -					lines = self.lines;
    90         -					prop = ss.clone(self.prop);
    91         -					mk = self.mk;
    92         -					add = self.add;
    93         -				}
    94         -			end;
    95         -			lines = {};
    96         -			prop = {};
    97         -			mk = function(self, ln)
    98         -				local p = ss.clone(self.prop)
    99         -				p.txt = ln
   100         -				return p
          410  +	function rs.renderBlock(rc, b, sec, outerBlockRenderContext)
          411  +		if blockRenderers[b.kind] then
          412  +			local rcc = rc:block()
          413  +			blockRenderers[b.kind](rcc, b, sec)
          414  +		end
          415  +	end
          416  +
          417  +	rs.sanitize = gsan
          418  +
          419  +	local skippedFirstPagebreak = doc.secorder[1]:visible()
          420  +	local deferrer = ss.declare {
          421  +		ident = 'groff-deferrer';
          422  +		mk = function(buf) return {ops={}, tgt=buf} end;
          423  +		fns = {
          424  +			esc = function(me, str) table.insert(me.ops, {0, str}) end;
          425  +			req = function(me, str) table.insert(me.ops, {1, str}) end;
          426  +			flush = function(me)
          427  +				for i=#me.ops,1,-1 do
          428  +					local d = me.ops[i]
          429  +					if d[1] == 0 then
          430  +						me.tgt:esc(d[2])
          431  +					elseif d[1] == 1 then
          432  +						me.tgt:req(d[2])
          433  +					end
          434  +				end
          435  +				me.ops = {}
   101    436   			end;
   102         -			add = function(self, ln)
   103         -				table.insert(self.lines, self:mk(ln))
   104         -			end;
   105         -		}
   106         -		if blockRenderers[b.kind] then
   107         -			blockRenderers[b.kind](rc, b, sec)
          437  +		};
          438  +	}
          439  +	function rs.emitSpan(gtxt, s)
          440  +		local defer = deferrer(gtxt)
          441  +		if s.bold or s.emph then
          442  +			if s.bold and s.emph then
          443  +				gtxt:esc 'f(BI'
          444  +			elseif s.bold then
          445  +				gtxt:esc 'fB'
          446  +			elseif s.emph then
          447  +				gtxt:esc 'fI'
          448  +			end
          449  +			defer:esc'f[]'
   108    450   		end
   109         -		return rc.lines
          451  +
          452  +		if s.color and opts.color then
          453  +			gtxt:esc('m[' .. s.color .. ']')
          454  +			defer:esc('m[]')
          455  +		end
          456  +		if s.strike then
          457  +			gtxt:req('ST "'..s.txt..'"')
          458  +		else
          459  +			gtxt:txt(s.txt)
          460  +		end
          461  +		defer:flush()
          462  +		if s.div then
          463  +			for div, body in pairs(s.div) do
          464  +				if div == 'fn' then
          465  +					gtxt:req 'ev footnote-env'
          466  +				end
          467  +				gtxt:req('boxa '..div)
          468  +				gtxt:txt(body)
          469  +				gtxt:raw '\n'
          470  +				gtxt:req 'boxa'
          471  +				if div == 'fn' then
          472  +					gtxt:req 'ev'
          473  +					gtxt:req 'nr footnote-pos (\\n[footnote-pos]u+\\n[dn]u)'
          474  +					gtxt:req 'ch footnote-print -(\\n[footnote-pos]u+1c)'
          475  +				end
          476  +			end
          477  +		end
   110    478   	end
          479  +	function rs.emitBlock(gtxt, b)
          480  +		local didfinalbreak = false
          481  +		local defer = deferrer(gtxt)
          482  +		local ln = b.prop
          483  +		if ln.chtitle then
          484  +			gtxt:req('ds title '..ln.chtitle)
          485  +		end
          486  +		if ln.breakBefore then
          487  +			if skippedFirstPagebreak then
          488  +				gtxt:req 'bp'
          489  +			else
          490  +				skippedFirstPagebreak = true
          491  +			end
          492  +		end
          493  +		if ln.indent then
          494  +			if ln.indent < 0 then
          495  +				gtxt:req('in '..tostring(ln.indent)..'m')
          496  +				defer:req 'in'
          497  +				gtxt:req('ll +'..tostring(-ln.indent)..'m')
          498  +				defer:req 'll'
          499  +			else
          500  +				gtxt:req('in +'..tostring(ln.indent)..'m')
          501  +				defer:req 'in'
          502  +			end
          503  +			defer:req 'br'
          504  +		end
          505  +		if ln.margin then
          506  +			if ln.margin.top then
          507  +				gtxt:req(string.format('sp %sm', ln.margin.top))
          508  +			end
          509  +		end
          510  +
          511  +		if ln.underline then
          512  +			defer:esc("D'l \\n[.ll]u-\\n[.in]u 0'")
          513  +			defer:esc"v'-0.5'"
          514  +			defer:req'br'
          515  +		end
          516  +
          517  +		if ln.dsz and ln.dsz > 0 then
          518  +			gtxt:req('ps +' .. tostring(ln.dsz) .. 'p')
          519  +			defer:req('ps -' .. tostring(ln.dsz) .. 'p')
          520  +		elseif ln.sz or ln.dsz then
          521  +			if ln.sz and ln.sz <= 0 then
          522  +				ln.origin:fail 'font sizes must be greater than 0'
          523  +			end
          524  +			gtxt:req('ps ' .. tostring(ln.sz or ln.dsz) ..'p')
          525  +			if ln.dsz then
          526  +				defer:req('ps +' .. tostring(0 - ln.dsz) .. 'p')
          527  +			else
          528  +				defer:req'ps'
          529  +			end
          530  +		end
          531  +
          532  +		for i,s in pairs(b.spans) do
          533  +			rs.emitSpan(gtxt, s)
          534  +		end
          535  +
   111    536   
   112         -	function rs.emitLine(ln)
   113         -		local q = ss.strac()
   114         -		if ln.dsz then
   115         -			q('\\ps +' .. tostring(ln.dsz))
   116         -		elseif ln.sz then
   117         -			q('\\ps ' .. tostring(ln.dsz))
          537  +		if ln.margin then
          538  +			if ln.margin.bottom then
          539  +				gtxt:req(string.format('sp %sm', ln.margin.bottom))
          540  +			end
   118    541   		end
   119    542   
   120         -		if ln.bold and ln.emph then
   121         -			q '\\f(BI'
   122         -		elseif ln.bold then
   123         -			q '\\fB'
   124         -		elseif ln.emph then
   125         -			q '\\fI'
   126         -		end
          543  +		defer:flush()
   127    544   
   128         -
   129         -		q(ln.txt)
   130         -
   131         -		if ln.bold or ln.emph then
   132         -			q'\\f[]'
   133         -		end
   134         -
   135         -		if ln.dsz then
   136         -			q('.ps -' .. tostring(ln.dsz))
   137         -		elseif ln.sz then
   138         -			q '.ps'
   139         -		end
   140         -		return q
          545  +		if not ln.margin then gtxt:brk() end
   141    546   	end
   142    547   
   143    548   	local ir = {}
   144    549   	for i, sec in ipairs(doc.secorder) do
   145    550   		if sec.kind == 'ordinary' then
   146         -			local blks = {}
          551  +			local rc = mkrc()
   147    552   			for j, b in ipairs(sec.blocks) do
   148         -				local r = rs.renderBlock(b, sec)
   149         -				if r then table.insert(blks, r) end
          553  +				rs.renderBlock(rc, b, sec)
   150    554   			end
   151         -			table.insert(ir, blks)
          555  +			table.insert(ir, {blocks = rc.blocks, src = sec})
   152    556   		end
   153    557   	end
   154    558   
   155         -	local rd = ss.strac()
          559  +	local gd = gtxt()
   156    560   	for i, s in ipairs(ir) do
   157         -		for j, b in ipairs(s) do
   158         -			for z, l in ipairs(b) do
   159         -				rd(rs.emitLine(l))
   160         -			end
   161         -			rd'\n'
          561  +		for j, b in ipairs(s.blocks) do
          562  +			rs.emitBlock(gd,b)
   162    563   		end
   163    564   	end
   164    565   
   165    566   	local macs = ss.strac()
   166    567   	for _, m in pairs(rs.macsNeeded.order) do
   167         -		for _, ln in pairs(m) do macs(ln) end
          568  +		for _,ln in pairs(rs.macsets[m]) do macs(ln) end
          569  +	end
          570  +	if rs.macsNeeded.map.color and opts.color then
          571  +		for k,v in pairs(rs.colors) do
          572  +			macs(mkColorDef(k,v))
          573  +		end
          574  +	end
          575  +
          576  +	local doctitle = '' if opts.title then
          577  +		doctitle = opts.title
          578  +	else
          579  +		local top = math.huge
          580  +		for i,s in ipairs(doc.secorder) do
          581  +			if s.heading_node and s.depth < top then
          582  +				top = s.depth
          583  +				doctitle = collectText(mkrc():block(), s.heading_node.spans, s.heading_node, s):compile()
          584  +			end
          585  +		end
   168    586   	end
   169         -	return macs:compile'\n' .. rd:compile''
          587  +	macs('.ds doctitle '..doctitle)
          588  +
          589  +	return macs:compile'\n' .. '\n' .. gd:compile()
   170    590   end

Modified render/html.lua from [c1f1be8e43] to [778a76ed09].

     6      6   --    good both on a screen and when printed.
     7      7   --  > cortav -m render:format html
     8      8   
     9      9   local ct = require 'cortav'
    10     10   local ss = require 'sirsem'
    11     11   
    12     12   -- install rendering function for html
    13         -function ct.render.html(doc, opts)
           13  +function ct.render.html(doc, opts, setup)
    14     14   	local doctitle = opts['title']
    15     15   	local f = string.format
    16     16   	local getSafeID = ct.tool.namespace()
    17     17   
    18     18   	local footnotes = {}
    19     19   	local footnotecount = 0
    20     20   
................................................................................
   417    417   		style_rules = styles; -- use stylesneeded if at all possible
   418    418   		style_add = addStyle;
   419    419   		stylesets = stylesets;
   420    420   		stylesets_active = stylesNeeded;
   421    421   		obj_htmlid = getSafeID;
   422    422   		-- remaining fields added later
   423    423   	}
          424  +
          425  +	-- this is kind of gross but the context object belongs to the parser,
          426  +	-- not the renderer, so that's not a suitable place for this information
          427  +	doc.stage = {
          428  +		kind = 'render';
          429  +		format = 'html';
          430  +		html_render_state = render_state_handle;
          431  +	}
          432  +
          433  +	setup(doc.stage)
   424    434   
   425    435   	local renderJob = doc:job('render_html', nil, render_state_handle)
   426    436   	doc.stage.job = renderJob;
   427    437   
   428    438   	local runhook = function(h, ...)
   429    439   		return renderJob:hook(h, render_state_handle, ...)
   430    440   	end
................................................................................
   559    569   				end
   560    570   			else
   561    571   				b.origin:fail('%s is not an object that can be embedded', t.ref)
   562    572   			end
   563    573   		end
   564    574   
   565    575   		function span_renderers.var(v,b,s)
   566         -			local val
   567         -			if v.pos then
   568         -				if not v.origin.invocation then
   569         -					v.origin:fail 'positional arguments can only be used in a macro invocation'
   570         -				elseif not v.origin.invocation.args[v.pos] then
   571         -					v.origin.invocation.origin:fail('macro invocation %s missing positional argument #%u', v.origin.invocation.macro, v.pos)
   572         -				end
   573         -				val = v.origin.invocation.args[v.pos]
   574         -			else
   575         -				val = v.origin.doc:context_var(v.var, v.origin)
   576         -			end
   577         -			if v.raw then
   578         -				return val
   579         -			else
   580         -				return htmlSpan(ct.parse_span(val, v.origin), b, s)
          576  +			local r, raw = ct.expand_var(v)
          577  +			if raw then return r else
          578  +				return htmlSpan(r , b, s)
   581    579   			end
   582    580   		end
   583    581   
   584    582   		function span_renderers.raw(v,b,s)
   585    583   			return htmlSpan(v.spans, b, s)
   586    584   		end
   587    585   

Modified sirsem.lua from [4b787982a7] to [550cdedbd6].

     1      1   -- [ʞ] sirsem.lua
     2         ---  ~ lexu hale <lexi@hale.su>
            2  +--  ~ lexi hale <lexi@hale.su>
            3  +--    glowpelt (hsl conversion)
     3      4   --  ? utility library with functionality common to
     4      5   --    cortav.lua and its extensions
     5      6   --    from Ranuir "software utility"
     6      7   --  > local ss = require 'sirsem.lua'
     7      8   
     8      9   local ss
     9     10   do -- pull ourselves up by our own bootstraps
................................................................................
    43     44   	local new = {}
    44     45   	for k, v in pairs(list) do
    45     46   		local nk,nv = fn(k,v)
    46     47   		new[nk or k] = nv or v
    47     48   	end
    48     49   	return new
    49     50   end
           51  +function ss.tmap(fn, a, ...)
           52  +	if a == nil then return end
           53  +	return fn(a), ss.tmap(fn, ...)
           54  +end
    50     55   
    51     56   function ss.kfilter(list, fn)
    52     57   	local new = {}
    53     58   	for k, v in pairs(list) do
    54     59   		if fn(k,v) then new[k] = v end
    55     60   	end
    56     61   	return new
................................................................................
   546    551   
   547    552   function ss.declare(c)
   548    553   	local cls = setmetatable({
   549    554   		__name = c.ident;
   550    555   	}, {
   551    556   		__name = 'class';
   552    557   		__tostring = function() return c.ident or '(class)' end;
          558  +	   __index = c.cfns;
   553    559   	})
   554    560   
   555    561   	cls.__call = c.call
   556    562   	cls.__index = function(self, k)
   557    563   		if c.default and c.default[k] then
   558    564   			return c.default[k]
   559    565   		end
................................................................................
   587    593   		if c.cast.string then
   588    594   			cls.__tostring = c.cast.string
   589    595   		end
   590    596   		if c.cast.number then
   591    597   			cls.__tonumber = c.cast.number
   592    598   		end
   593    599   	end
          600  +
          601  +	if c.op then
          602  +		cls.__add = c.op.sum
          603  +		cls.__sub = c.op.sub
          604  +		cls.__div = c.op.div
          605  +		cls.__mul = c.op.mul
          606  +		cls.__concat = c.op.cat
          607  +	end
   594    608   
   595    609   	cls.mk = function(...)
   596    610   		local val = setmetatable(c.mk and c.mk(...) or {}, cls)
   597    611   		if c.init then
   598    612   			for k,v in pairs(c.init) do
   599    613   				val[k] = v
   600    614   			end
................................................................................
   885    899   				end
   886    900   			else
   887    901   				me:react(sym)
   888    902   			end
   889    903   		end;
   890    904   	};
   891    905   }
          906  +
          907  +function ss.math.clamp(v, l, h)
          908  +	return math.max(math.min(v, h or 1), l or 0)
          909  +end
   892    910   
   893    911   -- convenience buffer for holding strings under
   894    912   -- construction, accumulating and compiling then in
   895    913   -- as quick a way as lua permits
   896    914   ss.strac = ss.declare {
   897    915   	ident = 'string-accumulator';
   898    916   	mk = function() return {
................................................................................
   935    953   		end;
   936    954   		wrap = function(self,a,b)
   937    955   			table.insert(self.strs, 1, a)
   938    956   			table.insert(self.strs, b)
   939    957   		end;
   940    958   	};
   941    959   }
          960  +
          961  +-- color class based on c.hale.su/sorcery's, hsl conversion
          962  +-- code written by glowpelt. TODO switch to LCH
          963  +local function clip(v, ...)
          964  +	if v == nil then return end
          965  +	return math.max(0,math.min(0xFF,math.floor(v))), clip(...)
          966  +end;
          967  +local function bytefrac(f, ...)
          968  +	if f == nil then return end
          969  +	return clip(f*0xFF), bytefrac(...)
          970  +end
          971  +ss.color = ss.declare {
          972  +	ident = 'color';
          973  +	mk = function(h,s,l,a) return {
          974  +		hue = h or 0.0;
          975  +		sat = s or 0.0;
          976  +		lum = l or 0.0;
          977  +		alpha = a or 1.0;
          978  +	} end;
          979  +	cfns = {
          980  +		byteclip = clip;
          981  +		bytefrac = bytefrac;
          982  +	};
          983  +	cast = {
          984  +		string = function(self) return self:hex() end;
          985  +		number = function(self) return self:u32() end;
          986  +	};
          987  +	op = {
          988  +		sum = function(self, other)
          989  +			if ss.color.is(other) then
          990  +				local fac = ss.math.lerp(self.alpha, 1, other.alpha)
          991  +				return self:blend(other, fac):warp(function(c)
          992  +					c.alpha = ss.math.clamp(self.alpha+other.alpha)
          993  +				end)
          994  +			else -- color + number = brighter color
          995  +				return self:warp(function(c)
          996  +					c.lum = c.lum + other
          997  +				end)
          998  +			end
          999  +		end;
         1000  +		mul = function(self, other)
         1001  +			if ss.color.is(other) then
         1002  +				ss.color.exn 'how the heck do you multiply in hsl anyway':throw()
         1003  +			else
         1004  +				return self:warp(function(c)
         1005  +					c.lum = c.lum * other
         1006  +				end)
         1007  +			end
         1008  +		end;
         1009  +	};
         1010  +	fns = {
         1011  +		tuple = function(self)
         1012  +			return self.hue, self.sat, self.lum, self.alpha
         1013  +		end;
         1014  +		warp = function(self, func)
         1015  +			local n = self:clone()
         1016  +			func(n)
         1017  +			return n
         1018  +		end;
         1019  +		blend = function(self, other, fac)
         1020  +			return ss.color(
         1021  +				ss.math.lerp(fac, self.hue, other.hue),
         1022  +				ss.math.lerp(fac, self.sat, other.sat),
         1023  +				ss.math.lerp(fac, self.lum, other.lum),
         1024  +				ss.math.lerp(fac, self.alpha, other.alpha))
         1025  +		end;
         1026  +		hex = function(self)
         1027  +			local r,g,b,a = bytefrac(self:rgb_t())
         1028  +			if self.alpha == 1 then a = nil end
         1029  +			return string.format('#'..string.rep('%02x',a and 4 or 3),
         1030  +				r,g,b,a)
         1031  +		end;
         1032  +		u32 = function(self)
         1033  +			local r,g,b,a = bytefrac(self:rgb_t())
         1034  +			return r<<24 | g << 16 | b << 8 | a
         1035  +		end;
         1036  +		bytes = function(self)
         1037  +			return { bytefrac(self:rgb_t()) }
         1038  +		end;
         1039  +		alt = function(self, fld, new)
         1040  +			if self[fld] then
         1041  +				return self:warp(function(c) c[fld]=new end)
         1042  +			else
         1043  +				ss.color.exn('no such field %s in color', fld):throw()
         1044  +			end
         1045  +		end;
         1046  +		rgb = function(self)
         1047  +			-- convenience function to get a standardized struct
         1048  +			local r,g,b,a = self:rgb_t()
         1049  +			return {
         1050  +				red = r;
         1051  +				green = g;
         1052  +				blue = b;
         1053  +				alpha = a;
         1054  +			}
         1055  +		end;
         1056  +		rgb_t = function(self)
         1057  +			-- returns rgba values as a tuple
         1058  +			local value = function(n1, n2, hue)
         1059  +				if hue > 360 then
         1060  +					hue = hue - 360
         1061  +				elseif hue < 0 then
         1062  +					hue = hue + 360
         1063  +				end
         1064  +				if hue < 60 then
         1065  +					return n1 + (n2 - n1) * hue/60
         1066  +				elseif hue < 180 then
         1067  +					return n2
         1068  +				elseif hue < 240 then
         1069  +					return n1 + (n2 - n1) * (240 - hue)/60
         1070  +				else
         1071  +					return n1
         1072  +				end
         1073  +			end
         1074  +			local h,s,l,alpha = self:tuple()
         1075  +			local m2
         1076  +			if l < 0.5 then
         1077  +				m2 = l * (1 + s)
         1078  +			else
         1079  +				m2 = l + s - l * s
         1080  +			end
         1081  +			local m1 = 2 * l - m2
         1082  +			if s == 0 then
         1083  +				-- Achromatic, there is no hue
         1084  +				-- In book this errors if hue is not undefined, but we set hue to 0 in this case, not nil or something, so
         1085  +				return l, l, l, alpha
         1086  +			else
         1087  +				-- Chromatic case, so there is a hue
         1088  +				return
         1089  +					value(m1, m2, h + 120),
         1090  +					value(m1, m2, h),
         1091  +					value(m1, m2, h - 120),
         1092  +					alpha
         1093  +			end
         1094  +		end;
         1095  +	};
         1096  +};
         1097  +ss.color.exn = ss.exnkind 'color error'
         1098  +
         1099  +ss.cmdfmt = function(cmd, ...)
         1100  +	return string.format(cmd, ss.tmap(function(s)
         1101  +		if typeof(s) == 'string' then
         1102  +			return string.format("%q", s)
         1103  +			-- FIXME this is incredibly lazy and uses lua quoting, not
         1104  +			-- bourne shell quoting. it *will* cause problems if anything
         1105  +			-- exotic finds its way in and needs to be fixed.
         1106  +			-- TODO provide a proper popen in the C wrapper so wrapped
         1107  +			-- versions at least can launch programs in a sane and secure
         1108  +			-- way.
         1109  +		else
         1110  +			return s
         1111  +		end
         1112  +	end, ...))
         1113  +end