On 8/29/2021 12:43 AM, Vítek Novotný wrote:
Hello all,
in Knuth's TeX, trailing spaces are removed very early on when a line is
being put to the input buffer. [1] According to Eijkhout's TeX by
Topic, this is because "these spaces are hard to see in an editor" [2].
[1]: https://texdoc.org/serve/tex.pdf/0#page=15
[2]: http://mirrors.ctan.org/info/texbytopic/TeXbyTopic.pdf
I develop and maintain the Markdown package [3] for plain TeX, ConTeXt,
and LaTeX. The package makes it possible to use the lightweight markup
of markdown [4] in TeX documents. In markdown, a hard line break can be
inserted by ending a line with two or more spaces. However, since
trailing spaces are removed by TeX, hard breaks are only recognized when
we' are inserting an external markdown file, not when markdown is typed
in the top-level document. This deficiency is known and documented [5],
but I am hoping we could resolve it with LuaTeX.
i wonder how this double spaces works out in practice, for instance one
needs to 'visualize' them in the editor so see them and also make sure
that the editor is not in 'prune space at the end of line' mode
[3]: https://github.com/witiko/markdown
[4]: https://daringfireball.net/projects/markdown
[5]: https://mirrors.ctan.org/macros/generic/markdown/markdown.pdf#page=20
In LuaTeX, the `process_input_buffer` callback [6] can be used to
intercept the text coming *out* of the input buffer. However, the
trailing spaces have already been removed by this point.
By adding a callback right after a line has entered the input buffer
[1], we could either replace the trailing space characters with tabs,
or place a character such as the zero-width non-joiner (U+200C) to the
right of the trailing spaces.
i fear that this will introduce a performance hit
[6]: https://www.pragma-ade.com/general/manuals/luatex.pdf#page=176
Is this something you would consider---if not for LuaTeX then perhaps
for LuaMetaTeX?
for a while i had a endofline handler but removed it because i never
used it as it made no sense (it's all too unpredictable) so i removed it
(i wanted to backport it but it doesn't really fit in now that luatex
also has some special \par handling added)
concerning context, how is \startmarkdown defined? I'm pretty sure that
this issue can handled without adding callbacks (i never needed/use(d)
markdown myself so i can only guess here) but we can discuss the needs
off-luatex-list
Hans
-----------------------------------------------------------------
Hans Hagen | PRAGMA ADE
Ridderstraat 27 | 8061 GH Hasselt | The Netherlands
tel: 038 477 53 69 | www.pragma-ade.nl | www.pragma-pod.nl
-----------------------------------------------------------------
