On 2/13/2016 3:21 AM, Will Robertson wrote:
On 12 Feb 2016, at 10:08 PM, Hans Hagen <[email protected]> wrote:
injecting stuff (at the user level) in the input stream in an uncontrolled way
is messy anyway but if one really wants to do that, one can easily kick in a
callback when reading the file and then add commands or whatever at unmarked
script/language boundaries (after all it's mostly trying to make structure from
non structure we're talking of)
Working directly before the input stream is processed seems unlikely to be able
address things like
\def\x{x}
\def\y{<chinese>}
\x\y
indeed, and a token injector has the same issue as it takes action at
some spot that might suits some use but not another use so it's always
half a solution (which is why we've chosen for callbacks at several spots)
And using virtual fonts to add Chinese to a roman font is not enough because
you also have to switch the language setup. Oh I see Ulrike already said that :)
sure but injecting tokens into the stream based on character properties
is just as not-enough as it can badly interfere with whatever gets
injected etc etc (i bet grouping is a nightmare) so i would never use
such a method in context
Since XeTeX’s interchartoken feature will never be provided directly in LuaTeX
(I assume), it would be nice at least if it could be emulated by 3rd party
authors so that XeTeX and LuaTeX code can be shared. So I’d support this
proposal:
you assume right (and compatibility with other engines is not on the
agenda as one can just use the other engines)
we might add a token.print function (if so it will be around version 0.95) in
which case one can use token.get_tok in an endless loop to pick up tokens and
print them back but (as with the callback) it will be very inefficient
… even if it is inefficient :)
i experimeted with it and it will probably show up (but one probably has
to fight a bit with expansion, lookahead and input levels)
Hans
-----------------------------------------------------------------
Hans Hagen | PRAGMA ADE
Ridderstraat 27 | 8061 GH Hasselt | The Netherlands
tel: 038 477 53 69 | www.pragma-ade.com | www.pragma-pod.nl
-----------------------------------------------------------------
