Re: [XeTeX] Erroneous \lastkern

[Ross Moore]

Hi Ulrike, Jonathan,

On 28 Jun 2019, at 6:59 pm, Jonathan Kew 
mailto:jfkth...@gmail.com>> wrote:

On 28/06/2019 08:32, Ulrike Fischer wrote:
> Am Thu, 27 Jun 2019 23:03:09 + schrieb Ross Moore:
>
>>> there is probably something wrong with XeLaTeX, but I cannot find what.
>>
>> The difference between xetex and xelatex is the font:
>>
>> I’m sorry but I don’t understand this as an answer.
>
> It wasn't meant as an answer. I only explained why you get different
> results with plain and latex: because they use different default
> fonts.

OK. But that just emphasises that there is a bug lurking here.

> xetex has a different typesetting engine: it doesn't handle chars
> but words as units.
>
> See page 31 here 
> http://xml.web.cern.ch/XML/lgc2/xetexmain.pdf.
>
> So I'm not really surprised that you get the y, I was more suprised
> that it doesn't happen with legacy fonts - there it seem to switch
> back to the "handle characters" mode.

Yes, the bug arises because of how xetex collects a series of characters
to be "shaped" by an opentype font, rather than the core tex engine
handling each character individually. So at the point when \lastkern is
encountered, the letter A has not yet been appended to the current node
list being built; it is "pending" in the buffer of characters that will
become a whole-word node.

OK, I understand that now.
In fact I was think of surmising that such a mechanism could be in play.

But surely this means that XeTeX's “current list” should be that buffer, not 
TeX’s usual horizontal list.
So  \lastkern , \lastpenalty  and  \lastskip  should be getting their values 
from there.

Alternatively, and perhaps equivalently, just set all these to zero whenever we 
are adding characters
to this “pending” buffer. Only when the words are ready to be put back into the 
horizontal list,
should these be set back to what is there, if those values are still relevant 
to any typesetting tasks
at the beginning of that resulting word.


Still, I would regard this as a bug that we ought to fix. I imagine
similar primitives like \lastpenalty or \lastskip probably share the
same buggy behavior.

Yes, I would think so.

BTW, this is relevant to my Tagged PDF work, as it must insert extra literal 
material via \special  commands.
To ensure the same typesetting as without those tags, it is frequently 
necessary to transfer that previous
\skip , \kern  or \penalty  to come *after* the \special  and nullify the one 
before it.

So far this is only developed for pdfTeX, but in future we’ll want it for XeTeX 
too.
Discovering this difference now, and fixing it, will surely avoid a headache 
when that time comes.


JK


Cheers,

Ross


Dr Ross Moore
Department of Mathematics and Statistics
12 Wally’s Walk, Level 7, Room 734
Macquarie University, NSW 2109, Australia
T: +61 2 9850 8955  |  F: +61 2 9850 8114
M:+61 407 288 255  |  E: ross.mo...@mq.edu.au
http://www.maths.mq.edu.au
[cid:image001.png@01D030BE.D37A46F0]
CRICOS Provider Number 2J. Think before you print.
Please consider the environment before printing this email.

This message is intended for the addressee named and may
contain confidential information. If you are not the intended
recipient, please delete it and notify the sender. Views expressed
in this message are those of the individual sender, and are not
necessarily the views of Macquarie University. 

Re: [XeTeX] Erroneous \lastkern

[Jonathan Kew]

On 28/06/2019 08:32, Ulrike Fischer wrote:

Am Thu, 27 Jun 2019 23:03:09 + schrieb Ross Moore:


there is probably something wrong with XeLaTeX, but I cannot find what.


The difference between xetex and xelatex is the font:

I’m sorry but I don’t understand this as an answer.


It wasn't meant as an answer. I only explained why you get different
results with plain and latex: because they use different default
fonts.


Put another way, the result of the test should not depend upon the initial use 
of  \kern2019sp  at all.
However, if it is commented-out, then  XeTeX does give  n , so the  y  is *not* 
coming from the letter A .


xetex has a different typesetting engine: it doesn't handle chars
but words as units.

See page 31 here http://xml.web.cern.ch/XML/lgc2/xetexmain.pdf.

So I'm not really surprised that you get the y, I was more suprised
that it doesn't happen with legacy fonts - there it seem to switch
back to the "handle characters" mode.


Yes, the bug arises because of how xetex collects a series of characters 
to be "shaped" by an opentype font, rather than the core tex engine 
handling each character individually. So at the point when \lastkern is 
encountered, the letter A has not yet been appended to the current node 
list being built; it is "pending" in the buffer of characters that will 
become a whole-word node.


Still, I would regard this as a bug that we ought to fix. I imagine 
similar primitives like \lastpenalty or \lastskip probably share the 
same buggy behavior.


JK

Re: [XeTeX] Erroneous \lastkern

[Ulrike Fischer]

Am Thu, 27 Jun 2019 23:03:09 + schrieb Ross Moore:

>> there is probably something wrong with XeLaTeX, but I cannot find what.
> 
> The difference between xetex and xelatex is the font:
> 
> I’m sorry but I don’t understand this as an answer.

It wasn't meant as an answer. I only explained why you get different
results with plain and latex: because they use different default
fonts. 

> Put another way, the result of the test should not depend upon the initial 
> use of  \kern2019sp  at all.
> However, if it is commented-out, then  XeTeX does give  n , so the  y  is 
> *not* coming from the letter A .

xetex has a different typesetting engine: it doesn't handle chars
but words as units. 

See page 31 here http://xml.web.cern.ch/XML/lgc2/xetexmain.pdf.

So I'm not really surprised that you get the y, I was more suprised
that it doesn't happen with legacy fonts - there it seem to switch
back to the "handle characters" mode. 



-- 
Ulrike Fischer 
http://www.troubleshooting-tex.de/

Re: [XeTeX] Erroneous \lastkern

[Ross Moore]

Hi Ulrike,

On 28 Jun 2019, at 6:27 am, Ulrike Fischer 
mailto:ne...@nililand.de>> wrote:

Am Thu, 27 Jun 2019 21:13:05 +0200 schrieb Christian Tellechea:

> there is probably something wrong with XeLaTeX, but I cannot find what.

The difference between xetex and xelatex is the font:

I’m sorry but I don’t understand this as an answer.
I would expect the test of \lastkern  to always give  n
no matter what the driver, font, with LaTeX or not.

The result from  \lastkern  (according to the TeXBook)
is that it is only non-zero when the last item on the current list is a kern.
(It is *not* a place-holder for the last kern that has been used.)


Thus the appearance of the letter A should have reset  \lastkern  to 0,
*except* if there is a kern provided by A itself (font dependent, perhaps yes);
but in that case it is unlikely to have the value of  2019sp .

Of course, maybe this value has been chosen, *because* that font does end with 
such a kern.
This would be the only valid reason that you could ever get a  y  from this 
code.


Put another way, the result of the test should not depend upon the initial use 
of  \kern2019sp  at all.
However, if it is commented-out, then  XeTeX does give  n , so the  y  is *not* 
coming from the letter A .

To me this *does* indicate a bug;
or at least a difference in how XeTeX behaves, compared with the original TeX.



\documentclass{article}

\begin{document}

Should be An: \kern2019sp A\ifnum\lastkern=2019 y\else n\fi\par

\fontencoding{OT1}\selectfont
Should be An: \kern2019sp A\ifnum\lastkern=2019 y\else n\fi\par

\end{document}

or in plain xetex:

Should be An: \kern2019sp A\ifnum\lastkern=2019 y\else n\fi\par

\font\test="[lmroman10-regular.otf]"\test
Should be An: \kern2019sp A\ifnum\lastkern=2019 y\else n\fi\par

\bye


--
Ulrike Fischer
https://www.troubleshooting-tex.de/


Cheers,

Ross


Dr Ross Moore
Department of Mathematics and Statistics
12 Wally’s Walk, Level 7, Room 734
Macquarie University, NSW 2109, Australia
T: +61 2 9850 8955  |  F: +61 2 9850 8114
M:+61 407 288 255  |  E: ross.mo...@mq.edu.au
http://www.maths.mq.edu.au
[cid:image001.png@01D030BE.D37A46F0]
CRICOS Provider Number 2J. Think before you print.
Please consider the environment before printing this email.

This message is intended for the addressee named and may
contain confidential information. If you are not the intended
recipient, please delete it and notify the sender. Views expressed
in this message are those of the individual sender, and are not
necessarily the views of Macquarie University. 

Re: [XeTeX] Erroneous \lastkern

[Ulrike Fischer]

Am Thu, 27 Jun 2019 21:13:05 +0200 schrieb Christian Tellechea:

> there is probably something wrong with XeLaTeX, but I cannot find what.

The difference between xetex and xelatex is the font:

\documentclass{article}

\begin{document}

Should be An: \kern2019sp A\ifnum\lastkern=2019 y\else n\fi\par

\fontencoding{OT1}\selectfont
Should be An: \kern2019sp A\ifnum\lastkern=2019 y\else n\fi\par

\end{document}

or in plain xetex:

Should be An: \kern2019sp A\ifnum\lastkern=2019 y\else n\fi\par

\font\test="[lmroman10-regular.otf]"\test
Should be An: \kern2019sp A\ifnum\lastkern=2019 y\else n\fi\par

\bye


-- 
Ulrike Fischer 
https://www.troubleshooting-tex.de/

[XeTeX] Erroneous \lastkern

[Christian Tellechea]

Hello.


This question refers to XeLaTeX and not XeTeX which gives correct 
results (see below).



The following code shows an incorrect \lastkern value during the \ifnum 
test: the first line shows abnormal behaviour while the second, thanks 
to the \relax, gives the expected result.


%% CODE %%

\documentclass{article}

\begin{document}

Should be An: \kern2019sp A\ifnum\lastkern=2019 y\else n\fi\par

Should be An: \kern2019sp A\relax\ifnum\lastkern=2019 y\else n\fi

\end{document}

%%



With XeTeX, the following code gives the expected results: there is 
probably something wrong with XeLaTeX, but I cannot find what.


%% CODE %%

Should be An: \kern2019sp A\ifnum\lastkern=2019 y\else n\fi\par

Should be An: \kern2019sp A\relax\ifnum\lastkern=2019 y\else n\fi

\bye

%%


Best.

Christian Tellechea