Re: [NTG-context] LMTX MkIV difference in expansion
On 11/21/2020 10:05, Wolfgang Schuster wrote:
Rik Kabel schrieb am 20.11.2020 um 00:18:
You are right about not quite understand.
There are cases where you want to pass a command to another command as
it is without replacing it with its content, e.g. when you store the
\TeX logo in the table of content the \TeX command should be written
in the register and not the content of the command.
In the following example the first line prints the definition of the
\TeX logo but in many cases you ant to preserve the command as in the
second line.
\starttext
\tex{TeX} = \detokenize\expandafter{\TeX}
\blank
\tex{TeX} = \detokenize{\TeX}
\stoptext
To make it easier to keep the command eTeX added a new command
\protected which can be used before \def to achieve this (ConTeXt
provides the same thing under the name \unexpanded).
The following example shows how you can use \protected\def to keep
always the current meaning of \foo when you print the content of \bar.
\starttext
\def\foo{foo}
\edef\bar{\foo}
\def\foo{bar}
\startlines
bar=\bar
foo=\foo
\stoplines
\blank
\protected\def\foo{foo}
\edef\bar{\foo}
\protected\def\foo{bar}
\startlines
bar=\bar
foo=\foo
\stoplines
\stoptext
A problem in older TeX engines is that \csname ...\endcsname didn't
respect this protection and replaced the protected command with its
content, recently Hans changed this behavior in LMTX which lead to the
error message in your document.
Does this mean that I can have the same definitions in MkIV and LMTX
(after some future update), or should I hunt down the \defines in
both, or that I should fork (or mode test) my source environment
files, one set for LMTX and one for MkIV?
When you use \define to store arguments which are passed as arguments
to other command you have to change this to \defineexpandable but its
best to do this in MkIV and LMTX because protected commands are the
wrong thing in this case. Even though it would work in MkIV in some
cases you run into problems when you pass argument to Lua.
Wolfgang
Thank you, Wolfgang, for the explanation and examples.
I have in fact already gone through and replaced the impacted
occurrences of \define with \defineexpandable. LMTX made it easy to
identify them.
--
Rik
___
If your question is of interest to others as well, please add an entry to the
Wiki!
maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context
webpage : http://www.pragma-ade.nl / http://context.aanhet.net
archive : https://bitbucket.org/phg/context-mirror/commits/
wiki : http://contextgarden.net
___
Re: [NTG-context] LMTX MkIV difference in expansion
Rik Kabel schrieb am 20.11.2020 um 00:18:
You are right about not quite understand.
There are cases where you want to pass a command to another command as
it is without replacing it with its content, e.g. when you store the
\TeX logo in the table of content the \TeX command should be written in
the register and not the content of the command.
In the following example the first line prints the definition of the
\TeX logo but in many cases you ant to preserve the command as in the
second line.
\starttext
\tex{TeX} = \detokenize\expandafter{\TeX}
\blank
\tex{TeX} = \detokenize{\TeX}
\stoptext
To make it easier to keep the command eTeX added a new command
\protected which can be used before \def to achieve this (ConTeXt
provides the same thing under the name \unexpanded).
The following example shows how you can use \protected\def to keep
always the current meaning of \foo when you print the content of \bar.
\starttext
\def\foo{foo}
\edef\bar{\foo}
\def\foo{bar}
\startlines
bar=\bar
foo=\foo
\stoplines
\blank
\protected\def\foo{foo}
\edef\bar{\foo}
\protected\def\foo{bar}
\startlines
bar=\bar
foo=\foo
\stoplines
\stoptext
A problem in older TeX engines is that \csname ...\endcsname didn't
respect this protection and replaced the protected command with its
content, recently Hans changed this behavior in LMTX which lead to the
error message in your document.
Does this mean that I can have the same definitions in MkIV and LMTX
(after some future update), or should I hunt down the \defines in both,
or that I should fork (or mode test) my source environment files, one
set for LMTX and one for MkIV?
When you use \define to store arguments which are passed as arguments to
other command you have to change this to \defineexpandable but its best
to do this in MkIV and LMTX because protected commands are the wrong
thing in this case. Even though it would work in MkIV in some cases you
run into problems when you pass argument to Lua.
Wolfgang
___
If your question is of interest to others as well, please add an entry to the
Wiki!
maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context
webpage : http://www.pragma-ade.nl / http://context.aanhet.net
archive : https://bitbucket.org/phg/context-mirror/commits/
wiki : http://contextgarden.net
___
Re: [NTG-context] LMTX MkIV difference in expansion
On 11/20/2020 12:18 AM, Rik Kabel wrote:
You are right about not quite understand.
Does this mean that I can have the same definitions in MkIV and LMTX
(after some future update), or should I hunt down the \defines in both,
or that I should fork (or mode test) my source environment files, one
set for LMTX and one for MkIV?
Kind of
\protected\def\MyCommand#1{\tricky{?}\stuff{!}}
\def\MyKeyword{tricky stuff}
because you want to always expand MyKeyword and in controlled cases
\MyCommand.
Hans
-
Hans Hagen | PRAGMA ADE
Ridderstraat 27 | 8061 GH Hasselt | The Netherlands
tel: 038 477 53 69 | www.pragma-ade.nl | www.pragma-pod.nl
-
___
If your question is of interest to others as well, please add an entry to the
Wiki!
maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context
webpage : http://www.pragma-ade.nl / http://context.aanhet.net
archive : https://bitbucket.org/phg/context-mirror/commits/
wiki : http://contextgarden.net
___
Re: [NTG-context] LMTX MkIV difference in expansion
On 11/19/2020 17:03, Hans Hagen wrote:
On 11/19/2020 9:41 PM, Wolfgang Schuster wrote:
Rik Kabel schrieb am 19.11.2020 um 21:20:
Another LMTX/MkIV difference, this time with expansion:
\define\Align{yes}
\starttext
\startalignment[\Align]
This works with MkIV but fails with LMTX, complaining: {\tt
tex error on line 3 in file G:/expand.mkvi: The file ended
when scanning an argument.}
\blank
It works in both when \tex{def} or \tex{defineexpandable} is
used instead of \tex{define}.
\blank
What changed?
\stopalignment
\stoptext
It may well be that I have been abusing some laxity in MkIV and that
LMTX is a bit stricter in what it accepts, but I would like to know
if this is an expected difference.
You have to wait for Hans to get an answer but here is a minimal
example.
\starttext
\protected\def\testparameter{test}
%\def\testparameter{test}
\def\test[#1]%
{\expandafter\let\expandafter\testargumentlist\csname#1\endcsname}
\test[\testparameter]
\stoptext
Often arguments to commands like \startsomething[xx] let the xx end up
in some \(if)csname expansion. A protected (\unexpanded in context
speak) macro doesn't expand inside for instance an \edef (or
comparable expandable situation). Now, from that it makes perfect
sense to also not let it expand inside a \csname or \ifcsname. One
reason is that when it does expand, you can get a pretty wild (nested)
sequence of nested expansions and one can be pretty sure that we then
don't have a proper csname. This is why in luatex we have a catch for
running wild csname checking.
The original \ifcsname test was inherited from etex. The \protected
feature also comes from etex. But \csname is a tex natural.
In pdftex (and luatex) a protected macro inside an \(if)csname does
expand which to makes no sense and smells like a bug. Or maybe it was
tricky to catch (the implementation of protected a bit of a hack).
In luametatex protected macros are native and in the process I also
decided to *not* expand them in a \(if)csname where I expect (as said)
protected macros to behave like in an edef. I nice side effect is that
running wild no longer happens (but we still catch it) which can save
quite some useless backup token list construction (needed because tex
has to push back stuff in order to be able to report an error).
So, when you still don't understand it (which I can understand) I'm
sure Wolfgang can explain it better now.
\starttext
\def\foo{foo}
\protected\def\oof{oof}
\csname foo\endcsname
\csname oof\endcsname
\csname \foo\endcsname
% error in luametatex, ok in pdftex/luatex:
% \csname \oof\endcsname
\ifcsname foo\endcsname yes\else nop\fi
\ifcsname oof\endcsname yes\else nop\fi
\ifcsname \foo\endcsname yes\else nop\fi
% nop in luametatex (error intercepted), yes in pdftex/luatex
\ifcsname \oof\endcsname yes\else nop\fi
\stoptext
Now, one can argue that if I consider it a but in the other engines,
why I don't argue that it should be solved. Well, there is too much
legacy code already that might use it as feature so it will not
change. But in luametatex we can 'fix' these things. (We also use the
csname in a rather predictable way in context so i don't expect issues
in the core.)
Hans
You are right about not quite understand.
Does this mean that I can have the same definitions in MkIV and LMTX
(after some future update), or should I hunt down the \defines in both,
or that I should fork (or mode test) my source environment files, one
set for LMTX and one for MkIV?
--
Rik
___
If your question is of interest to others as well, please add an entry to the
Wiki!
maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context
webpage : http://www.pragma-ade.nl / http://context.aanhet.net
archive : https://bitbucket.org/phg/context-mirror/commits/
wiki : http://contextgarden.net
___
Re: [NTG-context] LMTX MkIV difference in expansion
On 11/19/2020 9:41 PM, Wolfgang Schuster wrote:
Rik Kabel schrieb am 19.11.2020 um 21:20:
Another LMTX/MkIV difference, this time with expansion:
\define\Align{yes}
\starttext
\startalignment[\Align]
This works with MkIV but fails with LMTX, complaining: {\tt
tex error on line 3 in file G:/expand.mkvi: The file ended
when scanning an argument.}
\blank
It works in both when \tex{def} or \tex{defineexpandable} is
used instead of \tex{define}.
\blank
What changed?
\stopalignment
\stoptext
It may well be that I have been abusing some laxity in MkIV and that
LMTX is a bit stricter in what it accepts, but I would like to know if
this is an expected difference.
You have to wait for Hans to get an answer but here is a minimal example.
\starttext
\protected\def\testparameter{test}
%\def\testparameter{test}
\def\test[#1]%
{\expandafter\let\expandafter\testargumentlist\csname#1\endcsname}
\test[\testparameter]
\stoptext
Often arguments to commands like \startsomething[xx] let the xx end up
in some \(if)csname expansion. A protected (\unexpanded in context
speak) macro doesn't expand inside for instance an \edef (or comparable
expandable situation). Now, from that it makes perfect sense to also not
let it expand inside a \csname or \ifcsname. One reason is that when it
does expand, you can get a pretty wild (nested) sequence of nested
expansions and one can be pretty sure that we then don't have a proper
csname. This is why in luatex we have a catch for running wild csname
checking.
The original \ifcsname test was inherited from etex. The \protected
feature also comes from etex. But \csname is a tex natural.
In pdftex (and luatex) a protected macro inside an \(if)csname does
expand which to makes no sense and smells like a bug. Or maybe it was
tricky to catch (the implementation of protected a bit of a hack).
In luametatex protected macros are native and in the process I also
decided to *not* expand them in a \(if)csname where I expect (as said)
protected macros to behave like in an edef. I nice side effect is that
running wild no longer happens (but we still catch it) which can save
quite some useless backup token list construction (needed because tex
has to push back stuff in order to be able to report an error).
So, when you still don't understand it (which I can understand) I'm sure
Wolfgang can explain it better now.
\starttext
\def\foo{foo}
\protected\def\oof{oof}
\csname foo\endcsname
\csname oof\endcsname
\csname \foo\endcsname
% error in luametatex, ok in pdftex/luatex:
% \csname \oof\endcsname
\ifcsname foo\endcsname yes\else nop\fi
\ifcsname oof\endcsname yes\else nop\fi
\ifcsname \foo\endcsname yes\else nop\fi
% nop in luametatex (error intercepted), yes in pdftex/luatex
\ifcsname \oof\endcsname yes\else nop\fi
\stoptext
Now, one can argue that if I consider it a but in the other engines, why
I don't argue that it should be solved. Well, there is too much legacy
code already that might use it as feature so it will not change. But in
luametatex we can 'fix' these things. (We also use the csname in a
rather predictable way in context so i don't expect issues in the core.)
Hans
-
Hans Hagen | PRAGMA ADE
Ridderstraat 27 | 8061 GH Hasselt | The Netherlands
tel: 038 477 53 69 | www.pragma-ade.nl | www.pragma-pod.nl
-
___
If your question is of interest to others as well, please add an entry to the
Wiki!
maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context
webpage : http://www.pragma-ade.nl / http://context.aanhet.net
archive : https://bitbucket.org/phg/context-mirror/commits/
wiki : http://contextgarden.net
___
Re: [NTG-context] LMTX MkIV difference in expansion
Rik Kabel schrieb am 19.11.2020 um 21:20:
Another LMTX/MkIV difference, this time with expansion:
\define\Align{yes}
\starttext
\startalignment[\Align]
This works with MkIV but fails with LMTX, complaining: {\tt
tex error on line 3 in file G:/expand.mkvi: The file ended
when scanning an argument.}
\blank
It works in both when \tex{def} or \tex{defineexpandable} is
used instead of \tex{define}.
\blank
What changed?
\stopalignment
\stoptext
It may well be that I have been abusing some laxity in MkIV and that
LMTX is a bit stricter in what it accepts, but I would like to know if
this is an expected difference.
You have to wait for Hans to get an answer but here is a minimal example.
\starttext
\protected\def\testparameter{test}
%\def\testparameter{test}
\def\test[#1]%
{\expandafter\let\expandafter\testargumentlist\csname#1\endcsname}
\test[\testparameter]
\stoptext
Wolfgang
___
If your question is of interest to others as well, please add an entry to the
Wiki!
maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context
webpage : http://www.pragma-ade.nl / http://context.aanhet.net
archive : https://bitbucket.org/phg/context-mirror/commits/
wiki : http://contextgarden.net
___
