Re: [NTG-context] Bug: Segmentation fault

2019-12-11 Thread luigi scarso 
On Wed, Dec 11, 2019 at 12:10 AM Hans Hagen  wrote:

> On 12/11/2019 12:03 AM, luigi scarso wrote:
> > I guess that we have to check if catcodetable -1 is already defined...
> i already sent you a patch, probbaly also ok for trunk
>

Committed revision 7235.
-- 
luigi
___
If your question is of interest to others as well, please add an entry to the 
Wiki!

maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context
webpage  : http://www.pragma-ade.nl / http://context.aanhet.net
archive  : https://bitbucket.org/phg/context-mirror/commits/
wiki : http://contextgarden.net
___

Re: [NTG-context] Bug: Segmentation fault

2019-12-11 Thread Hans Hagen 

On 12/11/2019 1:28 AM, Henri Menke wrote:


$ luatex <(echo '\relax\directlua{print("\string\\,")}\bye')
This is LuaTeX, Version 1.11.2 (TeX Live 2020/dev)
  restricted system commands enabled.
(/dev/fd/63\,
)


I don't even want to think about what piping adds to this (some command 
line processing).


Hans

-
  Hans Hagen | PRAGMA ADE
  Ridderstraat 27 | 8061 GH Hasselt | The Netherlands
   tel: 038 477 53 69 | www.pragma-ade.nl | www.pragma-pod.nl
-
___
If your question is of interest to others as well, please add an entry to the 
Wiki!

maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context
webpage  : http://www.pragma-ade.nl / http://context.aanhet.net
archive  : https://bitbucket.org/phg/context-mirror/commits/
wiki : http://contextgarden.net
___

Re: [NTG-context] Bug: Segmentation fault

2019-12-11 Thread luigi scarso 
On Wed, Dec 11, 2019 at 1:28 AM Henri Menke  wrote:

> In this case I wanted to generate the thin space (\,).  My expectation
> (which worked elsewhere) was that full expansion will turn "\string\\,"
> into "\\," where the two backslashes have catcode 12 and will not be
> expanded further.  Then the Lua string will turn the escape sequence \\
> into a single backslash, therefore ending up with \, in the Lua string.
>
>


\directlua{token.set_macro("foo","\relax")}
(gdb) x/7c str
0x59376c18: 13 '\r' 101 'e' 108 'l' 97 'a'  120 'x' 32 ' '  0 '\000'
(gdb) p lstr
$24 = 6
where \foo is
 \foo=macro:
->#!.

vs
\directlua{token.set_macro("foo",[==[\relax]==])}
(gdb) x/8c str
0x59376ea8: 92 '\\' 114 'r' 101 'e' 108 'l' 97 'a'  120 'x' 32 ' '  0
'\000'
(gdb) p lstr
$25 = 7
\foo=macro:
->\relax .

Another example is
\directlua{token.set_macro("foo","\medskip")}\show\foo
which gives the error:   invalid escape sequence near '"^^Kskip \m'.
vs
\directlua{token.set_macro("foo",[==[\medskip]==])}\show\foo
 which gives
\foo=macro:
->\vskip \medskipamount .

Your example:
\directlua{token.set_macro("foo","\string\\,")}\show\foo
gives
(gdb) x/3c str
0x593765a8: 92 '\\' 44 ','  0 '\000'
(gdb) p lstr
$29 = 2
 \foo=macro:
->BAD,.

vs
\directlua{token.set_macro("foo",[==[\,]==])}\show\foo
(gdb) x/20c str
0x59376908: 92 '\\' 109 'm' 115 's' 107 'k' 105 'i' 112 'p' 32 ' '  92
'\\'
0x59376910: 116 't' 104 'h' 105 'i' 110 'n' 109 'm' 117 'u' 115 's' 107
'k'
0x59376918: 105 'i' 112 'p' 32 ' '  0 '\000'
(gdb) p lstr
$31 = 19
 \foo=macro:
->\mskip \thinmuskip .


-- 
luigi
___
If your question is of interest to others as well, please add an entry to the 
Wiki!

maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context
webpage  : http://www.pragma-ade.nl / http://context.aanhet.net
archive  : https://bitbucket.org/phg/context-mirror/commits/
wiki : http://contextgarden.net
___

Re: [NTG-context] Bug: Segmentation fault

2019-12-10 Thread Henri Menke 

On 12/11/19 12:23 PM, luigi scarso wrote:

On Wed, Dec 11, 2019 at 12:10 AM Hans Hagen  wrote:


On 12/11/2019 12:03 AM, luigi scarso wrote:



On Tue, Dec 10, 2019 at 9:27 PM Henri Menke mailto:henrime...@gmail.com>> wrote:

 Dear devs,

 The example below crashes LuaTeX with a segmentation fault.  It is
 reproducible
 with LuaTeX 1.10.1 7143 and LuaTeX 1.11.2 7226.  To this end I built
 7226 from
 source with --debugopt to get a backtrace, which can be found below
 as well.

 Cheers, Henri

 ---

 \directlua{token.set_macro(-1, "foo","\string\\,")}
 \show\foo
 \foo
 \bye


I guess that we have to check if catcodetable -1 is already defined...

i already sent you a patch, probbaly also ok for trunk




yes, seen




but one needs to be careful as fo rinstance \string\foo is seen as
oo so often one also needs to escape properly



  I always start with  [==[ ... ]==]  ie
\directlua{token.set_macro(-1, [==[foo]==],[==[\string\\,]==])}
  gdb reports this [==[\string\\,]==] string as
  ","
which is an array of char of length 3 ie {'\\','\\',','}

With \directlua{token.set_macro(-1, [==[foo]==],"\string\\,")}
gdb reports "\\,"
ie {'\\','\,'}

I always check which one is the correct one...


In this case I wanted to generate the thin space (\,).  My expectation
(which worked elsewhere) was that full expansion will turn "\string\\,"
into "\\," where the two backslashes have catcode 12 and will not be
expanded further.  Then the Lua string will turn the escape sequence \\
into a single backslash, therefore ending up with \, in the Lua string.
Demo below.

Cheers, Henri


$ luatex <(echo '\relax\directlua{print("\string\\,")}\bye')
This is LuaTeX, Version 1.11.2 (TeX Live 2020/dev)
 restricted system commands enabled.
(/dev/fd/63\,
)
warning  (pdf backend): no pages of output.
Transcript written on 63.log.




___
If your question is of interest to others as well, please add an entry to the 
Wiki!

maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context
webpage  : http://www.pragma-ade.nl / http://context.aanhet.net
archive  : https://bitbucket.org/phg/context-mirror/commits/
wiki : http://contextgarden.net
___


___
If your question is of interest to others as well, please add an entry to the 
Wiki!

maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context
webpage  : http://www.pragma-ade.nl / http://context.aanhet.net
archive  : https://bitbucket.org/phg/context-mirror/commits/
wiki : http://contextgarden.net
___

Re: [NTG-context] Bug: Segmentation fault

2019-12-10 Thread luigi scarso 
On Wed, Dec 11, 2019 at 12:10 AM Hans Hagen  wrote:

> On 12/11/2019 12:03 AM, luigi scarso wrote:
> >
> >
> > On Tue, Dec 10, 2019 at 9:27 PM Henri Menke  > > wrote:
> >
> > Dear devs,
> >
> > The example below crashes LuaTeX with a segmentation fault.  It is
> > reproducible
> > with LuaTeX 1.10.1 7143 and LuaTeX 1.11.2 7226.  To this end I built
> > 7226 from
> > source with --debugopt to get a backtrace, which can be found below
> > as well.
> >
> > Cheers, Henri
> >
> > ---
> >
> > \directlua{token.set_macro(-1, "foo","\string\\,")}
> > \show\foo
> > \foo
> > \bye
> >
> >
> > I guess that we have to check if catcodetable -1 is already defined...
> i already sent you a patch, probbaly also ok for trunk
>


yes, seen



> but one needs to be careful as fo rinstance \string\foo is seen as
> oo so often one also needs to escape properly
>

 I always start with  [==[ ... ]==]  ie
\directlua{token.set_macro(-1, [==[foo]==],[==[\string\\,]==])}
 gdb reports this [==[\string\\,]==] string as
 ","
which is an array of char of length 3 ie {'\\','\\',','}

With \directlua{token.set_macro(-1, [==[foo]==],"\string\\,")}
gdb reports "\\,"
ie {'\\','\,'}

I always check which one is the correct one...

-- 
luigi
___
If your question is of interest to others as well, please add an entry to the 
Wiki!

maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context
webpage  : http://www.pragma-ade.nl / http://context.aanhet.net
archive  : https://bitbucket.org/phg/context-mirror/commits/
wiki : http://contextgarden.net
___

Re: [NTG-context] Bug: Segmentation fault

2019-12-10 Thread Hans Hagen 

On 12/11/2019 12:03 AM, luigi scarso wrote:



On Tue, Dec 10, 2019 at 9:27 PM Henri Menke > wrote:


Dear devs,

The example below crashes LuaTeX with a segmentation fault.  It is
reproducible
with LuaTeX 1.10.1 7143 and LuaTeX 1.11.2 7226.  To this end I built
7226 from
source with --debugopt to get a backtrace, which can be found below
as well.

Cheers, Henri

---

\directlua{token.set_macro(-1, "foo","\string\\,")}
\show\foo
\foo
\bye


I guess that we have to check if catcodetable -1 is already defined...

i already sent you a patch, probbaly also ok for trunk

but one needs to be careful as fo rinstance \string\foo is seen as 
oo so often one also needs to escape properly


Hans


-
  Hans Hagen | PRAGMA ADE
  Ridderstraat 27 | 8061 GH Hasselt | The Netherlands
   tel: 038 477 53 69 | www.pragma-ade.nl | www.pragma-pod.nl
-
___
If your question is of interest to others as well, please add an entry to the 
Wiki!

maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context
webpage  : http://www.pragma-ade.nl / http://context.aanhet.net
archive  : https://bitbucket.org/phg/context-mirror/commits/
wiki : http://contextgarden.net
___

Re: [NTG-context] Bug: Segmentation fault

2019-12-10 Thread luigi scarso 
On Tue, Dec 10, 2019 at 9:27 PM Henri Menke  wrote:

> Dear devs,
>
> The example below crashes LuaTeX with a segmentation fault.  It is
> reproducible
> with LuaTeX 1.10.1 7143 and LuaTeX 1.11.2 7226.  To this end I built 7226
> from
> source with --debugopt to get a backtrace, which can be found below as
> well.
>
> Cheers, Henri
>
> ---
>
> \directlua{token.set_macro(-1, "foo","\string\\,")}
> \show\foo
> \foo
> \bye
>

I guess that we have to check if catcodetable -1 is already defined...

--
luigi
___
If your question is of interest to others as well, please add an entry to the 
Wiki!

maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context
webpage  : http://www.pragma-ade.nl / http://context.aanhet.net
archive  : https://bitbucket.org/phg/context-mirror/commits/
wiki : http://contextgarden.net
___

Re: [NTG-context] Bug: Segmentation fault

2019-12-10 Thread Hans Hagen 

On 12/10/2019 9:26 PM, Henri Menke wrote:

Dear devs,

The example below crashes LuaTeX with a segmentation fault.  It is 
reproducible
with LuaTeX 1.10.1 7143 and LuaTeX 1.11.2 7226.  To this end I built 
7226 from
source with --debugopt to get a backtrace, which can be found below as 
well.

i see what's going on ... more later


-
  Hans Hagen | PRAGMA ADE
  Ridderstraat 27 | 8061 GH Hasselt | The Netherlands
   tel: 038 477 53 69 | www.pragma-ade.nl | www.pragma-pod.nl
-
___
If your question is of interest to others as well, please add an entry to the 
Wiki!

maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context
webpage  : http://www.pragma-ade.nl / http://context.aanhet.net
archive  : https://bitbucket.org/phg/context-mirror/commits/
wiki : http://contextgarden.net
___

Re: [NTG-context] Bug: Segmentation fault

2019-12-10 Thread luigi scarso 
On Tue, Dec 10, 2019 at 9:27 PM Henri Menke  wrote:

> Dear devs,
>
> The example below crashes LuaTeX with a segmentation fault.  It is
> reproducible
> with LuaTeX 1.10.1 7143 and LuaTeX 1.11.2 7226.  To this end I built 7226
> from
> source with --debugopt to get a backtrace, which can be found below as
> well.
>
>
confirmed, we will see asap
(btw,  I use --debug ie CFLAGS="-g -O0" because it's hard to follow the
flow of optimized code)


-- 
luigi
___
If your question is of interest to others as well, please add an entry to the 
Wiki!

maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context
webpage  : http://www.pragma-ade.nl / http://context.aanhet.net
archive  : https://bitbucket.org/phg/context-mirror/commits/
wiki : http://contextgarden.net
___

[NTG-context] Bug: Segmentation fault

2019-12-10 Thread Henri Menke 

Dear devs,

The example below crashes LuaTeX with a segmentation fault.  It is reproducible
with LuaTeX 1.10.1 7143 and LuaTeX 1.11.2 7226.  To this end I built 7226 from
source with --debugopt to get a backtrace, which can be found below as well.

Cheers, Henri

---

\directlua{token.set_macro(-1, "foo","\string\\,")}
\show\foo
\foo
\bye

---

(gdb) run
Starting program: /usr/local/bin/luatex test.tex
This is LuaTeX, Version 1.11.2 (TeX Live 2020/dev)
 restricted system commands enabled.
(./test.tex
Program received signal SIGSEGV, Segmentation fault.
get_sa_item (head=0x41002, n=n@entry=92) at 
../../../source/texk/web2c/luatexdir/utils/managed-sa.c:65
65  if (head->tree != NULL) {
(gdb) bt full
#0  get_sa_item (head=0x41002, n=n@entry=92) at 
../../../source/texk/web2c/luatexdir/utils/managed-sa.c:65
No locals.
#1  0x005607d0 in get_cat_code (h=h@entry=-1, n=n@entry=92) at 
../../../source/texk/web2c/luatexdir/tex/textcodes.c:68
sa_value = 
s = 
#2  0x004d8a8c in set_macro (L=) at 
../../../source/texk/web2c/luatexdir/lua/lnewtokenlib.c:1204
p = 
q = 
t = 
se = 0x295bf9a ""
name = 
str = 
s = 
lname = 3
lstr = 2
cs = 743
cc = 
ct = -1
n = 
a = 0
nncs = 
#3  0x006dc924 in luaD_precall (L=L@entry=0x1202678, 
func=func@entry=0x2214490, nresults=nresults@entry=0)
at ../../../source/libs/lua53/lua53-src/src/ldo.c:434
n = 
f = 0x4d8890 
ci = 0x123c830
#4  0x006f62c2 in luaV_execute (L=L@entry=0x1202678) at 
../../../source/libs/lua53/lua53-src/src/lvm.c:1134
b = 
nresults = 0
i = 
ra = 0x2214490
ci = 0x12221c0
cl = 0x2952f00
k = 0x294ea40
base = 
#5  0x006dcef7 in luaD_call (nResults=, func=, L=0x1202678) at ../../../source/libs/lua53/lua53-src/src/ldo.c:499
No locals.
#6  luaD_callnoyield (L=0x1202678, func=, nResults=) at ../../../source/libs/lua53/lua53-src/src/ldo.c:509
No locals.
#7  0x006db78c in luaD_rawrunprotected (L=0x1202678, f=0x6d0800 
, ud=0x7fffdf80) at 
../../../source/libs/lua53/lua53-src/src/ldo.c:142
oldnCcalls = 0
lj = {previous = 0x0, b = {{__jmpbuf = {832, -3576387603206626035, 
18884312, 1, 18884216, 0, 3576387604335174925, -3576387282536356595},
  __mask_was_saved = 0, __saved_mask = {__val = {0, 18884312, 
18884216, 7198250, 35733824, 72057594037927937, 140737488346976, 0, 0, 0, 0, 0, 
0,
  0, 0, 0, status = 0}
#8  0x006dd3af in luaD_pcall (L=L@entry=0x1202678, func=func@entry=0x6d0800 
, u=u@entry=0x7fffdf80, old_top=832, ef=)
at ../../../source/libs/lua53/lua53-src/src/ldo.c:729
status = 
old_ci = 0x12026d8
old_allowhooks = 1 '\001'
old_nny = 1
old_errfunc = 0
oldtop = 
#9  0x006d3907 in lua_pcallk (L=0x1202678, nargs=nargs@entry=0, 
nresults=nresults@entry=0, errfunc=errfunc@entry=51, ctx=ctx@entry=0, 
k=k@entry=0x0)
at ../../../source/libs/lua53/lua53-src/src/lapi.c:969
c = {func = 0x2214480, nresults = 0}
status = 
func = 
#10 0x00463b5f in luatokencall (p=p@entry=9030, 
nameptr=nameptr@entry=0) at 
../../../source/texk/web2c/luatexdir/lua/luastuff.c:653
base = 51
ls = {s = 0x295e890 "\240܂\367\377\177", size = 0}
i = 0
l = 32
s = 0x0
lua_id = 
stacktop = 50
#11 0x00567a83 in conv_toks () at 
../../../source/texk/web2c/luatexdir/tex/textoken.c:2859
old_setting = 
p = 
q = 
save_scanner_status = 0
save_def_ref = 0
save_warning_index = 0
bool = 
s = 9030
sn = 0
u = 0
c = 
str = 
i = 0
#12 0x00503a25 in expand () at 
../../../source/texk/web2c/luatexdir/tex/expand.c:259
t = 
p = 
---Type  to continue, or q  to quit---
cur_ptr = 
cv_backup = 0
cvl_backup = 0
radix_backup = 0
co_backup = 0
backup_backup = 7961
save_scanner_status = 
#13 0x00503f85 in get_x_token () at 
../../../source/texk/web2c/luatexdir/tex/expand.c:501
No locals.
#14 0x00513ae5 in main_control () at 
../../../source/texk/web2c/luatexdir/tex/maincontrol.c:1014
No locals.
#15 0x0050f4ac in main_body () at 
../../../source/texk/web2c/luatexdir/tex/mainbody.c:567
pdftex_map = "pdftex.map"
bad = 
#16 0x00446d2e in main (ac=, av=) at 
../../../source/texk/web2c/luatexdir/luatex.c:609
No locals.
___
If your question is of interest to others as well, please add an entry to the 
Wiki!

maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context
webpage  : http://www.pragma-ade.nl / http://context.aan