On Tue, 29 Oct 2024 at 07:49, user202729--- via luatex <luatex@tug.org>
wrote:
> I notice that repeated calls to `token.set_macro` consumes up memory, even
> though the macro themselves are freed.
>
> For example: run the following command
>
> ```
> lualatex '\directlua{for i=1,50000000 do token.set_macro("a", "")
> end}\stop' </dev/null
> ```
>
> This will consume a few GB of memory before terminating.
>
> After looking at the source code, the reason is because the reference
> count is not correctly zeroed, which leads to the token list not being
> freed when the macro is overwritten.
>
> The following patch should fix the issue.
>
>
> ```
> diff --git a/source/texk/web2c/luatexdir/lua/lnewtokenlib.c
> b/source/texk/web2c/luatexdir/lua/lnewtokenlib.c
> index 7c81e46..1b163b7 100644
> --- a/source/texk/web2c/luatexdir/lua/lnewtokenlib.c
> +++ b/source/texk/web2c/luatexdir/lua/lnewtokenlib.c
> @@ -1265,9 +1265,9 @@ static int set_macro(lua_State * L)
> const char *se = str + lstr;
> p = temp_token_head;
> set_token_link(p, null);
> - /* this left brace is used to store the number of arguments */
> - fast_store_new_token(left_brace_token);
> - /* and this ends the not present arguments, and no: we will not
> support arguments here*/
> + /* reference count */
> + fast_store_new_token(0);
> + /* this ends the not present arguments, and no: we will not
> support arguments here*/
> fast_store_new_token(end_match_token);
> while (str < se) {
> /* hh: str2uni could return len too (also elsewhere) */
> @@ -1334,7 +1334,7 @@ static int set_macro(lua_State * L)
> halfword q; /* new node being added to the token list via
> |store_new_token| */
> p = temp_token_head;
> set_token_info(p,null);
> - fast_store_new_token(left_brace_token);
> + fast_store_new_token(0);
> fast_store_new_token(end_match_token);
> define(cs, call_cmd + (a % 4), token_link(temp_token_head));
> }
> ```
>
> Please take a look. Thank you.
>
>
commit cd4888ec08e37cf0beae04be0a904d1df17eac41
Author: Luigi Scarso <luigi.sca...@gmail.com>
Date: Sun Nov 3 13:17:40 2024 +0100
Fixed a memory leak in token.put_next() and token.set_macro() (thanks
to user202...@protonmail.com)
(also in texlive)
--
luigi
