On 9/15/2026 7:16 AM, Max Chernoff via ntg-context wrote:
Hi autumnus, Hans,
On Mon, 2026-09-14 at 07:36 +0000, autumnus wrote:
Moreover, even after the cache has been generated,
each compilation takes far longer than before.
Yes, the 2026.09.07 update changed "data-tld.lua" from the equivalent of
local delta = os.difftime(lsrtime,hashtime)
if delta > 0 then
function_one()
else
function_two()
end
to
local delta = math.abs(os.difftime(lsrtime,hashtime))
if delta > 0 then
function_one()
else
function_two()
end
Since the absolute value is always non-negative, this means that the
first branch is always taken, which means that ConTeXt regenerates its
filename database on every compile. I can confirm that removing
"math.abs" fixes the problem.
Hm I see no function_one and last time I checked it the approach worked
ok here. Anyway, I'll remove the abs (until of course i see it failing
here) till we get another issue.
Hans: ConTeXt shouldn't ever need to call "mktexlsr", since TeX Live
will always keep its filename database up-to-date whenever packages are
updated. Furthermore, "mktexlsr" usually won't do anything at all here,
since users typically run ConTeXt as an unprivileged user, but
$TEXMFDIST is usually owned by root, so it will skip updating it to
avoid permission errors.
Well, I need the mktexlsr when I test here (some time ago Mikael and I
were testing some luatex patch in tl). Compared to compiling luatex and
then remaking all these formats the extra second is irrelevant. (I admit
that I only use tex live a few times per year for that so I don't
qualify as sample user.)
Also, a few other comments:
if string.find(hashes[i].name,"texmf%-dist") then
Lots of Linux distros (downstream of TL) install things into
"/usr/share/texmf/", so this check will miss those cases. The best way
to check for TL is to read the "texmfcnf.lua" file, which for anything
downstream of TL should always look something like:
So we should also check for /share/texmf then which is easier. I'll add
that.
return {
type = "configuration",
version = "1.1.3",
[...]
target = "texlive",
details = {
[...]
comment = "Based off of TeX Live (2026-09-10 10:02 A)",
private = {
[...]
derived_from = "TeX Live (2026-09-10 10:02 A)",
}
},
}
Or if reading the "texmfcnf.lua" file won't work here, I can also add
some other marker file, or write something to "mtxrun.lua" if that would
make things easier for you.
I'm not going to complicate my tex live even more.
Also, regarding the comments in "m-texlive.mkxl":
%D I guess that the reason for this hackery is that a \TEXLIVE\ doesn't do
%D an update of the file database after installing some \CONTEXT\ related
%D files. In a similar manner, when a used updated something \CONTEXT\ we
%D don't generate an lsr file. Here both are checked.
So, the actual problem here is that most users install TL as root to
"/usr/local/texlive", but then run ConTeXt as an unprivileged user
_without_ write access to "/usr/", so ConTeXt falls back to writing its
caches to "~/.texlive2026".
Ok, so:
%D don't generate an lsr file. Here both are checked. It can also be that
%D a user installs \TEX\ as root and therefore other stuff ends up in user
%D space. Whatever.
> Previously, "tlmgr" would run "mtxrun --generate" whenever any files
were updated, and "context --make" whenever the ConTeXt core files were
updated, and these commands would both write their updated files to
"/usr/local/texlive". But I had Karl remove this code from "tlmgr"
because ConTeXt (when ran as an unprivileged user) was completely
ignoring these files, and instead using the user-level files in
"~/.texlive2026", even when they were out-of-date.
So to prevent users from accidentally using out-of-date files, I added
(the initial version of) "resolvers.checktexlive()" to TL to ensure that
the user-level were always up-to-date. But this isn't ideal, since it
means that users will have to wait for ConTeXt to remake its formats
whenever the update TL (and I personally update TL every day, so this
happens quite often).
I wonder about the fuzz of remaking formats: any decent operating system
caches files, so whatever runs first (mktexlsr, mtxrun, context --make)
carries the burden of populating that cache (and later run tex programs
benefit from it).
I'm getting tired of these 'costs time' discussions esp because when I
update tl it takes ages for other formats to build without even being
able to see what happens while my system freezes for minutes. If someone
updates, one pays a little price. If I'd really noticed a bottleneck,
we'd taken care of it. In fact, we always did, even when a simple
mktexlsr or kpsewhich was adding significanet runtime (that we worked
around).
(FWIW: most discussions about (run)timing in tex are kind of weird and
naive.)
I did submit a patch last year to try and fix this
https://mailman.ntg.nl/archives/list/[email protected]/thread/MLMF5WHFVC6ZXBERNKPD4FSYMKV57HCL/#7GM73WIWMDDV74E2BUNQXBZ4R7ZL35L6
but the implementation was pretty ugly. I have heard quite a few users
complain about this though, so I think that we should try and figure out
some other solution. Let's start with things that I _can't_ do:
You mean reading some lsr files? Indeed, gorget about it. Gains nothing
anyway.
1. I can't reduce the total number of files in TL. I also think that it
installs way too many files by default, but this is completely out of
my control.
Well, such it it, right? So a user pays a little price. When I start up
a WSL instance and run "context --generate" I get:
real 0m9.738s
user 0m2.300s
sys 0m2.574s
second time (populated file cache):
real 0m2.900s
user 0m2.253s
sys 0m0.650s
and after that mkltexlsr thansk to the populated file cache:
real 0m0.727s
user 0m0.559s
sys 0m0.391s
So, what are we talking about. Broken operating systems? Weird file
systenms? Bad installations? Impatient users? On a more modern system
these times get close to zero.
And if one loves to update context frequently using the small
installation makes sense.
(How many is 'quite some users' ... real users? Or latex users who
torture themselves by also installing something they don't use or like
to complain about?)
2. There is no way to have "tlmgr" when ran as root update any
user-level files: many systems have hundreds of users, so it's
impossible for "tlmgr" to try and process hundreds of
"~/.texlive2026" files.
Things that I _can_ do:
1. I can arrange for "tlmgr" to call "context --generate" or any other
command of your choosing (as root) whenever it updates any files in
$TEXMFDIST.
Makes sense, just do it after the mktexlsr calls and the system cache is
populated so that mktexlsr takes the hickups.
2. I can arrange for "tlmgr" to call "context --make" or any other
command of your choosing (as root) whenever it updates any of the
core ConTeXt files or LuaMetaTeX itself. (This is the only way that
format files are updated for the other TeX formats/engines, so you
can count on this being 99.9% reliable)
Normally context will make a new format itself when it notices a change
(update) .. I have no clue how that works on tl.
3. I can add any settings you want to the "texmfcnf.lua" file in TL.
Dunno, here it seems to work ok on my installed tl anyway. I don't want
to keep up with changes in tex live setups. I assume that after decades
(moving from tetex / fptex / gwtex onwards) things have settled by now.
4. I can write and maintain whatever TeX/Lua code is necessary to
integrate this into ConTeXt/TL (as long suitable hooks are available,
since I don't want to patch core files).
Possible solutions:
1. ConTeXt could be configured to _exclusively_ use the format
files/filename databases stored in "/usr/local/texlive". Then running
"context --[make|generate]" as an unprivileged user would fail, but
this would be fine because "tlmgr" would ensure that the files in
"/usr/local/texlive" are _always_ up-to-date.
Maybe. I have no opinion on that. Here I never expect something tex
there, in fact: /data/texlive/2026 it is for me.
2. When running "context --[make|generate]" as an unprivileged user,
ConTeXt could be configured to copy the files from
"/usr/local/texlive" to "~/.texlive2026" if they are newer. This way,
the user-level caches would always be up-to-date, but users wouldn't
need to wait for ConTeXt to remake the formats since copying should
be instantaneous.
Dunno, I assume things like that don't need adaptation at our end.
3. When running "context --generate", ConTeXt could be configured to
parse TL's "ls-R" files instead of walking the filesystem, which
should be much faster since it won't need to do as much disk I/O.
Sounda crazy to me. Why parse these files if we can just glob the
directories instead. Fast enough. Why bother a about a few seconds.
Updating (installing) texlive takes ages so what are we talking about.
4. Some other fourth solution that I haven't thought of.
Anyways, please let me know if you have any questions, if you want me to
go ahead and implement one of those solutions, or if you want to suggest
something else entirely.
(And sorry about not replying to some of your earlier messages; I've
been fairly busy recently with the start of the school year.)
that's definitely more important
Hans
-----------------------------------------------------------------
Hans Hagen | PRAGMA ADE
Ridderstraat 27 | 8061 GH Hasselt | The Netherlands
tel: 038 477 53 69 | www.pragma-ade.nl | www.pragma-pod.nl
-----------------------------------------------------------------
___________________________________________________________________________________
If your question is of interest to others as well, please add an entry to the
Wiki!
maillist : [email protected] /
https://mailman.ntg.nl/mailman3/lists/ntg-context.ntg.nl
webpage : https://www.pragma-ade.nl / https://context.aanhet.net (mirror)
archive : https://github.com/contextgarden/context
wiki : https://wiki.contextgarden.net
___________________________________________________________________________________
