Hi autumnus, Hans,
On Mon, 2026-09-14 at 07:36 +0000, autumnus wrote:
> Moreover, even after the cache has been generated,
> each compilation takes far longer than before.
Yes, the 2026.09.07 update changed "data-tld.lua" from the equivalent of
local delta = os.difftime(lsrtime,hashtime)
if delta > 0 then
function_one()
else
function_two()
end
to
local delta = math.abs(os.difftime(lsrtime,hashtime))
if delta > 0 then
function_one()
else
function_two()
end
Since the absolute value is always non-negative, this means that the
first branch is always taken, which means that ConTeXt regenerates its
filename database on every compile. I can confirm that removing
"math.abs" fixes the problem.
Hans: ConTeXt shouldn't ever need to call "mktexlsr", since TeX Live
will always keep its filename database up-to-date whenever packages are
updated. Furthermore, "mktexlsr" usually won't do anything at all here,
since users typically run ConTeXt as an unprivileged user, but
$TEXMFDIST is usually owned by root, so it will skip updating it to
avoid permission errors.
Also, a few other comments:
if string.find(hashes[i].name,"texmf%-dist") then
Lots of Linux distros (downstream of TL) install things into
"/usr/share/texmf/", so this check will miss those cases. The best way
to check for TL is to read the "texmfcnf.lua" file, which for anything
downstream of TL should always look something like:
return {
type = "configuration",
version = "1.1.3",
[...]
target = "texlive",
details = {
[...]
comment = "Based off of TeX Live (2026-09-10 10:02 A)",
private = {
[...]
derived_from = "TeX Live (2026-09-10 10:02 A)",
}
},
}
Or if reading the "texmfcnf.lua" file won't work here, I can also add
some other marker file, or write something to "mtxrun.lua" if that would
make things easier for you.
Also, regarding the comments in "m-texlive.mkxl":
%D I guess that the reason for this hackery is that a \TEXLIVE\ doesn't do
%D an update of the file database after installing some \CONTEXT\ related
%D files. In a similar manner, when a used updated something \CONTEXT\ we
%D don't generate an lsr file. Here both are checked.
So, the actual problem here is that most users install TL as root to
"/usr/local/texlive", but then run ConTeXt as an unprivileged user
_without_ write access to "/usr/", so ConTeXt falls back to writing its
caches to "~/.texlive2026".
Previously, "tlmgr" would run "mtxrun --generate" whenever any files
were updated, and "context --make" whenever the ConTeXt core files were
updated, and these commands would both write their updated files to
"/usr/local/texlive". But I had Karl remove this code from "tlmgr"
because ConTeXt (when ran as an unprivileged user) was completely
ignoring these files, and instead using the user-level files in
"~/.texlive2026", even when they were out-of-date.
So to prevent users from accidentally using out-of-date files, I added
(the initial version of) "resolvers.checktexlive()" to TL to ensure that
the user-level were always up-to-date. But this isn't ideal, since it
means that users will have to wait for ConTeXt to remake its formats
whenever the update TL (and I personally update TL every day, so this
happens quite often).
I did submit a patch last year to try and fix this
https://mailman.ntg.nl/archives/list/[email protected]/thread/MLMF5WHFVC6ZXBERNKPD4FSYMKV57HCL/#7GM73WIWMDDV74E2BUNQXBZ4R7ZL35L6
but the implementation was pretty ugly. I have heard quite a few users
complain about this though, so I think that we should try and figure out
some other solution. Let's start with things that I _can't_ do:
1. I can't reduce the total number of files in TL. I also think that it
installs way too many files by default, but this is completely out of
my control.
2. There is no way to have "tlmgr" when ran as root update any
user-level files: many systems have hundreds of users, so it's
impossible for "tlmgr" to try and process hundreds of
"~/.texlive2026" files.
Things that I _can_ do:
1. I can arrange for "tlmgr" to call "context --generate" or any other
command of your choosing (as root) whenever it updates any files in
$TEXMFDIST.
2. I can arrange for "tlmgr" to call "context --make" or any other
command of your choosing (as root) whenever it updates any of the
core ConTeXt files or LuaMetaTeX itself. (This is the only way that
format files are updated for the other TeX formats/engines, so you
can count on this being 99.9% reliable)
3. I can add any settings you want to the "texmfcnf.lua" file in TL.
4. I can write and maintain whatever TeX/Lua code is necessary to
integrate this into ConTeXt/TL (as long suitable hooks are available,
since I don't want to patch core files).
Possible solutions:
1. ConTeXt could be configured to _exclusively_ use the format
files/filename databases stored in "/usr/local/texlive". Then running
"context --[make|generate]" as an unprivileged user would fail, but
this would be fine because "tlmgr" would ensure that the files in
"/usr/local/texlive" are _always_ up-to-date.
2. When running "context --[make|generate]" as an unprivileged user,
ConTeXt could be configured to copy the files from
"/usr/local/texlive" to "~/.texlive2026" if they are newer. This way,
the user-level caches would always be up-to-date, but users wouldn't
need to wait for ConTeXt to remake the formats since copying should
be instantaneous.
3. When running "context --generate", ConTeXt could be configured to
parse TL's "ls-R" files instead of walking the filesystem, which
should be much faster since it won't need to do as much disk I/O.
4. Some other fourth solution that I haven't thought of.
Anyways, please let me know if you have any questions, if you want me to
go ahead and implement one of those solutions, or if you want to suggest
something else entirely.
(And sorry about not replying to some of your earlier messages; I've
been fairly busy recently with the start of the school year.)
Thanks,
-- Max
___________________________________________________________________________________
If your question is of interest to others as well, please add an entry to the
Wiki!
maillist : [email protected] /
https://mailman.ntg.nl/mailman3/lists/ntg-context.ntg.nl
webpage : https://www.pragma-ade.nl / https://context.aanhet.net (mirror)
archive : https://github.com/contextgarden/context
wiki : https://wiki.contextgarden.net
___________________________________________________________________________________
