On 12/22/2023 1:21 AM, Reinhard Kotucha via luatex wrote:
On 2023-12-21 at 10:27:48 +0100, Hans Hagen wrote:
> I stop here as I've written plenty about performance the last
> decades in various docs,
Hi Hans,
please let me mention this though:
A few months ago I uncompressed the lualatex format file and
compressed it with lz4. The lz4-compressed file was smaller than the
original format file. Though I know that luatex doesn't use maximum
gzip compression, I didn't really expect this.
26702648 lualatex.fmt-uncompressed
12201683 lualatex.fmt
9926586 lualatex.fmt.lz4
But what I absolutely didn't expect is that de-compression of lz4
files is about 7.5 times as fast as de-compression of the gzip'ed
format file.
In order to determine speed I directed the de-compressed output to
/dev/null because when writing to disk results differ much more
between successive runs and a format file is extracted in memory and
not written to disk.
Maybe it's worthwhile to keep an eye on lz4.
Ok, it's a bit off topic but ... in luametatex (context lmtx) we don't
zip the format at all. One reason is that is not really needed because
the format file smaller and also because there is that little overhead
involved (actually endian swapping also takes time but in a thread on
this list a while ago that was not deemed relevant).
Anyway, I played with (and interface to) zip (gz), lzo, lz4 and zstd for
various purposes (one is that i can have a real fast texlive files
uncompression when i need to see what is in these xz files, esp when it
comes to fonts).
cont-en.fmt:
uncompressed gzip -3 format
luametatex: 19,387,682 6,483,137 (we don't compress)
luatex : 19,819,371 6,281,775 11,433,340
(there is more code in luametatex but we save some elsewhere in tex)
So, actually luates doesn't compress agressive (11 vs 6 MB). I have no
clue how large other macro package format files are (compressed or
uncompressed).
(bit as mentioned, in earlier threads about performance formats when i
mentioned formats it was concluded that that had little impact)
it's very unlikely that we will change luatex in this respect,
Hans
-----------------------------------------------------------------
Hans Hagen | PRAGMA ADE
Ridderstraat 27 | 8061 GH Hasselt | The Netherlands
tel: 038 477 53 69 | www.pragma-ade.nl | www.pragma-pod.nl
-----------------------------------------------------------------
