Since some time I am the happy user of xz to compress linux kernel (2.6.27 patched, 2.6.32 and now 3.0). All tests below were made with xz-5.0.3. Machine has 1 GB memory, x86 32 bits.
On 2.6.32, only lzma option is available and by default, kernel use lzma -9 and this require much more memory than needed. As one compiler of my distrib reported a compilation breakage on a 512 MB VM during kernel compression, I started hacking scripts/Makefile.lib, removed -9 and added -vv. I then played with information displayed during compression to adjust xz memory requirement. Using the value from linux kernel (lzma -9), adding -vv show as result lzma: Filter chain: --lzma1=dict=64MiB,lc=3,lp=0,pb=2,mode=normal,nice=64,mf=bt4,depth=0 lzma: 674 MiB of memory is required. The limit is 17592186044416 MiB. lzma: Decompression will need 65 MiB of memory. (stdin): 1413.3 KiB / 3437.4 KiB = 0.411, 741 KiB/s, 0:04 Removing -9 show as result lzma: Filter chain: --lzma1=dict=8MiB,lc=3,lp=0,pb=2,mode=normal,nice=64,mf=bt4,depth=0 lzma: 94 MiB of memory is required. The limit is 17592186044416 MiB. lzma: Decompression will need 9 MiB of memory. (stdin): 1413.3 KiB / 3437.4 KiB = 0.411, 747 KiB/s, 0:04 So still same size result with less memory requirement. What I elected to use as the data size to be compressed is know and near 3 MiB was sed -i 's/lzma -9/lzma -vv --lzma1=dict=3MiB/' scripts/Makefile.lib showing lzma: Filter chain: --lzma1=dict=3MiB,lc=3,lp=0,pb=2,mode=normal,nice=64,mf=bt4,depth=0 lzma: 38 MiB of memory is required. The limit is 17592186044416 MiB. lzma: Decompression will need 4 MiB of memory. (stdin): 1413.0 KiB / 3437.4 KiB = 0.411, 752 KiB/s, 0:04 Still same size and even less memory requirement. Should not a patch be pushed on LKLM to at least remove the -9 part? I understand it could be not easy to do more tuning like adding --lzma1=dict=<size> setting as that will not work with every lzma versions. Or a test will be required to identify lzma when xz in disguise, capable to support that option and adjust dictionary size. Secondly, could I trust the decompression memory requirement displayed by xz? Is the kernel decompressor really requiring the same memory size that xz display during compression? Gilles
