Thanks for doing the research.
What was the build time on the zstd L19 compression vs. xz? Was there
still an improvement?
On 9/28/22 11:50, Etienne Cordonnier wrote:
I tested it a bit more today. I used the standard poky's local.conf
and added those lines. You can see that at zstd level 9 there is still
a significant difference in the compression ratio with xz for gcc-dbg
which is a big file. At zstd level 19, gcc-dbg is 241MB big instead of
214MB with xz (12.6% more).
PACKAGE_CLASSES = "package_ipk"
ZSTD_DEFAULTS = "-T0 -19"
PACKAGECONFIG:append:pn-opkg-native = " zstd"
OPKGBUILDCMD = "opkg-build -Z zstd -a ${ZSTD_DEFAULTS}"
Zstd at level 9:
build$ ls -lh tmp-zstd-level9/deploy/ipk/core2-64/ --sort=size | head
-n10
total 1.5G
-rw-r--r-- 3 ecordonnier ecordonnier 302M Sep 28 18:33
gcc-dbg_12.2.0-r0_core2-64.ipk
-rw-r--r-- 3 ecordonnier ecordonnier 249M Sep 28 18:33
openssl-dbg_3.0.5-r0_core2-64.ipk
-rw-r--r-- 3 ecordonnier ecordonnier 106M Sep 28 18:33
openssl-ptest_3.0.5-r0_core2-64.ipk
-rw-r--r-- 3 ecordonnier ecordonnier 44M Sep 28 18:33
binutils-dbg_2.39-r0_core2-64.ipk
-rw-r--r-- 3 ecordonnier ecordonnier 26M Sep 28 18:33
perl-ptest_5.36.0-r0_core2-64.ipk
-rw-r--r-- 3 ecordonnier ecordonnier 23M Sep 28 18:33
gcc_12.2.0-r0_core2-64.ipk
-rw-r--r-- 3 ecordonnier ecordonnier 15M Sep 28 18:32
elfutils-ptest_0.187-r0_core2-64.ipk
-rw-r--r-- 3 ecordonnier ecordonnier 15M Sep 28 18:33
gcc-src_12.2.0-r0_core2-64.ipk
-rw-r--r-- 3 ecordonnier ecordonnier 13M Sep 28 18:33
coreutils-dbg_9.1-r0_core2-64.ipk
zstd at level 19:
build$ ls -lh tmp-zstd-level19/deploy/ipk/core2-64/ --sort=size |
head -n10
total 1.1G
-rw-r--r-- 3 ecordonnier ecordonnier 241M Sep 28 18:04
gcc-dbg_12.2.0-r0_core2-64.ipk
-rw-r--r-- 3 ecordonnier ecordonnier 213M Sep 28 18:03
openssl-dbg_3.0.5-r0_core2-64.ipk
-rw-r--r-- 3 ecordonnier ecordonnier 36M Sep 28 17:58
binutils-dbg_2.39-r0_core2-64.ipk
-rw-r--r-- 3 ecordonnier ecordonnier 30M Sep 28 18:00
openssl-ptest_3.0.5-r0_core2-64.ipk
-rw-r--r-- 3 ecordonnier ecordonnier 22M Sep 28 17:59
perl-ptest_5.36.0-r0_core2-64.ipk
-rw-r--r-- 3 ecordonnier ecordonnier 20M Sep 28 17:58
gcc_12.2.0-r0_core2-64.ipk
-rw-r--r-- 3 ecordonnier ecordonnier 14M Sep 28 17:57
elfutils-ptest_0.187-r0_core2-64.ipk
-rw-r--r-- 3 ecordonnier ecordonnier 12M Sep 28 17:58
gcc-src_12.2.0-r0_core2-64.ipk
-rw-r--r-- 3 ecordonnier ecordonnier 12M Sep 28 17:57
g++_12.2.0-r0_core2-64.ipk
xz at level 9 (the default):
build$ ls -lh tmp-xz/deploy/ipk/core2-64/ --sort=size | head -n10
total 963M
-rw-r--r-- 3 ecordonnier ecordonnier 214M Sep 14 10:44
gcc-dbg_12.2.0-r0_core2-64.ipk
-rw-r--r-- 3 ecordonnier ecordonnier 193M Sep 14 10:44
openssl-dbg_3.0.5-r0_core2-64.ipk
-rw-r--r-- 3 ecordonnier ecordonnier 35M Sep 14 10:44
openssl-ptest_3.0.5-r0_core2-64.ipk
-rw-r--r-- 3 ecordonnier ecordonnier 32M Sep 14 10:44
binutils-dbg_2.39-r0_core2-64.ipk
-rw-r--r-- 3 ecordonnier ecordonnier 20M Sep 14 10:44
perl-ptest_5.36.0-r0_core2-64.ipk
-rw-r--r-- 3 ecordonnier ecordonnier 19M Sep 14 10:44
gcc_12.2.0-r0_core2-64.ipk
-rw-r--r-- 3 ecordonnier ecordonnier 13M Sep 14 10:44
elfutils-ptest_0.187-r0_core2-64.ipk
-rw-r--r-- 3 ecordonnier ecordonnier 12M Sep 14 10:44
gcc-src_12.2.0-r0_core2-64.ipk
-rw-r--r-- 3 ecordonnier ecordonnier 11M Sep 14 10:44
g++_12.2.0-r0_core2-64.ipk
On Wed, Sep 14, 2022 at 5:42 PM Khem Raj <raj.k...@gmail.com> wrote:
On Wed, Sep 14, 2022 at 8:37 AM Alex Stewart <alex.stew...@ni.com>
wrote:
>
> Thanks for checking.
>
> I'd be interested to know if setting a higher compression level
for zstd
> can get us to a similar compression ratio to xz. If so, then I
think it
> could be some real value to distro maintainers to be able to *tune*
> their compression.
yeah it will be interesting to say try something like level 9 but
I think times
might regress with that but it might be good to know the balance
and perhaps
suggest size mode and performance mode of zstd instead of xz
--
Alex Stewart
Software Engineer - NI Real-Time OS
NI (National Instruments)
alex.stew...@ni.com
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#171152):
https://lists.openembedded.org/g/openembedded-core/message/171152
Mute This Topic: https://lists.openembedded.org/mt/93654146/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-
