Re: [OE-core] [PATCH] opkg: enable zstd support

2022-09-28 Thread Alex Stewart
Thanks for doing the research. What was the build time on the zstd L19 compression vs. xz? Was there still an improvement? On 9/28/22 11:50, Etienne Cordonnier wrote: I tested it a bit more today. I used the standard poky's local.conf and added those lines. You can see that at zstd level 9

Re: [OE-core] [PATCH] opkg: enable zstd support

2022-09-28 Thread Etienne Cordonnier via lists.openembedded.org
I tested it a bit more today. I used the standard poky's local.conf and added those lines. You can see that at zstd level 9 there is still a significant difference in the compression ratio with xz for gcc-dbg which is a big file. At zstd level 19, gcc-dbg is 241MB big instead of 214MB with xz

Re: [OE-core] [PATCH] opkg: enable zstd support

2022-09-14 Thread Khem Raj
On Wed, Sep 14, 2022 at 8:37 AM Alex Stewart wrote: > > Thanks for checking. > > I'd be interested to know if setting a higher compression level for zstd > can get us to a similar compression ratio to xz. If so, then I think it > could be some real value to distro maintainers to be able to *tune*

Re: [OE-core] [PATCH] opkg: enable zstd support

2022-09-14 Thread Alex Stewart
Thanks for checking. I'd be interested to know if setting a higher compression level for zstd can get us to a similar compression ratio to xz. If so, then I think it could be some real value to distro maintainers to be able to *tune* their compression. That's not blocking for your new PR

Re: [OE-core] [PATCH] opkg: enable zstd support

2022-09-14 Thread Etienne Cordonnier via lists.openembedded.org
Also note that zstd's default compression level is 3 per default (from a 1 to 19 scale). I did not test other compression levels. On Wed, Sep 14, 2022 at 11:58 AM Etienne Cordonnier < ecordonn...@snapchat.com> wrote: > I ran a build of core-image-full-cmdline using xz and zstd, using >

Re: [OE-core] [PATCH] opkg: enable zstd support

2022-09-14 Thread Etienne Cordonnier via lists.openembedded.org
I ran a build of core-image-full-cmdline using xz and zstd, using pre-populated downloads and sstate-cache directories but with empty tmp directory. Here are the numbers: zstd: bitbake core-image-full-cmdline took 2m52.768s (real), the resulting directory tmp/deploy/ipk is 1.6GB big. xz: bitbake

Re: [OE-core] [PATCH] opkg: enable zstd support

2022-09-13 Thread Alex Stewart
On 9/13/22 15:20, Alex Feinman wrote: I do have some numbers. When I was selling this change internally, I did a comparison on our internal build. Combined write IPK times (Σ t do_package_write_ipk) xz 162m 35s gz 52m 13s zstd 33m 49s Compression rate for zstd

Re: [OE-core] [PATCH] opkg: enable zstd support

2022-09-13 Thread Khem Raj
Thanks for sharing, really appreciate it. How close were compression rates in numbers ? I think some users might be sensitive to the archive sizes, if its something with in acceptable range, I do agree, zstd would be a better default these days. On Tue, Sep 13, 2022 at 1:20 PM Alex Feinman

Re: [OE-core] [PATCH] opkg: enable zstd support

2022-09-13 Thread Alex Feinman via lists.openembedded.org
On Tue, Sep 13, 2022 at 1:20 PM Alex Feinman wrote: > I do have some numbers. When I was selling this change internally, I did a > comparison on our internal build. > > Combined write IPK times (Σ t do_package_write_ipk) > > xz 162m 35s > > gz 52m 13s > > zstd 33m

Re: [OE-core] [PATCH] opkg: enable zstd support

2022-09-13 Thread Khem Raj
On Tue, Sep 13, 2022 at 12:19 PM Alex Stewart wrote: > > ACK from me - apart from enabling zstd by default. > > On 9/13/22 07:37, Etienne Cordonnier via lists.openembedded.org wrote: > > This allows the use of zstd for opkg packages by using OPKGBUILDCMD: > > OPKGBUILDCMD = "opkg-build -Z zstd" >

Re: [OE-core] [PATCH] opkg: enable zstd support

2022-09-13 Thread Alex Stewart
ACK from me - apart from enabling zstd by default. On 9/13/22 07:37, Etienne Cordonnier via lists.openembedded.org wrote: This allows the use of zstd for opkg packages by using OPKGBUILDCMD: OPKGBUILDCMD = "opkg-build -Z zstd" Signed-off-by: Alex Feinman Signed-off-by: Etienne Cordonnier ---

[OE-core] [PATCH] opkg: enable zstd support

2022-09-13 Thread Etienne Cordonnier via lists.openembedded.org
This allows the use of zstd for opkg packages by using OPKGBUILDCMD: OPKGBUILDCMD = "opkg-build -Z zstd" Signed-off-by: Alex Feinman Signed-off-by: Etienne Cordonnier --- meta/recipes-devtools/opkg/opkg_0.6.0.bb | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git