Hi Sebastian, On Sun, Apr 12, 2020 at 1:09 AM Sebastian Andrzej Siewior <sebast...@breakpoint.cc> wrote: > > On 2018-03-11 21:51:05 [+0100], Balint Reczey wrote: > > For the recompressed firefox .deb (Ubuntu's > > firefox_58.0.2+build1-0ubuntu0.17.10.1_amd64.deb) increased ~9% in > > size but decompressed in <20% of the original time: > > So you are saying that the decompression speed that is the bottleneck > here? I *assumed* that it is mostly the disk speed since I get around 60 > to 80MiB/sec out of xz.
I would not say bottleneck, but a very big contributor to the CPU time used. Some systems can have very slow IO and very fast CPU, but I think those typically correlate positively and SSDs are more common than spinning disks improving the typical IO speed. > > $ du -s firefox-*deb > > 43960 firefox-xz.deb > > 47924 firefox-zstd.deb > > 48M linux-image-5.5.0-1-amd64_5.5.13-2_amd64.data.tar.xz > 54M linux-image-5.5.0-1-amd64_5.5.13-2_amd64.data.tar.19.zstd > > 766M linux-image-5.5.0-1-amd64-dbg_5.5.13-2_amd64.data.tar.xz > 901M linux-image-5.5.0-1-amd64-dbg_5.5.13-2_amd64.data.tar.19.zstd > > zstd -19 -T16 > |linux-image-5.5.0-1-amd64-dbg_5.5.13-2_amd64.data.tar : Completed in 287.37 > sec (cpu load : 1533%) > | > |real 4m47,416s > |user 73m23,825s > |sys 0m2,753s > | > > xz -T16 > | real 4m15,447s > | user 66m51,572s > | sys 0m3,201s > > > > $ rm -rf firefox-xz/* ;time dpkg-deb -R firefox-xz.deb firefox-xz/ > > real 0m4,270s > > user 0m4,220s > > sys 0m0,630s > > $ rm -rf firefox-zstd/* ;time dpkg-deb -R firefox-zstd.deb firefox-zstd/ > > real 0m0,765s > > user 0m0,556s > > sys 0m0,462s > > So this looks impressive. Is dpkg-deb also performing sync() on the > output or is the report when the files hit the disk cache? Either way, > should be noticeable on ssd/nvme which write at higher performance. > > > Tests on the full Ubuntu main archive showed ~6% average increase in > > the size of the binary packages. > > I guess the vast majority of packages are small and hardly increase in > size. The bigger packages then increase more. Yes, the increase ratio seem to be approximately uniform, looking at the data posted by Julian, thus bigger packages increase more: https://lists.ubuntu.com/archives/ubuntu-devel/2018-March/040245.html > > The patches are also available on Salsa [2]. > > While I read the whole thread here, I did not find any consent other > than discuss it d-devel. Is this still the case? Yes, ideally the divergence should be avoided thus we are waiting if the package format could be set to stone in Debian. Cheers, Balint -- Balint Reczey Ubuntu & Debian Developer
