[Bug 1717040] Re: Please backport libzstd 1.3.1+dfsg-1 (universe) from artful
Update confirmed ! Thanks ! -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1717040 Title: Please backport libzstd 1.3.1+dfsg-1 (universe) from artful To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/libzstd/+bug/1717040/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1717040] Re: Please backport libzstd 1.3.1+dfsg-1 (universe) from artful
This bug was fixed in the package libzstd - 1.3.1+dfsg-1~ubuntu0.16.04.1 --- libzstd (1.3.1+dfsg-1~ubuntu0.16.04.1) xenial; urgency=medium * Backport to xenial (LP: #1717040): - d/compat, d/control: use debhelper 9 since version 10 is only in xenial-backports - d/control, d/rules: new libzstd1-dev package. Added a Provides for libzstd-dev to keep d-shlibmove happy. * d/rules: also run the clean Makefile target in the pzstd directory. libzstd (1.3.1+dfsg-1) unstable; urgency=medium * Team upload * New upstream version * Exclude examples from upstream tarball Closes: #869581 * debhelper 10 * cme fix dpkg-control * Standards-Version: 4.1.0 (no changes needed) * Fix copyright libzstd (1.2.0-1) unstable; urgency=medium * New upstream release (Closes: #863159). libzstd (1.1.2-1) unstable; urgency=medium * Team upload. * New upstream version 1.1.2 libzstd (1.1.1-1) unstable; urgency=medium * New upstream version 1.1.1 (Closes: #844248) * Whitespace fix to d/control * Ensure hardening flags pass through to compiler by appending to CPPFLAGS libzstd (1.1.0-1) unstable; urgency=medium * New upstream version 1.1.0 (Closes: #839960) * Change uploader email * Build new pzstd binary, including manpage * Fix miscellaneous issues with d/rules and d/*.install * Move docs from libzstd-dev to the zstd binary libzstd (1.0.0-1) unstable; urgency=medium * Imported Upstream version 1.0.0 (Closes: #836574) * Bump library package name to libzstd1 * Changed project URLs, copyrights after move to facebook. * d/rules: hardening=+all libzstd (0.8.0-1) unstable; urgency=medium [ Kevin Murray ] * New upstream version (Closes: #834114) [ Andreas Tille ] * hardening=+bindnow libzstd (0.5.1-1) unstable; urgency=medium [ Kevin Murray ] * New upstream version (fixes a FTBFS) * Package pkgconfig file (Closes: #813854) * Fix path to tagged archives in d/watch * Add vcs URLs * Bump to standards version 3.9.7 [ Mattia Rizzolo ] * debian/rules: Remove a lot of uneeded comments and lines libzstd (0.4.7-1) unstable; urgency=low * New upstream version * Remove build date encoding to enable reproducible build libzstd (0.4.5-1) unstable; urgency=low * Initial release (Closes: #806767) -- Andreas Hasenack Tue, 16 Jan 2018 21:50:37 + ** Changed in: libzstd (Ubuntu Xenial) Status: Fix Committed => Fix Released -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1717040 Title: Please backport libzstd 1.3.1+dfsg-1 (universe) from artful To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/libzstd/+bug/1717040/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1717040] Re: Please backport libzstd 1.3.1+dfsg-1 (universe) from artful
I'm told xenial will be handled after 16.04.4 is released. The task is still open, and listed in https://people.canonical.com/~ubuntu-archive /pending-sru.html, and an SRU team member shall pick it up then following the normal SRU workflow. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1717040 Title: Please backport libzstd 1.3.1+dfsg-1 (universe) from artful To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/libzstd/+bug/1717040/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1717040] Re: Please backport libzstd 1.3.1+dfsg-1 (universe) from artful
andreas@nsnx:~$ rmadison -s xenial libzstd libzstd | 0.5.1-1 | xenial/universe | source andreas@nsnx:~$ rmadison -s xenial-updates libzstd andreas@nsnx:~$ rmadison -s xenial-proposed libzstd libzstd | 1.3.1+dfsg-1~ubuntu0.16.04.1 | xenial-proposed/universe | source -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1717040 Title: Please backport libzstd 1.3.1+dfsg-1 (universe) from artful To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/libzstd/+bug/1717040/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1717040] Re: Please backport libzstd 1.3.1+dfsg-1 (universe) from artful
This bug was fixed in the package libzstd - 1.3.1+dfsg-1ubuntu0.1 --- libzstd (1.3.1+dfsg-1ubuntu0.1) artful; urgency=medium * d/control: New transitional package libzstd1-dev to facilitate the release upgrade from xenial (LP: #1717040) * d/rules: also run the clean Makefile target in the pzstd directory. libzstd (1.3.1+dfsg-1) unstable; urgency=medium * Team upload * New upstream version * Exclude examples from upstream tarball Closes: #869581 * debhelper 10 * cme fix dpkg-control * Standards-Version: 4.1.0 (no changes needed) * Fix copyright libzstd (1.2.0-1) unstable; urgency=medium * New upstream release (Closes: #863159). libzstd (1.1.2-1) unstable; urgency=medium * Team upload. * New upstream version 1.1.2 libzstd (1.1.1-1) unstable; urgency=medium * New upstream version 1.1.1 (Closes: #844248) * Whitespace fix to d/control * Ensure hardening flags pass through to compiler by appending to CPPFLAGS libzstd (1.1.0-1) unstable; urgency=medium * New upstream version 1.1.0 (Closes: #839960) * Change uploader email * Build new pzstd binary, including manpage * Fix miscellaneous issues with d/rules and d/*.install * Move docs from libzstd-dev to the zstd binary libzstd (1.0.0-1) unstable; urgency=medium * Imported Upstream version 1.0.0 (Closes: #836574) * Bump library package name to libzstd1 * Changed project URLs, copyrights after move to facebook. * d/rules: hardening=+all libzstd (0.8.0-1) unstable; urgency=medium [ Kevin Murray ] * New upstream version (Closes: #834114) [ Andreas Tille ] * hardening=+bindnow libzstd (0.5.1-1) unstable; urgency=medium [ Kevin Murray ] * New upstream version (fixes a FTBFS) * Package pkgconfig file (Closes: #813854) * Fix path to tagged archives in d/watch * Add vcs URLs * Bump to standards version 3.9.7 [ Mattia Rizzolo ] * debian/rules: Remove a lot of uneeded comments and lines libzstd (0.4.7-1) unstable; urgency=low * New upstream version * Remove build date encoding to enable reproducible build libzstd (0.4.5-1) unstable; urgency=low * Initial release (Closes: #806767) -- Andreas Hasenack Tue, 16 Jan 2018 19:28:18 + ** Changed in: libzstd (Ubuntu Artful) Status: Fix Committed => Fix Released -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1717040 Title: Please backport libzstd 1.3.1+dfsg-1 (universe) from artful To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/libzstd/+bug/1717040/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1717040] Re: Please backport libzstd 1.3.1+dfsg-1 (universe) from artful
I believe this completes all testing from the SRU test case. Adjusting tags to reflect that state. ** Tags removed: verification-needed-artful verification-needed-xenial ** Tags added: verification-done-artful verification-done-xenial -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1717040 Title: Please backport libzstd 1.3.1+dfsg-1 (universe) from artful To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/libzstd/+bug/1717040/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1717040] Re: Please backport libzstd 1.3.1+dfsg-1 (universe) from artful
RELEASE UPGRADE FROM ARTFUL TO BIONIC - without the transitional package installed: PASS root@artful2bionic:~# dpkg -l|grep zstd ii libzstd-dev1.3.1+dfsg-1ubuntu0.1 amd64fast lossless compression algorithm -- development files ii libzstd1 1.3.1+dfsg-1ubuntu0.1 amd64fast lossless compression algorithm ii zstd 1.3.1+dfsg-1ubuntu0.1 amd64fast lossless compression algorithm -- CLI tool do-release-upgrade -d details: Upgrade: libzstd-dev libzstd1 zstd At the end, nothing about zstd Final state in bionic: root@artful2bionic:~# dpkg -l|grep zstd ii libzstd-dev 1.3.3+dfsg-1ubuntu1amd64 fast lossless compression algorithm -- development files ii libzstd1 1.3.3+dfsg-1ubuntu1amd64 fast lossless compression algorithm ii zstd 1.3.3+dfsg-1ubuntu1amd64 fast lossless compression algorithm -- CLI tool root@artful2bionic:~# apt-cache policy zstd zstd: Installed: 1.3.3+dfsg-1ubuntu1 Candidate: 1.3.3+dfsg-1ubuntu1 Version table: *** 1.3.3+dfsg-1ubuntu1 500 500 http://br.archive.ubuntu.com/ubuntu bionic/universe amd64 Packages 100 /var/lib/dpkg/status -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1717040 Title: Please backport libzstd 1.3.1+dfsg-1 (universe) from artful To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/libzstd/+bug/1717040/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1717040] Re: Please backport libzstd 1.3.1+dfsg-1 (universe) from artful
RELEASE UPGRADE FROM ARTFUL TO BIONIC - with all zstd packages installed PASS root@artful2bionic:~# dpkg -l|grep zstd ii libzstd-dev1.3.1+dfsg-1ubuntu0.1 amd64fast lossless compression algorithm -- development files ii libzstd1 1.3.1+dfsg-1ubuntu0.1 amd64fast lossless compression algorithm ii libzstd1-dev 1.3.1+dfsg-1ubuntu0.1 amd64transitional package for libzstd-dev ii zstd 1.3.1+dfsg-1ubuntu0.1 amd64fast lossless compression algorithm -- CLI tool do-release-upgrade -d details: upgrade: libzstd-dev libzstd1 libzstd1-dev zstd At the end, nothing about zstd Final state of zstd packages in bionic: root@artful2bionic:~# dpkg -l|grep zstd ii libzstd-dev 1.3.3+dfsg-1ubuntu1amd64 fast lossless compression algorithm -- development files ii libzstd1 1.3.3+dfsg-1ubuntu1amd64 fast lossless compression algorithm ii libzstd1-dev 1.3.3+dfsg-1ubuntu1amd64 transitional package for libzstd-dev ii zstd 1.3.3+dfsg-1ubuntu1amd64 fast lossless compression algorithm -- CLI tool And they come from bionic universe: root@artful2bionic:~# apt-cache policy zstd zstd: Installed: 1.3.3+dfsg-1ubuntu1 Candidate: 1.3.3+dfsg-1ubuntu1 Version table: *** 1.3.3+dfsg-1ubuntu1 500 500 http://br.archive.ubuntu.com/ubuntu bionic/universe amd64 Packages 100 /var/lib/dpkg/status -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1717040 Title: Please backport libzstd 1.3.1+dfsg-1 (universe) from artful To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/libzstd/+bug/1717040/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1717040] Re: Please backport libzstd 1.3.1+dfsg-1 (universe) from artful
RELEASE UPGRADE FROM XENIAL TO ARTFUL - starting with the 1.3.1 dev package in xenial PASS root@xenial2artful:~# dpkg -l|grep zstd ii libzstd0 0.5.1-1 amd64fast lossless compression algorithm ii libzstd1 1.3.1+dfsg-1~ubuntu0.16.04.1 amd64fast lossless compression algorithm ii libzstd1-dev 1.3.1+dfsg-1~ubuntu0.16.04.1 amd64fast lossless compression algorithm -- development files ii zstd 1.3.1+dfsg-1~ubuntu0.16.04.1 amd64fast lossless compression algorithm -- CLI tool Details of upgrade: Install: libzstd-dev Upgrade: libzstd1 libzstd1-dev zstd At the end: Remove (was auto installed) libpython3.5 libzstd0 libzstd1-dev end result (transitional package was removed): root@xenial2artful:~# dpkg -l|grep zstd ii libzstd-dev1.3.1+dfsg-1ubuntu0.1 amd64fast lossless compression algorithm -- development files ii libzstd1 1.3.1+dfsg-1ubuntu0.1 amd64fast lossless compression algorithm ii zstd 1.3.1+dfsg-1ubuntu0.1 amd64fast lossless compression algorithm -- CLI tool Confirming proposed was used in the release upgrade: root@xenial2artful:~# apt-cache policy zstd zstd: Installed: 1.3.1+dfsg-1ubuntu0.1 Candidate: 1.3.1+dfsg-1ubuntu0.1 Version table: *** 1.3.1+dfsg-1ubuntu0.1 500 500 http://br.archive.ubuntu.com/ubuntu artful-proposed/universe amd64 Packages 100 /var/lib/dpkg/status 1.3.1+dfsg-1 500 500 http://br.archive.ubuntu.com/ubuntu artful/universe amd64 Packages -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1717040 Title: Please backport libzstd 1.3.1+dfsg-1 (universe) from artful To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/libzstd/+bug/1717040/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1717040] Re: Please backport libzstd 1.3.1+dfsg-1 (universe) from artful
RELEASE UPGRADE FROM XENIAL TO ARTFUL - with libzstd-dev 0.5.1, others at 1.3.1 PASS Starting with the original xenial packages: ubuntu@xenial2artful:~$ dpkg -l|grep zstd ii libzstd-dev 0.5.1-1 amd64fast lossless compression algorithm -- development files ii libzstd0 0.5.1-1 amd64fast lossless compression algorithm ii zstd 0.5.1-1 amd64fast lossless compression algorithm -- CLI tool dist-upgrading to proposed: ubuntu@xenial2artful:~$ dpkg -l|grep zstd ii libzstd-dev 0.5.1-1 amd64fast lossless compression algorithm -- development files ii libzstd0 0.5.1-1 amd64fast lossless compression algorithm ii libzstd1 1.3.1+dfsg-1~ubuntu0.16.04.1 amd64fast lossless compression algorithm ii zstd 1.3.1+dfsg-1~ubuntu0.16.04.1 amd64fast lossless compression algorithm -- CLI tool Configuring update-manager to allow third-party repositories (not sure if -proposed falls into that category): # echo -e "[Sources]\nAllowThirdParty=yes" > /etc/update-manager/release-upgrades.d/allow.cfg Changing prompt to "normal" in /etc/update-manager/release-upgrades do-release-upgrade to artful In details about upgraded packages, we see these mentioned: libzstd-dev libzstd1 zstd At the end, libzstd0 is removed: Remove (was auto installed) libpython3.5 libzstd0 Final result in the now artful system: root@xenial2artful:~# dpkg -l|grep zstd ii libzstd-dev1.3.1+dfsg-1ubuntu0.1 amd64fast lossless compression algorithm -- development files ii libzstd1 1.3.1+dfsg-1ubuntu0.1 amd64fast lossless compression algorithm ii zstd 1.3.1+dfsg-1ubuntu0.1 amd64fast lossless compression algorithm -- CLI tool Showing that proposed was used: root@xenial2artful:~# apt-cache policy zstd zstd: Installed: 1.3.1+dfsg-1ubuntu0.1 Candidate: 1.3.1+dfsg-1ubuntu0.1 Version table: *** 1.3.1+dfsg-1ubuntu0.1 500 500 http://br.archive.ubuntu.com/ubuntu artful-proposed/universe amd64 Packages 100 /var/lib/dpkg/status 1.3.1+dfsg-1 500 500 http://br.archive.ubuntu.com/ubuntu artful/universe amd64 Packages -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1717040 Title: Please backport libzstd 1.3.1+dfsg-1 (universe) from artful To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/libzstd/+bug/1717040/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1717040] Re: Please backport libzstd 1.3.1+dfsg-1 (universe) from artful
b) artful package upgrade tests - PASS Starting with the original artful packages: dpubuntu@artful-zstd:~$ dpkg -l|grep zstd ii libzstd-dev1.3.1+dfsg-1 amd64fast lossless compression algorithm -- development files ii libzstd1 1.3.1+dfsg-1 amd64fast lossless compression algorithm ii zstd 1.3.1+dfsg-1 amd64fast lossless compression algorithm -- CLI tool dist-upgrade pulls in no new packages and just upgrades the ones I have already: The following packages will be upgraded: libzstd-dev libzstd1 zstd 3 upgraded, 0 newly installed, 0 to remove and 0 not upgraded. resulting in: ubuntu@artful-zstd:~$ dpkg -l|grep zstd ii libzstd-dev1.3.1+dfsg-1ubuntu0.1 amd64fast lossless compression algorithm -- development files ii libzstd1 1.3.1+dfsg-1ubuntu0.1 amd64fast lossless compression algorithm ii zstd 1.3.1+dfsg-1ubuntu0.1 amd64fast lossless compression algorithm -- CLI tool installing the libzstd1-dev transitional package: ubuntu@artful-zstd:~$ sudo apt install libzstd1-dev Reading package lists... Done Building dependency tree Reading state information... Done The following NEW packages will be installed: libzstd1-dev 0 upgraded, 1 newly installed, 0 to remove and 0 not upgraded. removing libzstd-dev removes the transitional package: ubuntu@artful-zstd:~$ sudo apt purge libzstd-dev Reading package lists... Done Building dependency tree Reading state information... Done The following packages will be REMOVED: libzstd-dev* libzstd1-dev* 0 upgraded, 0 newly installed, 2 to remove and 0 not upgraded. installing the transitional package pulls in the unversioned dev package: ubuntu@artful-zstd:~$ sudo apt install libzstd1-dev Reading package lists... Done Building dependency tree Reading state information... Done The following additional packages will be installed: libzstd-dev The following NEW packages will be installed: libzstd-dev libzstd1-dev -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1717040 Title: Please backport libzstd 1.3.1+dfsg-1 (universe) from artful To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/libzstd/+bug/1717040/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1717040] Re: Please backport libzstd 1.3.1+dfsg-1 (universe) from artful
a) compression tests - artful - PASS starting with the original artful packages: ubuntu@artful-zstd:~$ dpkg -l|grep zstd ii libzstd1 1.3.1+dfsg-1 amd64fast lossless compression algorithm ii zstd 1.3.1+dfsg-1 amd64fast lossless compression algorithm -- CLI tool ubuntu@artful-zstd:~$ zstd /usr/bin/snap -o snap.1.3.1.zst /usr/bin/snap: 31.03% (11318760 => 3512707 bytes, snap.1.3.1.zst) gives: -rwxr-xr-x 1 ubuntu ubuntu 3.4M Nov 30 16:42 snap.1.3.1.zst Now upgrading to the package in proposed: The following additional packages will be installed: libzstd1 The following packages will be upgraded: libzstd1 zstd ... Get:1 http://br.archive.ubuntu.com/ubuntu artful-proposed/universe amd64 zstd amd64 1.3.1+dfsg-1ubuntu0.1 [238 kB] Get:2 http://br.archive.ubuntu.com/ubuntu artful-proposed/universe amd64 libzstd1 amd64 1.3.1+dfsg-1ubuntu0.1 [152 kB] ... Compressing with the new package: ubuntu@artful-zstd:~$ zstd /usr/bin/snap -o snap.1.3.1-updated.zst /usr/bin/snap: 31.03% (11318760 => 3512707 bytes, snap.1.3.1-updated.zst) Decompressing the previous one: ubuntu@artful-zstd:~$ zstd -d snap.1.3.1.zst -o snap.1.3.1 snap.1.3.1.zst : 11318760 bytes Decompressing the updated one: ubuntu@artful-zstd:~$ zstd -d snap.1.3.1-updated.zst -o snap.1.3.1-updated snap.1.3.1-updated.zst: 11318760 bytes MD5 of everything: ubuntu@artful-zstd:~$ md5sum /usr/bin/snap snap.1.3.1 snap.1.3.1-updated 0b65998bc775d444b2ee7d00cd18634a /usr/bin/snap 0b65998bc775d444b2ee7d00cd18634a snap.1.3.1 0b65998bc775d444b2ee7d00cd18634a snap.1.3.1-updated Downgrading back to the original artful version: ubuntu@artful-zstd:~$ sudo apt install zstd=1.3.1+dfsg-1 libzstd1=1.3.1+dfsg-1 Reading package lists... Done Building dependency tree Reading state information... Done The following packages will be DOWNGRADED: libzstd1 zstd Decompressing and another md5 snapshot: ubuntu@artful-zstd:~$ zstd -d snap.1.3.1.zst -o snap.1.3.1-new snap.1.3.1.zst : 11318760 bytes ubuntu@artful-zstd:~$ md5sum snap.1.3.1-new /usr/bin/snap 0b65998bc775d444b2ee7d00cd18634a snap.1.3.1-new 0b65998bc775d444b2ee7d00cd18634a /usr/bin/snap ** Description changed: [Special SRU Notes] The Artful SRU must be published before the Xenial SRU in order to preserve the upgrade path. [Impact] * libzstd v0.5.1 is an experimental version, which generates and read an experimental format incompatible with official libzstd v1+ format. * Backporting a newer version >= v1.0, such as artful's v1.3.1, fixes this issue * This backport/SRU is being requested by upstream. [Test Case] There are three major test cases: compression tests, package upgrade tests, and release upgrade tests. This SRU has so many tests because it's introducing a new package and a carefully thought out upgrade plan. a) compression tests * with the pristine package from each release, compress a big file. For example, /usr/bin/snap. Name the resulting compressed file according to the version that was used to compress it, like this: Xenial: $ zstd /usr/bin/snap -o snap.0.5.1.zst Compressed 15528016 bytes into 4134889 bytes ==> 26.63% - Zesty: - $ zstd /usr/bin/snap -o snap.1.1.2.zst - /usr/bin/snap: 30.59% (10825608 => 3311295 bytes, snap.1.1.2.zst) + Artful: + $ zstd /usr/bin/snap -o snap.1.3.1.zst + /usr/bin/snap: 31.03% (11318760 => 3512707 bytes, snap.1.3.1.zst) * Upgrade to the package made available through this SRU and compress it again: Xenial: $ zstd /usr/bin/snap -o snap.1.3.1.zst /usr/bin/snap: 25.11% (15528016 => 3899137 bytes, snap.1.3.1.zst) - Zesty: - $ zstd /usr/bin/snap -o snap.1.3.1.zst - /usr/bin/snap: 30.59% (10825608 => 3311290 bytes, snap.1.3.1.zst) + Artful: + $ zstd /usr/bin/snap -o snap.1.3.1-updated.zst + /usr/bin/snap: 31.03% (11318760 => 3512707 bytes, snap.1.3.1-updated.zst) * Uncompress all the generated files using the new version. This proves it can handle files compressed by the old one: Xenial: $ zstd -d snap.0.5.1.zst -o snap.0.5.1 snap.0.5.1.zst : 15528016 bytes $ zstd -d snap.1.3.1.zst -o snap.1.3.1 snap.1.3.1.zst : 15528016 bytes - Zesty: - $ zstd -d snap.1.1.2.zst -o snap.1.1.2 - snap.1.1.2.zst : 10825608 bytes + Artful: $ zstd -d snap.1.3.1.zst -o snap.1.3.1 - snap.1.3.1.zst : 10825608 bytes + snap.1.3.1.zst : 11318760 bytes + $ zstd -d snap.1.3.1-updated.zst -o snap.1.3.1-updated + snap.1.3.1-updated.zst: 11318760 bytes * Take md5sums of all uncompressed files and the original. They must of course match: Xenial: $ md5sum /usr/bin/snap snap.0.5.1 snap.1.3.1 0fa4fa69d79ef468
[Bug 1717040] Re: Please backport libzstd 1.3.1+dfsg-1 (universe) from artful
b) xenial package upgrade tests - PASS starting with: $ dpkg -l|grep zstd ii libzstd-dev 0.5.1-1 amd64fast lossless compression algorithm -- development files ii libzstd0 0.5.1-1 amd64fast lossless compression algorithm ii zstd 0.5.1-1 amd64fast lossless compression algorithm -- CLI tool dist-upgrade: The following NEW packages will be installed: libzstd1 The following packages will be upgraded: zstd 1 upgraded, 1 newly installed, 0 to remove and 0 not upgraded. results in: ubuntu@xenial-zstd:~$ dpkg -l|grep zstd ii libzstd-dev 0.5.1-1 amd64fast lossless compression algorithm -- development files ii libzstd0 0.5.1-1 amd64fast lossless compression algorithm ii libzstd1 1.3.1+dfsg-1~ubuntu0.16.04.1 amd64fast lossless compression algorithm ii zstd 1.3.1+dfsg-1~ubuntu0.16.04.1 amd64fast lossless compression algorithm -- CLI tool Installing the new versioned dev package should remove the unversioned one: ubuntu@xenial-zstd:~$ sudo apt install libzstd1-dev Reading package lists... Done Building dependency tree Reading state information... Done The following packages will be REMOVED: libzstd-dev The following NEW packages will be installed: libzstd1-dev end result: ubuntu@xenial-zstd:~$ dpkg -l|grep zstd ii libzstd0 0.5.1-1 amd64fast lossless compression algorithm ii libzstd1 1.3.1+dfsg-1~ubuntu0.16.04.1 amd64fast lossless compression algorithm ii libzstd1-dev 1.3.1+dfsg-1~ubuntu0.16.04.1 amd64fast lossless compression algorithm -- development files ii zstd 1.3.1+dfsg-1~ubuntu0.16.04.1 amd64fast lossless compression algorithm -- CLI tool * Conversely, installing the unversioned dev package removes the versioned one: ubuntu@xenial-zstd:~$ sudo apt install libzstd-dev Reading package lists... Done Building dependency tree Reading state information... Done The following packages will be REMOVED: libzstd1-dev The following NEW packages will be installed: libzstd-dev * Verify that a simple apt upgrade (not dist-upgrade) upgrades zstd and installs the new libzstd1: Starting with: ubuntu@xenial-zstd:~$ dpkg -l|grep zstd ii libzstd-dev 0.5.1-1 amd64fast lossless compression algorithm -- development files ii libzstd0 0.5.1-1 amd64fast lossless compression algorithm ii zstd 0.5.1-1 amd64fast lossless compression algorithm -- CLI tool ubuntu@xenial-zstd:~$ sudo apt upgrade Reading package lists... Done Building dependency tree Reading state information... Done Calculating upgrade... Done The following NEW packages will be installed: libzstd1 The following packages will be upgraded: zstd correctly results in: ubuntu@xenial-zstd:~$ dpkg -l|grep zstd ii libzstd-dev 0.5.1-1 amd64fast lossless compression algorithm -- development files ii libzstd0 0.5.1-1 amd64fast lossless compression algorithm ii libzstd1 1.3.1+dfsg-1~ubuntu0.16.04.1 amd64fast lossless compression algorithm ii zstd 1.3.1+dfsg-1~ubuntu0.16.04.1 amd64fast lossless compression algorithm -- CLI tool -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1717040 Title: Please backport libzstd 1.3.1+dfsg-1 (universe) from artful To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/libzstd/+bug/1717040/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1717040] Re: Please backport libzstd 1.3.1+dfsg-1 (universe) from artful
The artful package is stuck in the "new" queue as well, I'll skip the artful tests for now. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1717040 Title: Please backport libzstd 1.3.1+dfsg-1 (universe) from artful To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/libzstd/+bug/1717040/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1717040] Re: Please backport libzstd 1.3.1+dfsg-1 (universe) from artful
I'll post my verification in steps, since the list of checks is long. a) compression tests - xenial - PASS ubuntu@xenial-zstd:~$ zstd /usr/bin/snap -o snap.0.5.1.zst Compressed 17183248 bytes into 4541079 bytes ==> 26.43% ubuntu@xenial-zstd:~$ ll snap.0.5.1.zst -rw-rw-r-- 1 ubuntu ubuntu 4541079 Feb 19 17:52 snap.0.5.1.zst ubuntu@xenial-zstd:~$ Package from proposed: *** 1.3.1+dfsg-1~ubuntu0.16.04.1 400 400 http://br.archive.ubuntu.com/ubuntu xenial-proposed/universe amd64 Packages ubuntu@xenial-zstd:~$ zstd /usr/bin/snap -o snap.1.3.1.zst /usr/bin/snap: 24.91% (17183248 => 4281096 bytes, snap.1.3.1.zst) ubuntu@xenial-zstd:~$ ll snap.1.3.1.zst -rwxr-xr-x 1 ubuntu ubuntu 4281096 Nov 30 19:48 snap.1.3.1.zst* Uncompressing: ubuntu@xenial-zstd:~$ zstd -d snap.0.5.1.zst -o snap.0.5.1 snap.0.5.1.zst : 17183248 bytes ubuntu@xenial-zstd:~$ zstd -d snap.1.3.1.zst -o snap.1.3.1 snap.1.3.1.zst : 17183248 bytes md5 verification is OK: ubuntu@xenial-zstd:~$ md5sum /usr/bin/snap snap.0.5.1 snap.1.3.1 50ff3c166c443c32f043294416a5b5d9 /usr/bin/snap 50ff3c166c443c32f043294416a5b5d9 snap.0.5.1 50ff3c166c443c32f043294416a5b5d9 snap.1.3.1 xenial downgrade fails to recognize the new format, as expected: ubuntu@xenial-zstd:~$ zstd -d snap.1.3.1.zst -o /dev/null zstd: /dev/null already exists; do you wish to overwrite (y/N) ? y zstd: snap.1.3.1.zst: not in zstd format -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1717040 Title: Please backport libzstd 1.3.1+dfsg-1 (universe) from artful To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/libzstd/+bug/1717040/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1717040] Re: Please backport libzstd 1.3.1+dfsg-1 (universe) from artful
I've got an explanation for the "DOWNGRADED" statement : I had ppa:ginggs/backports in my list of ppa. This ppa contains an updated version of zstd. So that's why when applying `apt-get install zstd/xenial-proposed`, it features the "DOWNGRADED" statement. Removing the PPA, and re-installing the default zstd v0.5.1 from Xenial, I now get : ``` sudo apt install zstd/xenial-proposed Reading package lists... Done Building dependency tree Reading state information... Done Selected version '1.3.1+dfsg-1~ubuntu0.16.04.1' (Ubuntu:16.04/xenial-proposed [amd64]) for 'zstd' Selected version '5.4.0-6ubuntu1~16.04.9' (Ubuntu:16.04/xenial-proposed [amd64]) for 'libstdc++6' because of 'zstd' The following additional packages will be installed: libzstd1 The following NEW packages will be installed: libzstd1 The following packages will be upgraded: zstd 1 upgraded, 1 newly installed, 0 to remove and 157 not upgraded. Need to get 0 B/391 kB of archives. After this operation, 905 kB of additional disk space will be used. ``` which is the expected outcome. then : ``` zstd -V *** zstd command line interface 64-bits v1.3.1, by Yann Collet *** ``` -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1717040 Title: Please backport libzstd 1.3.1+dfsg-1 (universe) from artful To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/libzstd/+bug/1717040/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1717040] Re: Please backport libzstd 1.3.1+dfsg-1 (universe) from artful
I could test `apt-get install zstd/xenial-proposed` today. And it works as intended : library is there and usable, cli works fine too. Just as a little detail, here is the message I'm receiving during installation : ``` The following additional packages will be installed: libzstd1 The following packages will be REMOVED: libzstd-dev The following NEW packages will be installed: zstd The following packages will be DOWNGRADED: libzstd1 0 upgraded, 1 newly installed, 1 downgraded, 1 to remove and 157 not upgraded. Need to get 391 kB of archives. After this operation, 258 kB of additional disk space will be used. ``` I was surprised by the DOWNGRADED statement because libzstd1 wasn't installed on this VM, so there should be nothing to downgrade. Anyway, this is mostly a cosmetic issue, final statement is : it works all fine. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1717040 Title: Please backport libzstd 1.3.1+dfsg-1 (universe) from artful To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/libzstd/+bug/1717040/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1717040] Re: Please backport libzstd 1.3.1+dfsg-1 (universe) from artful
It was stuck in the NEW queue, and just approved. Should arrive in the proposed pocket soon. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1717040 Title: Please backport libzstd 1.3.1+dfsg-1 (universe) from artful To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/libzstd/+bug/1717040/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1717040] Re: Please backport libzstd 1.3.1+dfsg-1 (universe) from artful
Yeah, it's not published yet for some reason: https://launchpad.net/ubuntu/+source/libzstd shows this for xenial: " Note: Some binary packages for this source are not yet published in the repository." Same for artful @racb do you know if something is wrong, or is it just the nowadays normal delay we have in builders and publishers and what else? -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1717040 Title: Please backport libzstd 1.3.1+dfsg-1 (universe) from artful To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/libzstd/+bug/1717040/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1717040] Re: Please backport libzstd 1.3.1+dfsg-1 (universe) from artful
I tried to follow the procedure presented on https://wiki.ubuntu.com/Testing/EnableProposed, but got blocked at this stage : ``` sudo apt-get install zstd/xenial-proposed Reading package lists... Done Building dependency tree Reading state information... Done E: Release 'xenial-proposed' for 'zstd' was not found ``` and indeed, a further call to `apt list --upgradable` does not show `zstd/xenial-proposed` in the list of packages. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1717040 Title: Please backport libzstd 1.3.1+dfsg-1 (universe) from artful To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/libzstd/+bug/1717040/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1717040] Re: Please backport libzstd 1.3.1+dfsg-1 (universe) from artful
Looks like ubuntustudio uses libzstd via blender -> libopenvdb3.2 -> libblosc1. Seems pretty obscure though - far from the common use case of blender AFAIK. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1717040 Title: Please backport libzstd 1.3.1+dfsg-1 (universe) from artful To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/libzstd/+bug/1717040/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
Re: [Bug 1717040] Re: Please backport libzstd 1.3.1+dfsg-1 (universe) from artful
I pinged the SRU team again today. The package was uploaded to proposed but needs SRU approval to actually migrate to it and become available. On Wed, Feb 14, 2018 at 8:41 PM, Yann Collet <1717...@bugs.launchpad.net> wrote: > Is the Xenial patch good to go ? > Or is there any blocker remaining at this stage ? > > -- > You received this bug notification because you are subscribed to the bug > report. > https://bugs.launchpad.net/bugs/1717040 > > Title: > Please backport libzstd 1.3.1+dfsg-1 (universe) from artful > > To manage notifications about this bug go to: > https://bugs.launchpad.net/ubuntu/+source/libzstd/+bug/ > 1717040/+subscriptions > -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1717040 Title: Please backport libzstd 1.3.1+dfsg-1 (universe) from artful To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/libzstd/+bug/1717040/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1717040] Re: Please backport libzstd 1.3.1+dfsg-1 (universe) from artful
Hello Yann, or anyone else affected, Accepted libzstd into xenial-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/libzstd/1.3.1+dfsg-1~ubuntu0.16.04.1 in a few hours, and then in the -proposed repository. Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed.Your feedback will aid us getting this update out to other Ubuntu users. If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested and change the tag from verification-needed-xenial to verification-done-xenial. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-xenial. In either case, details of your testing will help us make a better decision. Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance! ** Changed in: libzstd (Ubuntu Xenial) Status: In Progress => Fix Committed ** Tags added: verification-needed verification-needed-xenial ** Changed in: libzstd (Ubuntu Artful) Status: In Progress => Fix Committed ** Tags added: verification-needed-artful -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1717040 Title: Please backport libzstd 1.3.1+dfsg-1 (universe) from artful To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/libzstd/+bug/1717040/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1717040] Re: Please backport libzstd 1.3.1+dfsg-1 (universe) from artful
@Yann It was in the SRU queue. I've processed that now, so it will build shortly in xenial-proposed and artful-proposed and be published as a proposed update visible to all users. Then there's a seven day testing and aging period. After that, I'll look at the test reports and if all is well we can publish it into xenial-updates (and artful-updates) which will make it available as a general update to all users. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1717040 Title: Please backport libzstd 1.3.1+dfsg-1 (universe) from artful To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/libzstd/+bug/1717040/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1717040] Re: Please backport libzstd 1.3.1+dfsg-1 (universe) from artful
Is the Xenial patch good to go ? Or is there any blocker remaining at this stage ? -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1717040 Title: Please backport libzstd 1.3.1+dfsg-1 (universe) from artful To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/libzstd/+bug/1717040/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1717040] Re: Please backport libzstd 1.3.1+dfsg-1 (universe) from artful
This bug was fixed in the package libzstd - 1.3.3+dfsg-1ubuntu1 --- libzstd (1.3.3+dfsg-1ubuntu1) bionic; urgency=medium * d/control: New transitional package libzstd1-dev to facilitate the release upgrade from xenial (LP: #1717040) * d/rules: also run the clean Makefile target in the pzstd directory. -- Andreas Hasenack Fri, 02 Feb 2018 11:29:34 -0200 ** Changed in: libzstd (Ubuntu Bionic) Status: In Progress => Fix Released -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1717040 Title: Please backport libzstd 1.3.1+dfsg-1 (universe) from artful To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/libzstd/+bug/1717040/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1717040] Re: Please backport libzstd 1.3.1+dfsg-1 (universe) from artful
** Description changed: [Special SRU Notes] The Artful SRU must be published before the Xenial SRU in order to preserve the upgrade path. [Impact] * libzstd v0.5.1 is an experimental version, which generates and read an experimental format incompatible with official libzstd v1+ format. * Backporting a newer version >= v1.0, such as artful's v1.3.1, fixes this issue * This backport/SRU is being requested by upstream. [Test Case] + + There are three major test cases: compression tests, package upgrade + tests, and release upgrade tests. + + This SRU has so many tests because it's introducing a new package and a + carefully thought out upgrade plan. + + a) compression tests * with the pristine package from each release, compress a big file. For example, /usr/bin/snap. Name the resulting compressed file according to the version that was used to compress it, like this: Xenial: $ zstd /usr/bin/snap -o snap.0.5.1.zst Compressed 15528016 bytes into 4134889 bytes ==> 26.63% Zesty: $ zstd /usr/bin/snap -o snap.1.1.2.zst /usr/bin/snap: 30.59% (10825608 => 3311295 bytes, snap.1.1.2.zst) * Upgrade to the package made available through this SRU and compress it again: Xenial: $ zstd /usr/bin/snap -o snap.1.3.1.zst /usr/bin/snap: 25.11% (15528016 => 3899137 bytes, snap.1.3.1.zst) Zesty: $ zstd /usr/bin/snap -o snap.1.3.1.zst /usr/bin/snap: 30.59% (10825608 => 3311290 bytes, snap.1.3.1.zst) * Uncompress all the generated files using the new version. This proves it can handle files compressed by the old one: Xenial: $ zstd -d snap.0.5.1.zst -o snap.0.5.1 snap.0.5.1.zst : 15528016 bytes $ zstd -d snap.1.3.1.zst -o snap.1.3.1 snap.1.3.1.zst : 15528016 bytes Zesty: $ zstd -d snap.1.1.2.zst -o snap.1.1.2 snap.1.1.2.zst : 10825608 bytes $ zstd -d snap.1.3.1.zst -o snap.1.3.1 snap.1.3.1.zst : 10825608 bytes * Take md5sums of all uncompressed files and the original. They must of course match: Xenial: $ md5sum /usr/bin/snap snap.0.5.1 snap.1.3.1 0fa4fa69d79ef4685aaa93be5b3aa33f /usr/bin/snap 0fa4fa69d79ef4685aaa93be5b3aa33f snap.0.5.1 0fa4fa69d79ef4685aaa93be5b3aa33f snap.1.3.1 Zesty: $ md5sum /usr/bin/snap snap.1.1.2 snap.1.3.1 ba0a3ef5f519688bc5b0b58f190e73a4 /usr/bin/snap ba0a3ef5f519688bc5b0b58f190e73a4 snap.1.1.2 ba0a3ef5f519688bc5b0b58f190e73a4 snap.1.3.1 * Downgrade zstd back to the original version of the ubuntu release you are testing: Xenial: $ sudo apt install zstd=0.5.1-1 Zesty: $ sudo apt install zstd=1.1.2-1 libzstd1=1.1.2-1 * Try to decompress the zst file created with the 1.3.1 version from the previous test. In xenial only, it should fail to recognize the format: $ zstd -d snap.1.3.1.zst -o /dev/null zstd: /dev/null already exists; do you wish to overwrite (y/N) ? y zstd: snap.1.3.1.zst: not in zstd format In zesty it should work: $ zstd -d snap.1.3.1.zst -o snap.1.3.1-new snap.1.3.1.zst : 9909192 bytes * Still in zesty, compare the md5, which should match: $ md5sum snap.1.3.1-new /usr/bin/snap 7c980688861eef598a56c7970d952028 snap.1.3.1-new 7c980688861eef598a56c7970d952028 /usr/bin/snap + + + b) package upgrade tests + b.1)XENIAL + + * Install the xenial original packages: + $ sudo apt install zstd=0.5.1-1 libzstd0=0.5.1-1 libzstd-dev=0.5.1-1 + + * Enable xenial-proposed and verify that a dist-upgrade will upgrade zstd, pull in the new libzstd1, and leave libzstd-dev alone: + $ sudo apt dist-upgrade + ... + The following NEW packages will be installed: + libzstd1 + The following packages will be upgraded: + zstd + 1 upgraded, 1 newly installed, 0 to remove and 0 not upgraded. + + * In the end you will have these installed, at these versions: + ii libzstd-dev 0.5.1-1 + ii libzstd0 0.5.1-1 + ii libzstd1 1.3.1+dfsg-1~ubuntu0.16.04.1 + ii zstd 1.3.1+dfsg-1~ubuntu0.16.04.1 + + + * Installing the new versioned dev package should remove the unversioned one: + $ sudo apt install libzstd1-dev + Reading package lists... Done + Building dependency tree + Reading state information... Done + The following packages will be REMOVED: + libzstd-dev + The following NEW packages will be installed: + libzstd1-dev + 0 upgraded, 1 newly installed, 1 to remove and 0 not upgraded. + + * The end result being: + ii libzstd0 0.5.1-1 + ii libzstd1 1.3.1+dfsg-1~ubuntu0.16.04.1 + ii libzstd1-dev 1.3.1+dfsg-1~ubuntu0.16.04.1 + ii zstd 1.3.1+dfsg-1~ubuntu0.16.04.1 + + + * Conversely, installing the unversioned dev package removes the versioned one:: + Installing libzstd-dev removes libzstd1-dev: + $ sudo apt install libzstd-dev + Reading package lists... Done + Building dependency tree + Reading state information..
[Bug 1717040] Re: Please backport libzstd 1.3.1+dfsg-1 (universe) from artful
** Description changed: + [Special SRU Notes] + + The Artful SRU must be published before the Xenial SRU in order to + preserve the upgrade path. + [Impact] * libzstd v0.5.1 is an experimental version, which generates and read an experimental format incompatible with official libzstd v1+ format. * Backporting a newer version >= v1.0, such as artful's v1.3.1, fixes this issue - * This backport/SRU is being requested by upstream. + * This backport/SRU is being requested by upstream. [Test Case] * with the pristine package from each release, compress a big file. For example, /usr/bin/snap. Name the resulting compressed file according to the version that was used to compress it, like this: Xenial: $ zstd /usr/bin/snap -o snap.0.5.1.zst Compressed 15528016 bytes into 4134889 bytes ==> 26.63% Zesty: $ zstd /usr/bin/snap -o snap.1.1.2.zst /usr/bin/snap: 30.59% (10825608 => 3311295 bytes, snap.1.1.2.zst) * Upgrade to the package made available through this SRU and compress it again: Xenial: $ zstd /usr/bin/snap -o snap.1.3.1.zst /usr/bin/snap: 25.11% (15528016 => 3899137 bytes, snap.1.3.1.zst) Zesty: $ zstd /usr/bin/snap -o snap.1.3.1.zst /usr/bin/snap: 30.59% (10825608 => 3311290 bytes, snap.1.3.1.zst) * Uncompress all the generated files using the new version. This proves it can handle files compressed by the old one: Xenial: $ zstd -d snap.0.5.1.zst -o snap.0.5.1 snap.0.5.1.zst : 15528016 bytes $ zstd -d snap.1.3.1.zst -o snap.1.3.1 snap.1.3.1.zst : 15528016 bytes Zesty: $ zstd -d snap.1.1.2.zst -o snap.1.1.2 snap.1.1.2.zst : 10825608 bytes $ zstd -d snap.1.3.1.zst -o snap.1.3.1 snap.1.3.1.zst : 10825608 bytes * Take md5sums of all uncompressed files and the original. They must of course match: Xenial: $ md5sum /usr/bin/snap snap.0.5.1 snap.1.3.1 0fa4fa69d79ef4685aaa93be5b3aa33f /usr/bin/snap 0fa4fa69d79ef4685aaa93be5b3aa33f snap.0.5.1 0fa4fa69d79ef4685aaa93be5b3aa33f snap.1.3.1 Zesty: $ md5sum /usr/bin/snap snap.1.1.2 snap.1.3.1 ba0a3ef5f519688bc5b0b58f190e73a4 /usr/bin/snap ba0a3ef5f519688bc5b0b58f190e73a4 snap.1.1.2 ba0a3ef5f519688bc5b0b58f190e73a4 snap.1.3.1 * Downgrade zstd back to the original version of the ubuntu release you are testing: Xenial: $ sudo apt install zstd=0.5.1-1 Zesty: $ sudo apt install zstd=1.1.2-1 libzstd1=1.1.2-1 * Try to decompress the zst file created with the 1.3.1 version from the previous test. In xenial only, it should fail to recognize the format: $ zstd -d snap.1.3.1.zst -o /dev/null zstd: /dev/null already exists; do you wish to overwrite (y/N) ? y zstd: snap.1.3.1.zst: not in zstd format In zesty it should work: $ zstd -d snap.1.3.1.zst -o snap.1.3.1-new snap.1.3.1.zst : 9909192 bytes * Still in zesty, compare the md5, which should match: $ md5sum snap.1.3.1-new /usr/bin/snap 7c980688861eef598a56c7970d952028 snap.1.3.1-new 7c980688861eef598a56c7970d952028 /usr/bin/snap [Regression Potential] * During transition, if some user switches to newer zstd version, and then send compressed data to another user has not yet updated zstd, the receiver will not be able to decompress as long as he does not update. * Note though that newer version contains a compatibility module which makes it able to read data generated by older versions, such as v0.5.1. Any existing document compressed with v0.5.1 will still be readable after update to v1.3.1. [Other Info] * Linux kernel 4.14 now integrates zstd compression / decompression, primarily for BtrFS and SquashFS, with patches available for reiser4, zram, and initrd. Kernel version does not support compatibility module, hence cannot read data from v0.5.1. Userland tools associated with these services will fail if they link to Ubuntu LTS libzstd, as generating incompatible data. * On top of the incompatible format issue, libzstd v0.5.1 API is old and missing several features that applications relying on libzstd need, such as streaming interface, or bulk processing for dictionary compression. --- Original description --- Please backport libzstd 1.3.1+dfsg-1 (universe) from artful to xenial. Reason for the backport: Current version in Xenial is v0.5.1, it's an experimental version, using its own, incompatible format. As a consequence, zstd on Ubuntu Xenial is not compatible with the rest of the world. This is of important concern for products using libzstd as a shared library : on Ubuntu Xenial, produced data is different, not compatible with v1+. This issue has been made more pressing with the integration of zstd in Linux Kernel, as userland tools must also be updated to read and generate zstd. Note : this request was first improperly filled at :
[Bug 1717040] Re: Please backport libzstd 1.3.1+dfsg-1 (universe) from artful
xenial apt upgrade test: a) zstd, libzstd0, libzstd-dev installed ubuntu@xenial-zstd-sru-1717040:~$ dpkg-query -W zstd libzstd0 libzstd-dev libzstd-dev 0.5.1-1 libzstd00.5.1-1 zstd0.5.1-1 ubuntu@xenial-zstd-sru-1717040:~$ sudo apt upgrade Reading package lists... Done Building dependency tree Reading state information... Done Calculating upgrade... Done The following package was automatically installed and is no longer required: libfreetype6 Use 'sudo apt autoremove' to remove it. The following NEW packages will be installed: libzstd1 The following packages will be upgraded: zstd 1 upgraded, 1 newly installed, 0 to remove and 0 not upgraded. Need to get 392 kB of archives. After this operation, 904 kB of additional disk space will be used. Do you want to continue? [Y/n] after: ubuntu@xenial-zstd-sru-1717040:~$ dpkg -l|grep zstd ii libzstd-dev 0.5.1-1 amd64fast lossless compression algorithm -- development files ii libzstd0 0.5.1-1 amd64fast lossless compression algorithm ii libzstd1 1.3.1+dfsg-1~ubuntu0.16.04.1~ppa5 amd64fast lossless compression algorithm ii zstd 1.3.1+dfsg-1~ubuntu0.16.04.1~ppa5 amd64fast lossless compression algorithm -- CLI tool ubuntu@xenial-zstd-sru-1717040:~$ sudo apt upgrade Reading package lists... Done Building dependency tree Reading state information... Done Calculating upgrade... Done The following packages were automatically installed and are no longer required: libfreetype6 libzstd0 Use 'sudo apt autoremove' to remove them. The following NEW packages will be installed: libzstd1 The following packages will be upgraded: zstd 1 upgraded, 1 newly installed, 0 to remove and 0 not upgraded. Need to get 392 kB of archives. After this operation, 904 kB of additional disk space will be used. Do you want to continue? [Y/n] end result: ubuntu@xenial-zstd-sru-1717040:~$ dpkg -l|grep zstd ii libzstd0 0.5.1-1 amd64fast lossless compression algorithm ii libzstd1 1.3.1+dfsg-1~ubuntu0.16.04.1~ppa5 amd64fast lossless compression algorithm ii zstd 1.3.1+dfsg-1~ubuntu0.16.04.1~ppa5 amd64fast lossless compression algorithm -- CLI tool c) motd motd acks the available upgrades: Welcome to Ubuntu 16.04.3 LTS (GNU/Linux 4.4.0-112-generic x86_64) * Documentation: https://help.ubuntu.com * Management: https://landscape.canonical.com * Support:https://ubuntu.com/advantage Get cloud support with Ubuntu Advantage Cloud Guest: http://www.ubuntu.com/business/services/cloud 2 packages can be updated. 0 updates are security updates. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1717040 Title: Please backport libzstd 1.3.1+dfsg-1 (universe) from artful To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/libzstd/+bug/1717040/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1717040] Re: Please backport libzstd 1.3.1+dfsg-1 (universe) from artful
xenial desktop upgrade via notification: Starting with these installed: libzstd-dev 0.5.1-1 libzstd00.5.1-1 zstd0.5.1-1 End result after the gui updated it (see screenshot for the prompt): ii libzstd-dev0.5.1-1 amd64fast lossless compression algorithm -- development files ii libzstd0 0.5.1-1 amd64fast lossless compression algorithm ii libzstd1 1.3.1+dfsg-1~ubuntu0.16.04.1~ppa5amd64fast lossless compression algorithm ii zstd 1.3.1+dfsg-1~ubuntu0.16.04.1~ppa5amd64fast lossless compression algorithm -- CLI tool ** Attachment added: "xenial-desktop-zstd-prompt.png" https://bugs.launchpad.net/ubuntu/+source/libzstd/+bug/1717040/+attachment/5047557/+files/xenial-desktop-zstd-prompt.png -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1717040 Title: Please backport libzstd 1.3.1+dfsg-1 (universe) from artful To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/libzstd/+bug/1717040/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1717040] Re: Please backport libzstd 1.3.1+dfsg-1 (universe) from artful
** Also affects: libzstd (Ubuntu Bionic) Importance: Undecided Assignee: Andreas Hasenack (ahasenack) Status: In Progress -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1717040 Title: Please backport libzstd 1.3.1+dfsg-1 (universe) from artful To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/libzstd/+bug/1717040/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1717040] Re: Please backport libzstd 1.3.1+dfsg-1 (universe) from artful
** Merge proposal linked: https://code.launchpad.net/~ahasenack/ubuntu/+source/libzstd/+git/libzstd/+merge/336258 ** Merge proposal linked: https://code.launchpad.net/~ahasenack/ubuntu/+source/libzstd/+git/libzstd/+merge/336259 ** Merge proposal linked: https://code.launchpad.net/~ahasenack/ubuntu/+source/libzstd/+git/libzstd/+merge/336260 -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1717040 Title: Please backport libzstd 1.3.1+dfsg-1 (universe) from artful To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/libzstd/+bug/1717040/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1717040] Re: Please backport libzstd 1.3.1+dfsg-1 (universe) from artful
I repeated the tests with the xenial package having the extra "Provides: libzstd-dev" and saw no ill effects. Attaching the new txt files with the details. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1717040 Title: Please backport libzstd 1.3.1+dfsg-1 (universe) from artful To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/libzstd/+bug/1717040/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1717040] Re: Please backport libzstd 1.3.1+dfsg-1 (universe) from artful
This artful test now includes a release-upgrade run to bionic ** Attachment added: "zstd-artful.txt" https://bugs.launchpad.net/ubuntu/+source/libzstd/+bug/1717040/+attachment/5038388/+files/zstd-artful.txt -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1717040 Title: Please backport libzstd 1.3.1+dfsg-1 (universe) from artful To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/libzstd/+bug/1717040/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1717040] Re: Please backport libzstd 1.3.1+dfsg-1 (universe) from artful
** Attachment added: "zstd-xenial.txt" https://bugs.launchpad.net/ubuntu/+source/libzstd/+bug/1717040/+attachment/5038387/+files/zstd-xenial.txt -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1717040 Title: Please backport libzstd 1.3.1+dfsg-1 (universe) from artful To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/libzstd/+bug/1717040/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1717040] Re: Please backport libzstd 1.3.1+dfsg-1 (universe) from artful
I have to retest this. d-shlibmove (why is that needed btw? Never heard of this yet another helper) is complaining the libzstd1-dev has no provides for libzstd-dev. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1717040 Title: Please backport libzstd 1.3.1+dfsg-1 (universe) from artful To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/libzstd/+bug/1717040/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1717040] Re: Please backport libzstd 1.3.1+dfsg-1 (universe) from artful
** Also affects: libzstd (Ubuntu Artful) Importance: Undecided Status: New -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1717040 Title: Please backport libzstd 1.3.1+dfsg-1 (universe) from artful To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/libzstd/+bug/1717040/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1717040] Re: Please backport libzstd 1.3.1+dfsg-1 (universe) from artful
** Changed in: libzstd (Ubuntu Artful) Status: New => Confirmed ** Changed in: libzstd (Ubuntu Artful) Status: Confirmed => In Progress ** Changed in: libzstd (Ubuntu Artful) Assignee: (unassigned) => Andreas Hasenack (ahasenack) ** Changed in: libzstd (Ubuntu) Status: Fix Released => In Progress ** Changed in: libzstd (Ubuntu) Assignee: (unassigned) => Andreas Hasenack (ahasenack) -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1717040 Title: Please backport libzstd 1.3.1+dfsg-1 (universe) from artful To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/libzstd/+bug/1717040/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1717040] Re: Please backport libzstd 1.3.1+dfsg-1 (universe) from artful
** Changed in: libzstd (Ubuntu Zesty) Status: In Progress => Won't Fix -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1717040 Title: Please backport libzstd 1.3.1+dfsg-1 (universe) from artful To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/libzstd/+bug/1717040/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1717040] Re: Please backport libzstd 1.3.1+dfsg-1 (universe) from artful
I have some good results. Xenial first, and artful in a subsequent but comment. XENIAL (details in the attached zstd-xenial.txt file): starting with: libzstd-dev 0.5.1-1 libzstd0 0.5.1-1 zstd 0.5.1-1 results in: libzstd-dev 0.5.1-1 libzstd0 0.5.1-1 libzstd1 1.3.1+dfsg-1~ubuntu0.16.04.1~ppa3 zstd 1.3.1+dfsg-1~ubuntu0.16.04.1~ppa3 Installing the new 1.3.1 dev package results in the removal of the old one: The following packages will be REMOVED: libzstd-dev The following NEW packages will be installed: libzstd1-dev which gives you: libzstd0 0.5.1-1 libzstd1 1.3.1+dfsg-1~ubuntu0.16.04.1~ppa3 libzstd1-dev 1.3.1+dfsg-1~ubuntu0.16.04.1~ppa3 zstd 1.3.1+dfsg-1~ubuntu0.16.04.1~ppa3 RELEASE UPGRADE TO ARTFUL a) starting with old 0.5.x dev package: libzstd-dev 0.5.1-1 libzstd0 0.5.1-1 libzstd1 1.3.1+dfsg-1~ubuntu0.16.04.1~ppa3 zstd 1.3.1+dfsg-1~ubuntu0.16.04.1~ppa3 You end up with: libzstd-dev 1.3.1+dfsg-1ubuntu0.1~ppa2 libzstd1 1.3.1+dfsg-1ubuntu0.1~ppa2 zstd 1.3.1+dfsg-1ubuntu0.1~ppa2 b) starting with new dev package: libzstd0 0.5.1-1 libzstd1 1.3.1+dfsg-1~ubuntu0.16.04.1~ppa3 libzstd1-dev 1.3.1+dfsg-1~ubuntu0.16.04.1~ppa3 zstd 1.3.1+dfsg-1~ubuntu0.16.04.1~ppa3 You end up with: libzstd-dev 1.3.1+dfsg-1ubuntu0.1~ppa2 libzstd1 1.3.1+dfsg-1ubuntu0.1~ppa2 libzstd1-dev 1.3.1+dfsg-1ubuntu0.1~ppa2 zstd 1.3.1+dfsg-1ubuntu0.1~ppa2 -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1717040 Title: Please backport libzstd 1.3.1+dfsg-1 (universe) from artful To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/libzstd/+bug/1717040/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1717040] Re: Please backport libzstd 1.3.1+dfsg-1 (universe) from artful
** Attachment added: "zstd-xenial.txt" https://bugs.launchpad.net/ubuntu/+source/libzstd/+bug/1717040/+attachment/5038187/+files/zstd-xenial.txt -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1717040 Title: Please backport libzstd 1.3.1+dfsg-1 (universe) from artful To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/libzstd/+bug/1717040/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1717040] Re: Please backport libzstd 1.3.1+dfsg-1 (universe) from artful
** Attachment added: "zstd-artful.txt" https://bugs.launchpad.net/ubuntu/+source/libzstd/+bug/1717040/+attachment/5038189/+files/zstd-artful.txt -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1717040 Title: Please backport libzstd 1.3.1+dfsg-1 (universe) from artful To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/libzstd/+bug/1717040/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1717040] Re: Please backport libzstd 1.3.1+dfsg-1 (universe) from artful
ARTFUL (details in the attached zstd-artful.txt file): starting with: libzstd-dev 1.3.1+dfsg-1 libzstd1 1.3.1+dfsg-1 zstd 1.3.1+dfsg-1 results in (no big deal): libzstd-dev 1.3.1+dfsg-1ubuntu0.1~ppa2 libzstd1 1.3.1+dfsg-1ubuntu0.1~ppa2 zstd 1.3.1+dfsg-1ubuntu0.1~ppa2 installing the new libzstd1-dev transitional package changes nothing, as expected: The following NEW packages will be installed: libzstd1-dev 0 upgraded, 1 newly installed, 0 to remove and 0 not upgraded. but removing libzstd-dev also removes libzstd1-dev: The following packages will be REMOVED: libzstd-dev* libzstd1-dev* 0 upgraded, 0 newly installed, 2 to remove and 0 not upgraded. and if you start from nothing and install libzstd1-dev (the transitional package); The following additional packages will be installed: libzstd-dev libzstd1 The following NEW packages will be installed: libzstd-dev libzstd1 libzstd1-dev 0 upgraded, 3 newly installed, 0 to remove and 0 not upgraded. you also get the "real" dev package, and the actual library. I'll do the same layout for bionic. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1717040 Title: Please backport libzstd 1.3.1+dfsg-1 (universe) from artful To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/libzstd/+bug/1717040/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
Re: [Bug 1717040] Re: Please backport libzstd 1.3.1+dfsg-1 (universe) from artful
On Tue, Jan 16, 2018 at 07:37:15PM -, Graham Inggs wrote: > "...an application hardwired to deal with v0.5.x experimental format, if > such a thing exist..." > > @ahasenack: Isn't this precisely we have Ubuntu Backports? I think we're somewhere in the middle. AIUI, virtually nothing else in the wider ecosystem can consume output produced by Xenial's zstd CLI tool. By publishing an update into xenial-updates, we'd be recommending that users update to it automatically (in practice the majority will get it without taking a separate step) because the version released with Xenial is effectively broken in terms of interoperability. > you need to opt-in to install it The point of considering an SRU for this fix, rather than backports, is precisely to remove this requirement. Users expect broken stuff to be fixed without having to individually opt-in to each fix, but instead by installing general "updates". I can see how this particular proposed update could go either way. But right now, we seem to be swinging in the direction of releasing it as an SRU, especially because we have a way to do that with no known downside. Our existing SRU policy does permit an update as a result of a change in the wider Internet environment. That we have an interop problem because the zstd CLI produces output that can't practically be handled anywhere else does, IMHO, qualify it for an SRU under this existing policy permission. The question is if and how we do it in the face of regression risk to existing users. Comments welcome, but please frame the discussion in terms of what we're trying to do and in terms of our existing SRU policy documented at https://wiki.ubuntu.com/StableReleaseUpdates. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1717040 Title: Please backport libzstd 1.3.1+dfsg-1 (universe) from artful To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/libzstd/+bug/1717040/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1717040] Re: Please backport libzstd 1.3.1+dfsg-1 (universe) from artful
Yeah, but the backports process is kind of broken and seeing no traction, that's why we are trying an SRU. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1717040 Title: Please backport libzstd 1.3.1+dfsg-1 (universe) from artful To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/libzstd/+bug/1717040/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1717040] Re: Please backport libzstd 1.3.1+dfsg-1 (universe) from artful
"...an application hardwired to deal with v0.5.x experimental format, if such a thing exist..." @ahasenack: Isn't this precisely we have Ubuntu Backports? We are saying: here is a new version of this software we have checked that it doesn't break anything in the archive you need to opt-in to install it and if you've built some local packages with it, you may have some breakage -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1717040 Title: Please backport libzstd 1.3.1+dfsg-1 (universe) from artful To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/libzstd/+bug/1717040/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1717040] Re: Please backport libzstd 1.3.1+dfsg-1 (universe) from artful
Ok, I'll handle it with a new set of packages. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1717040 Title: Please backport libzstd 1.3.1+dfsg-1 (universe) from artful To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/libzstd/+bug/1717040/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1717040] Re: Please backport libzstd 1.3.1+dfsg-1 (universe) from artful
I have a feeling that I don't fully grasp all the implied side effects here, so if this process feels safer based on your experience, please go ahead. I don't think the API will be an issue in this upgrade : as stated, all published symbols in v0.5.1 are still published in v1.3.1, so I would expect such application to continue linking properly. The change of format is another thing, and an application hardwired to deal with v0.5.x experimental format, if such a thing exist, would be surprised to deal with data using v1.x format. So some caution in the transition is indeed welcomed. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1717040 Title: Please backport libzstd 1.3.1+dfsg-1 (universe) from artful To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/libzstd/+bug/1717040/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1717040] Re: Please backport libzstd 1.3.1+dfsg-1 (universe) from artful
How about this: 1. Add transitional packages libzstd1-dev -> libzstd-dev in Artful and Bionic (no Conflicts). 2. Backport what is in Artful, but drop the transitional libzstd1-dev package, rename libzstd-dev to libzstd1-dev and have it Conflicts with libzstd-dev. This will land after Zesty EOLs, so don't worry about Zesty. This will result in: The zstd CLI from the zstd package will be updated, linked against libzstd1 and using new wire protocol. libzstd.so.1 will become available in Xenial. libzstd.so.0 will remain available. libzstd-dev will continue to exist and will not change, with the latest version remaining in the release pocket. libzstd1-dev will be added. Users will only be able to have either libzstd-dev or libzstd1-dev installed at once. The one installed will determine whether the old API or the new API is used, and whether the link will be against the old or new soname. Users already building using libzstd-dev locally in Xenial will not break and will continue to be able to rebuild (ie. "stay green") without any required intervention. If they wish, they can choose to use libzstd1-dev in Xenial instead. Or they can switch to use a newer release of Ubuntu or build directly against upstream sources, which I think is more likely for this use case anyway. Switching to libzstd1-dev will require code changes due to the API change, but this way this won't be forced upon them. Users building against libzstd-dev locally who switch to libzstd1-dev will be automatically transitioned to libzstd-dev again upon upgrade to Artful or Bionic. Users who hardcode the libzstd1-dev name in any automated builds will break after upgrading beyond Bionic, since we'll drop the transitional package at that point, so should really do something like "if release == Xenial: install libzstd1-dev; else install libzstd-dev" if they want to use a distribution provided library on Xenial *and* build against the newer wire protocol and library. If a security or bugfix updates to libzstd.so.0 becomes required, we'll need to adjust the release pocket source package to build only the libzstd0 binary package at that time. Yann: would you be happy with this? Rationale: There is an SRU policy exception "to adjust to changes in the environment, server protocols, web services, and similar". I think, under this exception, we want an SRU to cause the zstd CLI to produce new format output with backwards compatibility, and for libzstd.so.1 to be made available so that other third parties who link against the library we ship can do that also (we'd update other packages in the Ubuntu release too, but we don't have any in this particular case). > It is clearly labelled experimental, hence does not provide any API stability guarantee. Separate from an upstream guarantee though, Ubuntu _does_ try to ensure that stable releases do not change behaviour, except when it is deliberate and the project chooses to make an exception. This guarantee is separate from any upstream guarantees. A blanket policy by a distribution is much easier for users to rely upon versus individual upstream guarantees that may not all match. Part of the point of a distribution that does stable releases is to insulate users in this way. Perhaps an exception is warranted here, but I think it needs justification independent of the decision to update the CLI to produce and consume the newer wire protocol with backwards compatibility, which falls under one of our standing exceptions. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1717040 Title: Please backport libzstd 1.3.1+dfsg-1 (universe) from artful To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/libzstd/+bug/1717040/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1717040] Re: Please backport libzstd 1.3.1+dfsg-1 (universe) from artful
Hi Robie As far as I can tell, it is much more likely that an application expecting zstd >=v1.0.0 will break, most likely at link stage, and maybe at run stage, when provided with v0.5.1 instead. The other way round should be less problematic : applications expecting to work with v0.5.1 should still work with v.1.3.1. The reason is : All symbols defined in v0.5.1 are also present in the "stable" section of v1.3.1. For reference, here is the API for v0.5.1 : https://github.com/facebook/zstd/blob/v0.5.1/lib/zstd.h and here is the API for v1.3.1 : https://github.com/facebook/zstd/blob/v1.3.1/lib/zstd.h The major counter example is the streaming API. This one used to be available in a separate package : https://github.com/facebook/zstd/blob/v0.5.1/lib/zbuff.h It is clearly labelled experimental, hence does not provide any API stability guarantee. Nonetheless, in v1.3.1, this API is still available here : https://github.com/facebook/zstd/blob/v1.3.1/lib/deprecated/zbuff.h It is the exact same API, defining the same symbols. What has changed is that this API is now branded "deprecated", which means it will generate deprecation warnings during compilation (except if these warnings are explicitly disabled, using typically -Wno-deprecated-declarations for gcc, or the custom macro ZBUFF_DISABLE_DEPRECATE_WARNINGS). The deprecation warnings include comments to invite users to migrate towards the new "stable" streaming API, defined here : https://github.com/facebook/zstd/blob/v1.3.1/lib/zstd.h#L248 Of course, this is only useful for developers. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1717040 Title: Please backport libzstd 1.3.1+dfsg-1 (universe) from artful To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/libzstd/+bug/1717040/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1717040] Re: Please backport libzstd 1.3.1+dfsg-1 (universe) from artful
Yann, thank you for the detailed analysis. This is very helpful in giving us confidence in making changes to the stable releases. The addition of a libzstd1, and the update of zstd itself to use it, sounds reasonable to me. An additional thing for Regression Potential though I think? We're deliberately breaking the API provided by libzstd-dev in Xenial then, according to Yann's comment 10. If a user is relying on some local build against libzstd-dev then this proposed SRU will break it. We should call this out and make a decision on it. I presume Andreas and Yann are both in favor? I think this is what Adam was referring to in https://irclogs.ubuntu.com/2017/12/06/%23ubuntu-devel.html#t20:12 ? It doesn't sound counter to the intent to me. Any opinions on using libzstd1-dev in the SRU instead to mitigate this? Or are you asking to specifically not do that and change the API of libzstd-dev itself? I'm not particularly worried about justifications for Zesty. It's about to go EOL anyway. Anything that is acceptable for Xenial should also be acceptable for Zesty unless I'm missing some particular case where the circumstances are different. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1717040 Title: Please backport libzstd 1.3.1+dfsg-1 (universe) from artful To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/libzstd/+bug/1717040/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1717040] Re: Please backport libzstd 1.3.1+dfsg-1 (universe) from artful
Oh, I'm sorry. I can't read the major version number. Never mind! -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1717040 Title: Please backport libzstd 1.3.1+dfsg-1 (universe) from artful To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/libzstd/+bug/1717040/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1717040] Re: Please backport libzstd 1.3.1+dfsg-1 (universe) from artful
> Note though that newer version contains a compatibility module which makes it able to read data generated by older versions, such as v0.5.1. Any existing document compressed with v0.5.1 will still be readable after update to v1.3.1. Is this statement accidentally backwards? -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1717040 Title: Please backport libzstd 1.3.1+dfsg-1 (universe) from artful To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/libzstd/+bug/1717040/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1717040] Re: Please backport libzstd 1.3.1+dfsg-1 (universe) from artful
** Description changed: [Impact] * libzstd v0.5.1 is an experimental version, which generates and read an experimental format incompatible with official libzstd v1+ format. * Backporting a newer version >= v1.0, such as artful's v1.3.1, fixes this issue [Test Case] * download latest upstream zstd source tarball from https://github.com/facebook/zstd/releases * unpack, compile (make) * compress a file with upstream version * try to decompress with Ubuntu LTS 16.04 version -> it fails Second test: - * with the pristine package from each release, compress a big file. For example, /usr/bin/snap. Name the resulting compressed file according to the version that was used to compress it, like this: + * with the pristine package from each release, compress a big file. For example, /usr/bin/snap. Name the resulting compressed file according to the version that was used to compress it, like this: Xenial: $ zstd /usr/bin/snap -o snap.0.5.1.zst - Compressed 15528016 bytes into 4134889 bytes ==> 26.63% + Compressed 15528016 bytes into 4134889 bytes ==> 26.63% Zesty: $ zstd /usr/bin/snap -o snap.1.1.2.zst - /usr/bin/snap: 30.59% (10825608 => 3311295 bytes, snap.1.1.2.zst) + /usr/bin/snap: 30.59% (10825608 => 3311295 bytes, snap.1.1.2.zst) - - * Upgrade to the package made available through this SRU and compress it again: + * Upgrade to the package made available through this SRU and compress + it again: Xenial: $ zstd /usr/bin/snap -o snap.1.3.1.zst - /usr/bin/snap: 25.11% (15528016 => 3899137 bytes, snap.1.3.1.zst) + /usr/bin/snap: 25.11% (15528016 => 3899137 bytes, snap.1.3.1.zst) Zesty: $ zstd /usr/bin/snap -o snap.1.3.1.zst - /usr/bin/snap: 30.59% (10825608 => 3311290 bytes, snap.1.3.1.zst) + /usr/bin/snap: 30.59% (10825608 => 3311290 bytes, snap.1.3.1.zst) * Uncompress all the generated files using the new version. This proves it can handle files compressed by the old one: Xenial: $ zstd -d snap.0.5.1.zst -o snap.0.5.1 - snap.0.5.1.zst : 15528016 bytes + snap.0.5.1.zst : 15528016 bytes $ zstd -d snap.1.3.1.zst -o snap.1.3.1 - snap.1.3.1.zst : 15528016 bytes + snap.1.3.1.zst : 15528016 bytes Zesty: $ zstd -d snap.1.1.2.zst -o snap.1.1.2 - snap.1.1.2.zst : 10825608 bytes + snap.1.1.2.zst : 10825608 bytes $ zstd -d snap.1.3.1.zst -o snap.1.3.1 - snap.1.3.1.zst : 10825608 bytes + snap.1.3.1.zst : 10825608 bytes - - * Take md5sums of all uncompressed files and the original. They must of course match: + * Take md5sums of all uncompressed files and the original. They must of + course match: Xenial: $ md5sum /usr/bin/snap snap.0.5.1 snap.1.3.1 0fa4fa69d79ef4685aaa93be5b3aa33f /usr/bin/snap 0fa4fa69d79ef4685aaa93be5b3aa33f snap.0.5.1 0fa4fa69d79ef4685aaa93be5b3aa33f snap.1.3.1 Zesty: $ md5sum /usr/bin/snap snap.1.1.2 snap.1.3.1 ba0a3ef5f519688bc5b0b58f190e73a4 /usr/bin/snap ba0a3ef5f519688bc5b0b58f190e73a4 snap.1.1.2 ba0a3ef5f519688bc5b0b58f190e73a4 snap.1.3.1 + + * Downgrade zstd back to the original version of the ubuntu release you + are testing: + + Xenial: + $ sudo apt install zstd=0.5.1-1 + + Zesty: + $ sudo apt install zstd=1.1.2-1 libzstd1=1.1.2-1 + + + * Try to decompress the zst file created with the 1.3.1 version from the previous test. In xenial only, it should fail to recognize the format: + $ zstd -d snap.1.3.1.zst -o /dev/null + zstd: /dev/null already exists; do you wish to overwrite (y/N) ? y + zstd: snap.1.3.1.zst: not in zstd format + + In zesty it should work: + $ zstd -d snap.1.3.1.zst -o snap.1.3.1-new + snap.1.3.1.zst : 9909192 bytes + + * Still in zesty, compare the md5, which should match: + $ md5sum snap.1.3.1-new /usr/bin/snap + 7c980688861eef598a56c7970d952028 snap.1.3.1-new + 7c980688861eef598a56c7970d952028 /usr/bin/snap [Regression Potential] * During transition, if some user switches to newer zstd version, and then send compressed data to another user has not yet updated zstd, the receiver will not be able to decompress as long as he does not update. * Note though that newer version contains a compatibility module which makes it able to read data generated by older versions, such as v0.5.1. Any existing document compressed with v0.5.1 will still be readable after update to v1.3.1. [Other Info] * Linux kernel 4.14 now integrates zstd compression / decompression, primarily for BtrFS and SquashFS, with patches available for reiser4, zram, and initrd. Kernel version does not support compatibility module, hence cannot read data from v0.5.1. Userland tools associated with these services will fail if they link to Ubuntu LTS libzstd, as generating incompatible data. * On
[Bug 1717040] Re: Please backport libzstd 1.3.1+dfsg-1 (universe) from artful
Review on the MPs is complete and it is now sponsored for SRU review in [xenial/zesty]-unapproved. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1717040 Title: Please backport libzstd 1.3.1+dfsg-1 (universe) from artful To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/libzstd/+bug/1717040/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1717040] Re: Please backport libzstd 1.3.1+dfsg-1 (universe) from artful
@ahasenack : yes, completely -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1717040 Title: Please backport libzstd 1.3.1+dfsg-1 (universe) from artful To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/libzstd/+bug/1717040/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1717040] Re: Please backport libzstd 1.3.1+dfsg-1 (universe) from artful
** Merge proposal linked: https://code.launchpad.net/~ahasenack/ubuntu/+source/libzstd/+git/libzstd/+merge/335341 -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1717040 Title: Please backport libzstd 1.3.1+dfsg-1 (universe) from artful To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/libzstd/+bug/1717040/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1717040] Re: Please backport libzstd 1.3.1+dfsg-1 (universe) from artful
** Merge proposal linked: https://code.launchpad.net/~ahasenack/ubuntu/+source/libzstd/+git/libzstd/+merge/335340 -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1717040 Title: Please backport libzstd 1.3.1+dfsg-1 (universe) from artful To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/libzstd/+bug/1717040/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1717040] Re: Please backport libzstd 1.3.1+dfsg-1 (universe) from artful
@cyan4973, may I change your first test case into something that won't involve a download/build/install cycle? I was thinking about just using the 0.5.1 version in xenial to try to decompress the 1.3.1 compressed file I created in the test I just added to the bug description. Something like this: """ * Downgrade zstd back to xenial's original version: $ sudo apt install zstd=0.5.1-1 * Try to decompress the zst file created with the 1.3.1 version from the previous test. It should fail to recognize the format: $ zstd -d snap.1.3.1.zst -o /dev/null zstd: /dev/null already exists; do you wish to overwrite (y/N) ? y zstd: snap.1.3.1.zst: not in zstd format """ Would that be ok, and still be in the spirit of this update? -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1717040 Title: Please backport libzstd 1.3.1+dfsg-1 (universe) from artful To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/libzstd/+bug/1717040/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1717040] Re: Please backport libzstd 1.3.1+dfsg-1 (universe) from artful
** Description changed: [Impact] * libzstd v0.5.1 is an experimental version, -which generates and read an experimental format -incompatible with official libzstd v1+ format. + which generates and read an experimental format + incompatible with official libzstd v1+ format. - * Backporting a newer version >= v1.0, such as artful's v1.3.1, -fixes this issue + * Backporting a newer version >= v1.0, such as artful's v1.3.1, + fixes this issue [Test Case] * download latest upstream zstd source tarball from https://github.com/facebook/zstd/releases - * unpack, compile (make) + * unpack, compile (make) - * compress a file with upstream version + * compress a file with upstream version - * try to decompress with Ubuntu LTS 16.04 version -> it fails + * try to decompress with Ubuntu LTS 16.04 version -> it fails + + Second test: + * with the pristine package from each release, compress a big file. For example, /usr/bin/snap. Name the resulting compressed file according to the version that was used to compress it, like this: + + Xenial: + $ zstd /usr/bin/snap -o snap.0.5.1.zst + Compressed 15528016 bytes into 4134889 bytes ==> 26.63% + + Zesty: + $ zstd /usr/bin/snap -o snap.1.1.2.zst + /usr/bin/snap: 30.59% (10825608 => 3311295 bytes, snap.1.1.2.zst) + + + * Upgrade to the package made available through this SRU and compress it again: + + Xenial: + $ zstd /usr/bin/snap -o snap.1.3.1.zst + /usr/bin/snap: 25.11% (15528016 => 3899137 bytes, snap.1.3.1.zst) + + Zesty: + $ zstd /usr/bin/snap -o snap.1.3.1.zst + /usr/bin/snap: 30.59% (10825608 => 3311290 bytes, snap.1.3.1.zst) + + * Uncompress all the generated files using the new version. This proves + it can handle files compressed by the old one: + + Xenial: + $ zstd -d snap.0.5.1.zst -o snap.0.5.1 + snap.0.5.1.zst : 15528016 bytes + $ zstd -d snap.1.3.1.zst -o snap.1.3.1 + snap.1.3.1.zst : 15528016 bytes + + Zesty: + $ zstd -d snap.1.1.2.zst -o snap.1.1.2 + snap.1.1.2.zst : 10825608 bytes + $ zstd -d snap.1.3.1.zst -o snap.1.3.1 + snap.1.3.1.zst : 10825608 bytes + + + * Take md5sums of all uncompressed files and the original. They must of course match: + + Xenial: + $ md5sum /usr/bin/snap snap.0.5.1 snap.1.3.1 + 0fa4fa69d79ef4685aaa93be5b3aa33f /usr/bin/snap + 0fa4fa69d79ef4685aaa93be5b3aa33f snap.0.5.1 + 0fa4fa69d79ef4685aaa93be5b3aa33f snap.1.3.1 + + Zesty: + $ md5sum /usr/bin/snap snap.1.1.2 snap.1.3.1 + ba0a3ef5f519688bc5b0b58f190e73a4 /usr/bin/snap + ba0a3ef5f519688bc5b0b58f190e73a4 snap.1.1.2 + ba0a3ef5f519688bc5b0b58f190e73a4 snap.1.3.1 [Regression Potential] * During transition, if some user switches to newer zstd version, and then send compressed data to another user has not yet updated zstd, the receiver will not be able to decompress as long as he does not update. * Note though that newer version contains a compatibility module which makes it able to read data generated by older versions, such as v0.5.1. Any existing document compressed with v0.5.1 will still be readable after update to v1.3.1. - [Other Info] - * Linux kernel 4.14 now integrates zstd compression / decompression, + * Linux kernel 4.14 now integrates zstd compression / decompression, primarily for BtrFS and SquashFS, with patches available for reiser4, zram, and initrd. Kernel version does not support compatibility module, hence cannot read data from v0.5.1. Userland tools associated with these services will fail if they link to Ubuntu LTS libzstd, as generating incompatible data. - * On top of the incompatible format issue, libzstd v0.5.1 API is old + * On top of the incompatible format issue, libzstd v0.5.1 API is old and missing several features that applications relying on libzstd need, such as streaming interface, or bulk processing for dictionary compression. - --- Original description --- Please backport libzstd 1.3.1+dfsg-1 (universe) from artful to xenial. Reason for the backport: Current version in Xenial is v0.5.1, it's an experimental version, using its own, incompatible format. As a consequence, zstd on Ubuntu Xenial is not compatible with the rest of the world. This is of important concern for products using libzstd as a shared library : on Ubuntu Xenial, produced data is different, not compatible with v1+. This issue has been made more pressing with the integration of zstd in Linux Kernel, as userland tools must also be updated to read and generate zstd. Note : this request was first improperly filled at : https://bugs.launchpad.net/zesty-backports/+bug/1717037 Testing: Mark off items in the checklist [X] as you test them, but please leave the checklist so that backporters can quickly evaluate
[Bug 1717040] Re: Please backport libzstd 1.3.1+dfsg-1 (universe) from artful
Here are test packages for xenial and zesty: https://launchpad.net/~ahasenack/+archive/ubuntu/zstd-backport-1717040/ -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1717040 Title: Please backport libzstd 1.3.1+dfsg-1 (universe) from artful To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/libzstd/+bug/1717040/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1717040] Re: Please backport libzstd 1.3.1+dfsg-1 (universe) from artful
Ok, after an analysis done by a core dev (thanks @paelzer!), I have a way forward with this. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1717040 Title: Please backport libzstd 1.3.1+dfsg-1 (universe) from artful To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/libzstd/+bug/1717040/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1717040] Re: Please backport libzstd 1.3.1+dfsg-1 (universe) from artful
Hmm, why did it have to require a newer debhelper... I don't think an SRU can depend on something that's in backports. I will have to read up on "Standards-Version" and debhelper versions to see what I will have to change in the package to build properly with xenial's debhelper (version 9.x). -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1717040 Title: Please backport libzstd 1.3.1+dfsg-1 (universe) from artful To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/libzstd/+bug/1717040/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
Re: [Bug 1717040] Re: Please backport libzstd 1.3.1+dfsg-1 (universe) from artful
It's my next thing to do On Dec 15, 2017 20:35, "David Britton" wrote: > Hey Yann -- no news. Still in progress. > > -- > You received this bug notification because you are subscribed to the bug > report. > https://bugs.launchpad.net/bugs/1717040 > > Title: > Please backport libzstd 1.3.1+dfsg-1 (universe) from artful > > To manage notifications about this bug go to: > https://bugs.launchpad.net/ubuntu/+source/libzstd/+bug/ > 1717040/+subscriptions > -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1717040 Title: Please backport libzstd 1.3.1+dfsg-1 (universe) from artful To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/libzstd/+bug/1717040/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1717040] Re: Please backport libzstd 1.3.1+dfsg-1 (universe) from artful
Hey Yann -- no news. Still in progress. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1717040 Title: Please backport libzstd 1.3.1+dfsg-1 (universe) from artful To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/libzstd/+bug/1717040/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1717040] Re: Please backport libzstd 1.3.1+dfsg-1 (universe) from artful
Any news on this update ? -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1717040 Title: Please backport libzstd 1.3.1+dfsg-1 (universe) from artful To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/libzstd/+bug/1717040/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1717040] Re: Please backport libzstd 1.3.1+dfsg-1 (universe) from artful
** Changed in: libzstd (Ubuntu Xenial) Assignee: (unassigned) => Andreas Hasenack (ahasenack) ** Changed in: libzstd (Ubuntu Zesty) Assignee: (unassigned) => Andreas Hasenack (ahasenack) ** Changed in: libzstd (Ubuntu Xenial) Status: New => In Progress ** Changed in: libzstd (Ubuntu Zesty) Status: New => In Progress ** Changed in: libzstd (Ubuntu) Status: Confirmed => Fix Released -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1717040 Title: Please backport libzstd 1.3.1+dfsg-1 (universe) from artful To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/libzstd/+bug/1717040/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1717040] Re: Please backport libzstd 1.3.1+dfsg-1 (universe) from artful
Multithreading is the main new capability I can think of, and that's probably the one users will miss the most, but there are quite a few more ones, listed on the release page, and in the NEWS document : https://github.com/facebook/zstd/releases https://github.com/facebook/zstd/blob/dev/NEWS#L24 The list is pretty long though, and just copy/pasting it wouldn't give a sense of priority. Let's say there are a lot of small improvements and little bug fixing contributing to an overall better experience. Given a choice, I would recommend v1.3.1 over v1.1.2. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1717040 Title: Please backport libzstd 1.3.1+dfsg-1 (universe) from artful To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/libzstd/+bug/1717040/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1717040] Re: Please backport libzstd 1.3.1+dfsg-1 (universe) from artful
It's a justification issue, and also trying to keep the general spirit of *Stable* Release Updates (emphasis mine). What problems would we be fixing by updating zesty's zstd from 1.1.2 to 1.3.1, and if they are worth such an update. If it's just multithreading capability, then I think we can skip zesty, given that artful has it and zesty will be end of life in about a month (Jan 2018). -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1717040 Title: Please backport libzstd 1.3.1+dfsg-1 (universe) from artful To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/libzstd/+bug/1717040/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1717040] Re: Please backport libzstd 1.3.1+dfsg-1 (universe) from artful
I ask because otherwise we have to justify why we would be updating the zesty version (1.1.2) to artful's (1.3.1). -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1717040 Title: Please backport libzstd 1.3.1+dfsg-1 (universe) from artful To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/libzstd/+bug/1717040/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1717040] Re: Please backport libzstd 1.3.1+dfsg-1 (universe) from artful
Yes, it would be a pretty strong step forward compared to v0.5.1. v1.1.2 is from December 2016 : https://github.com/facebook/zstd/releases/tag/v1.1.2 The main feature developed since then is multithreaded compression. I guess a few users will miss it, but it's a minor setback compared to current situation. PS : any reason why zesty's 1.1.2 would be preferable to artful's 1.3.1 ? -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1717040 Title: Please backport libzstd 1.3.1+dfsg-1 (universe) from artful To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/libzstd/+bug/1717040/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1717040] Re: Please backport libzstd 1.3.1+dfsg-1 (universe) from artful
Is the zesty version ok? 1.1.2? -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1717040 Title: Please backport libzstd 1.3.1+dfsg-1 (universe) from artful To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/libzstd/+bug/1717040/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1717040] Re: Please backport libzstd 1.3.1+dfsg-1 (universe) from artful
New template updated ** Description changed: [Impact] - * An explanation of the effects of the bug on users and + * libzstd v0.5.1 is an experimental version, +which generates and read an experimental format +incompatible with official libzstd v1+ format. - * justification for backporting the fix to the stable release. - - * In addition, it is helpful, but not required, to include an -explanation of how the upload fixes this bug. + * Backporting a newer version >= v1.0, such as artful's v1.3.1, +fixes this issue [Test Case] - * detailed instructions how to reproduce the bug + * download latest upstream zstd source tarball from + https://github.com/facebook/zstd/releases - * these should allow someone who is not familiar with the affected -package to reproduce the bug and verify that the updated package fixes -the problem. + * unpack, compile (make) + + * compress a file with upstream version + + * try to decompress with Ubuntu LTS 16.04 version -> it fails + [Regression Potential] - * discussion of how regressions are most likely to manifest as a result - of this change. + * During transition, if some user switches to newer zstd version, and + then send compressed data to another user has not yet updated zstd, the + receiver will not be able to decompress as long as he does not update. - * It is assumed that any SRU candidate patch is well-tested before -upload and has a low overall risk of regression, but it's important -to make the effort to think about what ''could'' happen in the -event of a regression. + * Note though that newer version contains a compatibility module which + makes it able to read data generated by older versions, such as v0.5.1. + Any existing document compressed with v0.5.1 will still be readable + after update to v1.3.1. - * This both shows the SRU team that the risks have been considered, -and provides guidance to testers in regression-testing the SRU. [Other Info] - - * Anything else you think is useful to include - * Anticipate questions from users, SRU, +1 maintenance, security teams and the Technical Board - * and address these questions in advance + + * Linux kernel 4.14 now integrates zstd compression / decompression, + primarily for BtrFS and SquashFS, with patches available for reiser4, + zram, and initrd. Kernel version does not support compatibility module, + hence cannot read data from v0.5.1. Userland tools associated with these + services will fail if they link to Ubuntu LTS libzstd, as generating + incompatible data. + + * On top of the incompatible format issue, libzstd v0.5.1 API is old + and missing several features that applications relying on libzstd need, + such as streaming interface, or bulk processing for dictionary + compression. --- Original description --- Please backport libzstd 1.3.1+dfsg-1 (universe) from artful to xenial. Reason for the backport: Current version in Xenial is v0.5.1, it's an experimental version, using its own, incompatible format. As a consequence, zstd on Ubuntu Xenial is not compatible with the rest of the world. This is of important concern for products using libzstd as a shared library : on Ubuntu Xenial, produced data is different, not compatible with v1+. This issue has been made more pressing with the integration of zstd in Linux Kernel, as userland tools must also be updated to read and generate zstd. Note : this request was first improperly filled at : https://bugs.launchpad.net/zesty-backports/+bug/1717037 Testing: Mark off items in the checklist [X] as you test them, but please leave the checklist so that backporters can quickly evaluate the state of testing. Backport available from PPA: https://launchpad.net/~ginggs/+archive/ubuntu/backports * xenial: [X] Package builds without modification (needs debhelper 10) [X] zstd installs cleanly and runs [ ] zstd-dbgsym installs cleanly and runs [ ] libzstd1-dbgsym installs cleanly and runs [X] libzstd-dev installs cleanly and runs [X] libzstd1 installs cleanly and runs No reverse dependencies -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1717040 Title: Please backport libzstd 1.3.1+dfsg-1 (universe) from artful To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/libzstd/+bug/1717040/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1717040] Re: Please backport libzstd 1.3.1+dfsg-1 (universe) from artful
** Also affects: libzstd (Ubuntu Xenial) Importance: Undecided Status: New ** Also affects: libzstd (Ubuntu Zesty) Importance: Undecided Status: New -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1717040 Title: Please backport libzstd 1.3.1+dfsg-1 (universe) from artful To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/libzstd/+bug/1717040/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1717040] Re: Please backport libzstd 1.3.1+dfsg-1 (universe) from artful
** Also affects: libzstd (Ubuntu) Importance: Undecided Status: New ** Changed in: libzstd (Ubuntu) Status: New => Confirmed ** No longer affects: libzstd (Ubuntu) ** Project changed: xenial-backports => libzstd (Ubuntu) ** Description changed: + [Impact] + + * An explanation of the effects of the bug on users and + + * justification for backporting the fix to the stable release. + + * In addition, it is helpful, but not required, to include an +explanation of how the upload fixes this bug. + + [Test Case] + + * detailed instructions how to reproduce the bug + + * these should allow someone who is not familiar with the affected +package to reproduce the bug and verify that the updated package fixes +the problem. + + [Regression Potential] + + * discussion of how regressions are most likely to manifest as a result + of this change. + + * It is assumed that any SRU candidate patch is well-tested before +upload and has a low overall risk of regression, but it's important +to make the effort to think about what ''could'' happen in the +event of a regression. + + * This both shows the SRU team that the risks have been considered, +and provides guidance to testers in regression-testing the SRU. + + [Other Info] + + * Anything else you think is useful to include + * Anticipate questions from users, SRU, +1 maintenance, security teams and the Technical Board + * and address these questions in advance + + + --- Original description --- + Please backport libzstd 1.3.1+dfsg-1 (universe) from artful to xenial. Reason for the backport: Current version in Xenial is v0.5.1, it's an experimental version, using its own, incompatible format. As a consequence, zstd on Ubuntu Xenial is not compatible with the rest of the world. This is of important concern for products using libzstd as a shared library : on Ubuntu Xenial, produced data is different, not compatible with v1+. This issue has been made more pressing with the integration of zstd in Linux Kernel, as userland tools must also be updated to read and generate zstd. Note : this request was first improperly filled at : https://bugs.launchpad.net/zesty-backports/+bug/1717037 Testing: Mark off items in the checklist [X] as you test them, but please leave the checklist so that backporters can quickly evaluate the state of testing. Backport available from PPA: https://launchpad.net/~ginggs/+archive/ubuntu/backports * xenial: [X] Package builds without modification (needs debhelper 10) [X] zstd installs cleanly and runs [ ] zstd-dbgsym installs cleanly and runs [ ] libzstd1-dbgsym installs cleanly and runs [X] libzstd-dev installs cleanly and runs [X] libzstd1 installs cleanly and runs No reverse dependencies -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1717040 Title: Please backport libzstd 1.3.1+dfsg-1 (universe) from artful To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/libzstd/+bug/1717040/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs