Hi Wayne! Thanks for commenting. > It's only the 128-bit hash that depends on 0.8.0. > The 0.7 version works fine with rsync, giving it > the 64-bit and 32-bit hashes.
Yes. Except it seems that if you switch the libxxhash0 from 0.8 to 0.7, you get different behaviour. rsync doesn't check what kind of xxh128 is produced, so we end up with a mismatch. Steps to reproduce: wget -q https://launchpad.net/ubuntu/+archive/primary/+files/rsync_3.2.3-2ubuntu1_amd64.deb wget -q https://launchpad.net/ubuntu/+archive/primary/+files/libxxhash0_0.7.3-1_amd64.deb wget -q https://launchpad.net/ubuntu/+archive/primary/+files/libxxhash0_0.8.0-1ubuntu1.20.10.1_amd64.deb focal-node-1: sudo dpkg -i libxxhash0_0.7.3-1_amd64.deb \ rsync_3.2.3-2ubuntu1_amd64.deb touch empty-file.txt echo A > non-empty-file.txt focal-node-2: sudo dpkg -i rsync_3.2.3-2ubuntu1_amd64.deb \ libxxhash0_0.8.0-1ubuntu1.20.10.1_amd64.deb rsync -v --debug=nstr \ focal-node-1:*empty-file.txt \ . Result: Client negotiated checksum: xxh128 empty-file.txt WARNING: empty-file.txt failed verification -- update discarded (will try again). non-empty-file.txt WARNING: non-empty-file.txt failed verification -- update discarded (will try again). empty-file.txt ERROR: empty-file.txt failed verification -- update discarded. non-empty-file.txt ERROR: non-empty-file.txt failed verification -- update discarded. sent 104 bytes received 255 bytes 239.33 bytes/sec total size is 2 speedup is 0.01 rsync error: some files/attrs were not transferred (see previous errors) (code 23) at main.c(1816) [generator=3.2.3] focal-node-2: $ ls *empty* ls: cannot access '*empty*': No such file or directory I don't mind if I don't get xxh128 and get some poorer hash. But I _do_ mind if I get a hash that produces different results. If I install libxxhash0 0.7.3 on both: things work. If I install libxxhash0 0.8.x on both: things work. But when there is a mismatch, things break. And uselessly too. I ended up syncing lots of GBs multiple times because our job kept retrying. I hope that clarifies the situation. Walter P.S. Alternative solutions could be: - not exporting xxh128 functions from libxxhash0 0.7.3 (but it might be a bit late for that); - checking that xxh128 produces sane values in rsync before choosing that option. -- You received this bug notification because you are a member of Ubuntu Touch seeded packages, which is subscribed to rsync in Ubuntu. https://bugs.launchpad.net/bugs/1934992 Title: rsync 3.2.x in Groovy depends on broken libxxhash 0.7.x Status in rsync package in Ubuntu: New Bug description: **Problem** $ rsync root@focal-system:/etc/.pwd.lock . ERROR: .pwd.lock failed verification -- update discarded. rsync error: some files/attrs were not transferred (see previous errors) (code 23) at main.c(1816) [generator=3.2.3] $ rsync root@focal-system:/etc/.pwd.lock . --debug=all opening connection using: ssh -l root focal-system rsync --server --sender \ -e.LsfxCIvu . /etc/.pwd.lock (10 args) (Client) Protocol versions: remote=31, negotiated=31 Client negotiated checksum: xxh128 ... **Cause** focal-system# dpkg -l | grep -E 'libxxhash|rsync' ii libxxhash0:amd64 0.7.3-1 amd64 ii rsync 3.2.3-2ubuntu1 amd64 **Why this affects only us and not more people?** On Ubuntu/Focal, there is no rsync 3.2.3, only 3.1.3-8. But because we need the lz4 compression support we've fetched a newer rsync (from Groovy). However: the rsync 3.2.3 depends on libxxhash0 0.7.1+, while in fact it needs 0.8+. **Details** On a Ubuntu/Focal system we have installed a rsync 3.2.3 package from Ubuntu/Groovy because we need the lz4 compression support. focal-system# apt-cache show rsync Package: rsync ... Version: 3.2.3-2ubuntu1 Depends: lsb-base, libacl1 (>= 2.2.23), libc6 (>= 2.15), liblz4-1 (>= 0.0~r130), libpopt0 (>= 1.14), libssl1.1 (>= 1.1.0), libxxhash0 (>= 0.7.1), libzstd1 (>= 1.3.8), zlib1g (>= 1:1.1.4) ... Alongside this we had libxxhash0 0.7.3-1 from Focal: focal-system# apt-cache policy libxxhash0 libxxhash0: Installed: 0.7.3-1 Candidate: 0.7.3-1 Version table: *** 0.7.3-1 500 500 http://ARCHIVE/ubuntu focal/universe amd64 Packages 100 /var/lib/dpkg/status According to the dependencies, this should work. But the combination does not, as this quote from the rsync maintainer would tell you: https://github.com/WayneD/rsync/issues/122#issuecomment-737690913 > Yeah, Cyan4973 could have told you that the 128-bit xxhash only > just stabilized in its 0.8.0 release, so anything older than > that isn't compatible. **The fix** As the maintainer points out, version 0.7 is not stable (= broken for our intents and purposes) and thus not fit for use with rsync 3.2. I would argue that it's a good idea to bump the dependency of rsync 3.2.3 on Groovy to libxxhash0>=0.8 After all, in Groovy there is a libxxhash0 0.8.0-1ubuntu1.20.10.1, so that would not be a problem. And it would fix issues for those mixing and matching packages. Thanks! Walter Doekes OSSO B.V. (*) possible patch: $ diff -pu debian/control{.orig,} --- debian/control.orig 2021-07-08 09:56:57.646861644 +0200 +++ debian/control 2021-07-08 09:57:38.499029903 +0200 @@ -8,7 +8,7 @@ Build-Depends: debhelper-compat (= 13), libacl1-dev, libpopt-dev, liblz4-dev, - libxxhash-dev, + libxxhash-dev (>= 0.8), libzstd-dev, zlib1g-dev, libssl-dev To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/rsync/+bug/1934992/+subscriptions -- Mailing list: https://launchpad.net/~touch-packages Post to : touch-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~touch-packages More help : https://help.launchpad.net/ListHelp
