Russ Allbery wrote: > I discovered that pristine-tar could no longer generate the original > tarballs for openafs, including one that I just created on a different > system. The error messages are: > > wanderer:~/dvl/debian/openafs$ pristine-tar checkout > openafs_1.6.5.1.orig.tar.xz
> xdelta: /tmp/pristine-tar.MRIQzsqRqg/openafs_1.6.5.1.orig.tar.xz.tmp:
> Checksum validation failed, expected: f872ee04506b8baec8e0f64eb5e3c30f,
> received: dde5a1aa469e7221bf9f56cc836becad
> xdelta: /tmp/pristine-tar.yMNZDhZVIq/recreatetarball: Checksum validation
> failed, expected: 0172fb743ec9492629d5e0f172848ac9, received:
> a3b994c2709d720e8a091d9448b11532
> xdelta: /tmp/pristine-tar.MRIQzsqRqg/openafs_1.6.5.1.orig.tar.xz.tmp:
> Checksum validation failed, expected: f872ee04506b8baec8e0f64eb5e3c30f,
> received: dde5a1aa469e7221bf9f56cc836becad
> xdelta: /tmp/pristine-tar.yMNZDhZVIq/recreatetarball: Checksum validation
> failed, expected: 0172fb743ec9492629d5e0f172848ac9, received:
> a3b994c2709d720e8a091d9448b11532
> pristine-tar: Failed to reproduce original tarball. Please file a bug report.
> pristine-tar: failed to generate tarball
>
> The failure happens fairly quickly (I'm used to checkouts of xz files
> taking a noticable amount of time), so I believe from that and from
> the nature of the error messages that it's failing before the attempted
> xz compression.
>
> I then checked several older ones and they all fail as well. You can
> reproduce from the public OpenAFS packaging repository at:
>
> git://anonscm.debian.org/pkg-k5-afs/openafs.git
>
> I am able to check out these tarballs on another system that's running
> i386 instead of amd64 and (I suspect more relevantly) tar 1.26+dfsg-8.
> The system where this fails has tar 1.27-1. Maybe something changed
> in the new release?
Everything you say is consistent with tar's output changing. However, it does
not seem to be a general change affecting all packages; I can still
pristine-tar checkout old tarballs of pristine-tar with tar 1.27 installed.
Per investigation below, this bug probably affects all tarballs with files
long enough for tar to use a LongLink.
Let's produce two uncompressed tarballs using the two versions of tar with
--owner 0 --group 0 --numeric-owner --no-recursion --mode 0644 which should
result in stable output when given the same set of input files from openafs,
and compare them to see what causes them to differ. pristine-tar -vdk makes
this easy, just use the recreatetarball file it creates.
joey@wren:~/tmp/openafs#master-1.4>cmp
/home/joey/tmp/pristine-tar.epyfH0IIw6/recreatetarball
/home/joey/tmp/pristine-tar.3gBpeCiihU/recreatetarball
/home/joey/tmp/pristine-tar.epyfH0IIw6/recreatetarball
/home/joey/tmp/pristine-tar.3gBpeCiihU/recreatetarball differ: byte 31013481,
line 869603
Interesting, that's a long way into the file for them to start to
differ! Let's compare hexdumps..
--- old 2013-10-21 11:08:02.000000000 -0400
+++ new 2013-10-21 11:08:19.000000000 -0400
@@ -1836579,10 +1836579,10 @@
01d93a00 2e 2f 2e 2f 40 4c 6f 6e 67 4c 69 6e 6b 00 00 00 |././@LongLink...|
01d93a10 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................|
*
-01d93a60 00 00 00 00 30 30 30 30 30 30 30 00 30 30 30 30 |....0000000.0000|
+01d93a60 00 00 00 00 30 30 30 30 36 34 34 00 30 30 30 30 |....0000644.0000|
01d93a70 30 30 30 00 30 30 30 30 30 30 30 00 30 30 30 30 |000.0000000.0000|
-01d93a80 30 30 30 30 31 34 36 00 30 30 30 30 30 30 30 30 |0000146.00000000|
-01d93a90 30 30 30 00 30 31 31 35 36 36 00 20 4c 00 00 00 |000.011566. L...|
+01d93a80 30 30 30 30 31 34 36 00 31 32 32 33 31 32 34 31 |0000146.12231241|
+01d93a90 31 35 37 00 30 31 31 36 34 31 00 20 4c 00 00 00 |157.011641. L...|
01d93aa0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................|
*
01d93b00 00 75 73 74 61 72 20 20 00 72 6f 6f 74 00 00 00 |.ustar .root...|
@@ -1836617,10 +1836617,10 @@
01d94000 2e 2f 2e 2f 40 4c 6f 6e 67 4c 69 6e 6b 00 00 00 |././@LongLink...|
01d94010 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................|
*
-01d94060 00 00 00 00 30 30 30 30 30 30 30 00 30 30 30 30 |....0000000.0000|
+01d94060 00 00 00 00 30 30 30 30 36 34 34 00 30 30 30 30 |....0000644.0000|
01d94070 30 30 30 00 30 30 30 30 30 30 30 00 30 30 30 30 |000.0000000.0000|
-01d94080 30 30 30 30 31 36 31 00 30 30 30 30 30 30 30 30 |0000161.00000000|
-01d94090 30 30 30 00 30 31 31 35 36 33 00 20 4c 00 00 00 |000.011563. L...|
+01d94080 30 30 30 30 31 36 31 00 31 32 32 33 31 32 34 31 |0000161.12231241|
+01d94090 31 35 37 00 30 31 31 36 33 36 00 20 4c 00 00 00 |157.011636. L...|
01d940a0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................|
Just by inspection, this looks to me that old tar used permission 0 for
LongLink entries, and the new tar is using the 0644 set by the --mode
parameter. Let's see..
struct posix_header
{ /* byte offset */
char name[100]; /* 0 */
char mode[8]; /* 100 */
char uid[8]; /* 108 */
char gid[8]; /* 116 */
char size[12]; /* 124 */
char mtime[12]; /* 136 */
char chksum[8]; /* 148 */
char typeflag; /* 156 */
char linkname[100]; /* 157 */
So the differing fields are mode, mtime (and chksum, probably derived
from the others).
While embedding the tar generation code from tar 1.26 into pristine-tar is
always an option, it seems quite feasable to add another workaround to the
debian tar package, as was done earlier in #602907.
Comparing the write_gnu_long_link functions, it's quite clear what's
changed:
- header = start_private_header ("././@LongLink", size, time (NULL));
+ header = start_private_header ("././@LongLink", size, start_time.tv_sec);
- FILL (header->header.mtime, '0');
- FILL (header->header.mode, '0');
- FILL (header->header.uid, '0');
- FILL (header->header.gid, '0');
- FILL (header->header.devmajor, 0);
- FILL (header->header.devminor, 0);
Patch to tar almost writes itself, but I don't quite have time to put it
together right now. Perhaps someone will be motivated to do it before I
get the time next weekend or so? Also, Bdale will you mind if I inflict
another patch on you?
--
see shy jo
signature.asc
Description: Digital signature
