Bug#933031: pristine-tar: unable to unpack some deltas of version 2
Control: tags -1 +patch
On Tue, Jul 28, 2020 at 04:17:08PM +0100, Steve McIntyre wrote:
>Control: severity -1 grave
...
>IMHO this has to be a grave bug - without reimporting this repo we
>can't get our older revisions back. Then I'm worried that this will
>break things if we need to go back and rebuild older versions (using
>older tooling). pristine-tar needs to be safe for data here. :-(
>
>I've done a git bisect of the pristine-tar repo to see where things
>broke, and that clearly tells me:
>
>...
>f287f10b48c3ca5a30db13dbf3ba918c47a943bb is the first bad commit
>commit f287f10b48c3ca5a30db13dbf3ba918c47a943bb
>Author: Tomasz Buchert
>Date: Fri Jan 13 23:28:04 2017 +0100
>
>fix #851286
>
>:100755 100755 1a34e953ba363ae1e284b97f83f63f6ef80ea544
>3ba76f79d376d011451b8c572b20e185b715bed1 M pristine-tar
>:04 04 84bc809be6b4dcf81cfce71f0550858957a3e92f
>f8eb72beb458cbd9770f88f0f4efc00f0c75a299 M test
>bisect run success
>...
>
>I hope that helps the maintainers - shout if you'd like more help debugging.
I've carried on and looked at the code. The attached patch makes
things work for me, at the expense of YA thing added to the @try
loop. But I'll take that over failure!
Having looked at this code a little, I'm now also a little concerned
about the way the fallbacks work, i.e. applied one at a time
independently. I'm thinking that there could well be cases where a
stored tarball might need more than one of the workaround/retry
options. Maybe I'm over-thinking it.
Also submitted this as an MR:
https://salsa.debian.org/debian/pristine-tar/-/merge_requests/7
in case you'd prefer that.
--
Steve McIntyre, Cambridge, UK.st...@einval.com
< sladen> I actually stayed in a hotel and arrived to find a post-it
note stuck to the mini-bar saying "Paul: This fridge and
fittings are the correct way around and do not need altering"
diff --git a/pristine-tar b/pristine-tar
index 0fe132e..081dca1 100755
--- a/pristine-tar
+++ b/pristine-tar
@@ -560,6 +560,33 @@ sub recreatetarball_broken_numeric_owner {
return recreatetarball_helper();
}
+sub recreatetarball_broken_verbatim {
+ # To fix #851286, the option --verbatim-files-from was added by
+ # default. But now some older older stored tarballs won't reproduce
+ # (#933031). Try again *without* that option to tar.
+ my %options = @_;
+ my $tempdir = $recreatetarball_tempdir;
+
+ my $ret = "$tempdir/recreatetarball";
+ my @cmd = (
+$tar_program, "cf",
+$ret, "--owner",
+0,"--group",
+0,"--numeric-owner",
+"-C", "$tempdir/workdir",
+"--no-recursion", "--mode",
+"0644",
+"--files-from", "$tempdir/manifest"
+ );
+ if (exists $options{tar_format}) {
+push @cmd, ("-H", $options{tar_format});
+ }
+
+ doit(@cmd);
+
+ return $ret;
+}
+
sub gentar {
my $deltafile = shift;
my $tarball = shift;
@@ -614,6 +641,7 @@ sub gentar {
%opts
);
};
+ push @try, \_broken_verbatim;
push @try, \_broken_numeric_owner;
my $ok;
Bug#933031: pristine-tar: unable to unpack some deltas of version 2
Control: severity -1 grave On Thu, Jul 25, 2019 at 11:53:39PM +0300, Коля Гурьев wrote: >Package: pristine-tar >Version: 1.46 >Severity: important > >pristine-tar of version 1.46 available in Debian Unstable can't unpack >deltas of versions 2 generated by pristine-tar 1.33 from Ubuntu Xenial. > >I've committed a tarball for the rlottie package into my Git repository >using pristine-tar 1.33. Then I try to regenerate the tarball inside >Debian chroot and get the next error. > >$ pristine-tar --debug --verbose checkout >../rlottie_0~git20190721.24346d0+dfsg.orig.tar.gz >pristine-tar: git archive --format=tar >9fed0d3da5cfa7eabd4fa8c2590dd86e5b1442e1 | (cd '/tmp/pristine-tar.2a5pcCDc3n' >&& tar x) >pristine-tar: tar xf /tmp/pristine-tar.cBbx8nKDp6/tmpin -C >/tmp/pristine-tar.Dvxlxlx8Qn >pristine-tar: set subdir to rlottie >pristine-tar: subdir is rlottie >pristine-tar: mkdir /tmp/pristine-tar.o0lKEjWozz/workdir >pristine-tar: mv /tmp/pristine-tar.2a5pcCDc3n >/tmp/pristine-tar.o0lKEjWozz/workdir/rlottie >pristine-tar: rlottie/example/resource/360\302\272_degree.json is listed in >the manifest but may not be present in the source directory >pristine-tar: creating missing rlottie/example/resource/360\302\272_degree.json >pristine-tar: doing full tree sweep to catch missing files >pristine-tar: tar cf /tmp/pristine-tar.o0lKEjWozz/recreatetarball --owner 0 >--group 0 --numeric-owner -C /tmp/pristine-tar.o0lKEjWozz/workdir >--no-recursion --mode 0644 --verbatim-files-from --files-from >/tmp/pristine-tar.o0lKEjWozz/manifest >pristine-tar: xdelta patch --pristine /tmp/pristine-tar.Dvxlxlx8Qn/delta >/tmp/pristine-tar.o0lKEjWozz/recreatetarball >/tmp/pristine-tar.i1k0xBo1aP/rlottie_0~git20190721.24346d0+dfsg.orig.tar.gz.tmp >xdelta: expected from file (/tmp/pristine-tar.o0lKEjWozz/recreatetarball) of >length 12779520 bytes >pristine-tar: tar cf /tmp/pristine-tar.o0lKEjWozz/recreatetarball --owner 0 >--group 0 --numeric-owner -C /tmp/pristine-tar.o0lKEjWozz/workdir >--no-recursion --mode 0644 --verbatim-files-from --files-from >/tmp/pristine-tar.o0lKEjWozz/manifest >pristine-tar: xdelta patch --pristine /tmp/pristine-tar.Dvxlxlx8Qn/delta >/tmp/pristine-tar.o0lKEjWozz/recreatetarball >/tmp/pristine-tar.i1k0xBo1aP/rlottie_0~git20190721.24346d0+dfsg.orig.tar.gz.tmp >xdelta: expected from file (/tmp/pristine-tar.o0lKEjWozz/recreatetarball) of >length 12779520 bytes >pristine-tar: set subdir to rlottie >pristine-tar: subdir is rlottie >pristine-tar: mkdir /tmp/pristine-tar.4XNCSF8pDG/workdir >pristine-tar: mv /tmp/pristine-tar.o0lKEjWozz/workdir/rlottie >/tmp/pristine-tar.4XNCSF8pDG/workdir/rlottie >pristine-tar: tar cf /tmp/pristine-tar.4XNCSF8pDG/recreatetarball --owner 0 >--group 0 --numeric-owner -C /tmp/pristine-tar.4XNCSF8pDG/workdir >--no-recursion --mode 0644 --verbatim-files-from --files-from >/tmp/pristine-tar.4XNCSF8pDG/manifest -H gnu >pristine-tar: xdelta patch --pristine /tmp/pristine-tar.Dvxlxlx8Qn/delta >/tmp/pristine-tar.4XNCSF8pDG/recreatetarball >/tmp/pristine-tar.i1k0xBo1aP/rlottie_0~git20190721.24346d0+dfsg.orig.tar.gz.tmp >xdelta: expected from file (/tmp/pristine-tar.4XNCSF8pDG/recreatetarball) of >length 12779520 bytes >pristine-tar: set subdir to rlottie >pristine-tar: subdir is rlottie >pristine-tar: mkdir /tmp/pristine-tar.SY9ZY0yfKg/workdir >pristine-tar: mv /tmp/pristine-tar.4XNCSF8pDG/workdir/rlottie >/tmp/pristine-tar.SY9ZY0yfKg/workdir/rlottie >pristine-tar: tar cf /tmp/pristine-tar.SY9ZY0yfKg/recreatetarball --owner 0 >--group 0 --numeric-owner -C /tmp/pristine-tar.SY9ZY0yfKg/workdir >--no-recursion --mode 0644 --verbatim-files-from --files-from >/tmp/pristine-tar.SY9ZY0yfKg/manifest -H posix >pristine-tar: xdelta patch --pristine /tmp/pristine-tar.Dvxlxlx8Qn/delta >/tmp/pristine-tar.SY9ZY0yfKg/recreatetarball >/tmp/pristine-tar.i1k0xBo1aP/rlottie_0~git20190721.24346d0+dfsg.orig.tar.gz.tmp >xdelta: expected from file (/tmp/pristine-tar.SY9ZY0yfKg/recreatetarball) of >length 12779520 bytes >pristine-tar: tar cf /tmp/pristine-tar.SY9ZY0yfKg/recreatetarball --owner 0 >--group 0 --numeric-owner -C /tmp/pristine-tar.SY9ZY0yfKg/workdir >--no-recursion --mode 0644 --verbatim-files-from --files-from >/tmp/pristine-tar.SY9ZY0yfKg/manifest >pristine-tar: xdelta patch --pristine /tmp/pristine-tar.Dvxlxlx8Qn/delta >/tmp/pristine-tar.SY9ZY0yfKg/recreatetarball >/tmp/pristine-tar.i1k0xBo1aP/rlottie_0~git20190721.24346d0+dfsg.orig.tar.gz.tmp >xdelta: expected from file (/tmp/pristine-tar.SY9ZY0yfKg/recreatetarball) of >length 12779520 bytes >pristine-tar: Failed to reproduce original tarball. Please file a bug report. >pristine-tar: failed to generate tarball > >You'll find problematic delta in the repository of the rlottie package >under the mymedia/weird-delta tag. Steps to reproduce: > >git clone https://salsa.debian.org/debian/rlottie.git >git branch pristine-tar mymedia/weird-delta >pristine-tar checkout
Bug#933031: pristine-tar: unable to unpack some deltas of version 2
Package: pristine-tar Version: 1.46 Severity: important pristine-tar of version 1.46 available in Debian Unstable can't unpack deltas of versions 2 generated by pristine-tar 1.33 from Ubuntu Xenial. I've committed a tarball for the rlottie package into my Git repository using pristine-tar 1.33. Then I try to regenerate the tarball inside Debian chroot and get the next error. $ pristine-tar --debug --verbose checkout ../rlottie_0~git20190721.24346d0+dfsg.orig.tar.gz pristine-tar: git archive --format=tar 9fed0d3da5cfa7eabd4fa8c2590dd86e5b1442e1 | (cd '/tmp/pristine-tar.2a5pcCDc3n' && tar x) pristine-tar: tar xf /tmp/pristine-tar.cBbx8nKDp6/tmpin -C /tmp/pristine-tar.Dvxlxlx8Qn pristine-tar: set subdir to rlottie pristine-tar: subdir is rlottie pristine-tar: mkdir /tmp/pristine-tar.o0lKEjWozz/workdir pristine-tar: mv /tmp/pristine-tar.2a5pcCDc3n /tmp/pristine-tar.o0lKEjWozz/workdir/rlottie pristine-tar: rlottie/example/resource/360\302\272_degree.json is listed in the manifest but may not be present in the source directory pristine-tar: creating missing rlottie/example/resource/360\302\272_degree.json pristine-tar: doing full tree sweep to catch missing files pristine-tar: tar cf /tmp/pristine-tar.o0lKEjWozz/recreatetarball --owner 0 --group 0 --numeric-owner -C /tmp/pristine-tar.o0lKEjWozz/workdir --no-recursion --mode 0644 --verbatim-files-from --files-from /tmp/pristine-tar.o0lKEjWozz/manifest pristine-tar: xdelta patch --pristine /tmp/pristine-tar.Dvxlxlx8Qn/delta /tmp/pristine-tar.o0lKEjWozz/recreatetarball /tmp/pristine-tar.i1k0xBo1aP/rlottie_0~git20190721.24346d0+dfsg.orig.tar.gz.tmp xdelta: expected from file (/tmp/pristine-tar.o0lKEjWozz/recreatetarball) of length 12779520 bytes pristine-tar: tar cf /tmp/pristine-tar.o0lKEjWozz/recreatetarball --owner 0 --group 0 --numeric-owner -C /tmp/pristine-tar.o0lKEjWozz/workdir --no-recursion --mode 0644 --verbatim-files-from --files-from /tmp/pristine-tar.o0lKEjWozz/manifest pristine-tar: xdelta patch --pristine /tmp/pristine-tar.Dvxlxlx8Qn/delta /tmp/pristine-tar.o0lKEjWozz/recreatetarball /tmp/pristine-tar.i1k0xBo1aP/rlottie_0~git20190721.24346d0+dfsg.orig.tar.gz.tmp xdelta: expected from file (/tmp/pristine-tar.o0lKEjWozz/recreatetarball) of length 12779520 bytes pristine-tar: set subdir to rlottie pristine-tar: subdir is rlottie pristine-tar: mkdir /tmp/pristine-tar.4XNCSF8pDG/workdir pristine-tar: mv /tmp/pristine-tar.o0lKEjWozz/workdir/rlottie /tmp/pristine-tar.4XNCSF8pDG/workdir/rlottie pristine-tar: tar cf /tmp/pristine-tar.4XNCSF8pDG/recreatetarball --owner 0 --group 0 --numeric-owner -C /tmp/pristine-tar.4XNCSF8pDG/workdir --no-recursion --mode 0644 --verbatim-files-from --files-from /tmp/pristine-tar.4XNCSF8pDG/manifest -H gnu pristine-tar: xdelta patch --pristine /tmp/pristine-tar.Dvxlxlx8Qn/delta /tmp/pristine-tar.4XNCSF8pDG/recreatetarball /tmp/pristine-tar.i1k0xBo1aP/rlottie_0~git20190721.24346d0+dfsg.orig.tar.gz.tmp xdelta: expected from file (/tmp/pristine-tar.4XNCSF8pDG/recreatetarball) of length 12779520 bytes pristine-tar: set subdir to rlottie pristine-tar: subdir is rlottie pristine-tar: mkdir /tmp/pristine-tar.SY9ZY0yfKg/workdir pristine-tar: mv /tmp/pristine-tar.4XNCSF8pDG/workdir/rlottie /tmp/pristine-tar.SY9ZY0yfKg/workdir/rlottie pristine-tar: tar cf /tmp/pristine-tar.SY9ZY0yfKg/recreatetarball --owner 0 --group 0 --numeric-owner -C /tmp/pristine-tar.SY9ZY0yfKg/workdir --no-recursion --mode 0644 --verbatim-files-from --files-from /tmp/pristine-tar.SY9ZY0yfKg/manifest -H posix pristine-tar: xdelta patch --pristine /tmp/pristine-tar.Dvxlxlx8Qn/delta /tmp/pristine-tar.SY9ZY0yfKg/recreatetarball /tmp/pristine-tar.i1k0xBo1aP/rlottie_0~git20190721.24346d0+dfsg.orig.tar.gz.tmp xdelta: expected from file (/tmp/pristine-tar.SY9ZY0yfKg/recreatetarball) of length 12779520 bytes pristine-tar: tar cf /tmp/pristine-tar.SY9ZY0yfKg/recreatetarball --owner 0 --group 0 --numeric-owner -C /tmp/pristine-tar.SY9ZY0yfKg/workdir --no-recursion --mode 0644 --verbatim-files-from --files-from /tmp/pristine-tar.SY9ZY0yfKg/manifest pristine-tar: xdelta patch --pristine /tmp/pristine-tar.Dvxlxlx8Qn/delta /tmp/pristine-tar.SY9ZY0yfKg/recreatetarball /tmp/pristine-tar.i1k0xBo1aP/rlottie_0~git20190721.24346d0+dfsg.orig.tar.gz.tmp xdelta: expected from file (/tmp/pristine-tar.SY9ZY0yfKg/recreatetarball) of length 12779520 bytes pristine-tar: Failed to reproduce original tarball. Please file a bug report. pristine-tar: failed to generate tarball You'll find problematic delta in the repository of the rlottie package under the mymedia/weird-delta tag. Steps to reproduce: git clone https://salsa.debian.org/debian/rlottie.git git branch pristine-tar mymedia/weird-delta pristine-tar checkout ../rlottie_0~git20190721.24346d0+dfsg.orig.tar.gz Here is version numbers of dependencies of both programs. Name Version Architecture Description
