Bug#851285: Fails to import lbzip2: failed to generate delta
Hi Tomasz, On Fri, Jan 13, 2017 at 10:43:38PM +0100, Tomasz Buchert wrote: > On 13/01/17 18:56, Guido Günther wrote: > > On Fri, Jan 13, 2017 at 06:21:24PM +0100, Guido Günther wrote: > > > Package: pristine-tar > > > Version: 1.37 > > > Severity: normal > > > > > > Hi, > > > importing lbzip2 fails with: > > > > > [...] > > > > libxfce4ui, nsis, thunar-volman, thunar, tumbler, xfce4-cpufreq-plugin, > > xfce4-dev-tools, xfce4-dict, xfce4-weather-plugin fail with the same > > error so I'm not filing separate reports. > > Cheers, > > -- Guido > > I didn't check all of these packages, but indeed they do not > reproduce. However, note that these packages do not use pristine-tar > (or gbp, if my sample was significant), so nobody ever tried to > reproduce their tarballs with pristine-tar. :) They do not now but they might in the future. My goal here was to make gbp import-dsc work for all packages in the archive [0] for several reasons: * Making it simple to work on random packages by doing a gbp import-dsc (e.g. for security work or doing NMUs) - this requires an exact tarball. * Turning on pristine-tar by default in gbp as discussed at DC16[1] * (Potentially) create a history of "gbp import-dscs --debsnap" where others can fetch from (this doesn't require a reproducible tarball for all versions as long as we let users know (e.g. for ancient versions) pristine-tar actually already does an amazing job. We only have about 20 packages failing and the packages in this this bug form the largest group (followed by some others that claim to have too big delta, these are usually quiet big packages like chromium and I need to investigate this separtely). > The issue we have here is that it is impossible to "gbp import-dsc > ..." a random package, because in general tarballs are messy. Correct > me if I'm wrong, but it looks as if you don't intend to actually keep > pristine-tar for anything more than temporary help to use gbp with the > imported package? We're also using pristine-tar in team maintained packages to have one repository we can work from. This makes it a lot simpler for people not that familiar with Debian to just git clone one thing and then have everything at hand. > If the answer is yes, then I propose to introduce a mode to > pristine-tar that will: (1) try to import the given tarball using the > standard machinery and if that fails (2) import the final tarball > as-it-is. This sort of defeats the whole idea of pristine-tar, but may > be a necessary evil for this sort of use and will not harm anyone if > people won't use it to actually import tarballs into long-term package > repositories. What do you think? I intend to make "gbp import-dsc" just continue the import and warn the user (#851287) when pristine-tar fails. For import-orig we already roll back all changes and let the user retry without --pristine-tar. So I don't think adding this mode to pristine-tar is really necessary, if we can't create a long term stable tarball we'd better fail and let the user cover up with e.g. by using origtgz. Thanks a lot for a having a look at all the bugs I filed (and for maintaining pristine-tar)! Cheers, -- Guido [0] http://lists.alioth.debian.org/pipermail/vcs-pkg-discuss/2017-January/000954.html [1] http://lists.sigxcpu.org/pipermail/git-buildpackage/2016-July/000143.html
Bug#851285: Fails to import lbzip2: failed to generate delta
On 13/01/17 18:56, Guido Günther wrote: > On Fri, Jan 13, 2017 at 06:21:24PM +0100, Guido Günther wrote: > > Package: pristine-tar > > Version: 1.37 > > Severity: normal > > > > Hi, > > importing lbzip2 fails with: > > > [...] > > libxfce4ui, nsis, thunar-volman, thunar, tumbler, xfce4-cpufreq-plugin, > xfce4-dev-tools, xfce4-dict, xfce4-weather-plugin fail with the same > error so I'm not filing separate reports. > Cheers, > -- Guido I didn't check all of these packages, but indeed they do not reproduce. However, note that these packages do not use pristine-tar (or gbp, if my sample was significant), so nobody ever tried to reproduce their tarballs with pristine-tar. :) The issue we have here is that it is impossible to "gbp import-dsc ..." a random package, because in general tarballs are messy. Correct me if I'm wrong, but it looks as if you don't intend to actually keep pristine-tar for anything more than temporary help to use gbp with the imported package? If the answer is yes, then I propose to introduce a mode to pristine-tar that will: (1) try to import the given tarball using the standard machinery and if that fails (2) import the final tarball as-it-is. This sort of defeats the whole idea of pristine-tar, but may be a necessary evil for this sort of use and will not harm anyone if people won't use it to actually import tarballs into long-term package repositories. What do you think? Tomasz signature.asc Description: PGP signature
Bug#851285: Fails to import lbzip2: failed to generate delta
On Fri, Jan 13, 2017 at 06:21:24PM +0100, Guido Günther wrote: > Package: pristine-tar > Version: 1.37 > Severity: normal > > Hi, > importing lbzip2 fails with: > > $ gbp import-dsc apt://lbzip2/sid > gbp:info: Downloading 'lbzip2/sid' using 'apt-get'... > gbp:info: No git repository found, creating one. > gbp:info: Tag upstream/2.5 not found, importing Upstream tarball > gbp:debug: /usr/bin/pristine-tar [] ['commit', > '/tmp/tmpbShE_a/lbzip2_2.5.orig.tar.bz2', > '21bccd0fc0f8ee51e2a1b7d19949a319ee62a978'] > gbp:error: Couldn't commit to 'pristine-tar' with upstream > '21bccd0fc0f8ee51e2a1b7d19949a319ee62a978': pristine-bz2 failed to reproduce > build of /tmp/tmp4R3hbq/lbzip2_2.5.orig.tar.bz2 > (Please file a bug report.) > pristine-tar: command failed: pristine-bz2 --no-verbose --no-debug --no-keep > gendelta /tmp/tmp4R3hbq/lbzip2_2.5.orig.tar.bz2 > /tmp/pristine-tar.Oq_RoNQ9gp/wrapper > pristine-tar: failed to generate delta > > I've compared the tree genrated by gbp with the one downloaded via > apt-get source without any delta so the gbp end looks sane to me. libxfce4ui, nsis, thunar-volman, thunar, tumbler, xfce4-cpufreq-plugin, xfce4-dev-tools, xfce4-dict, xfce4-weather-plugin fail with the same error so I'm not filing separate reports. Cheers, -- Guido
Bug#851285: Fails to import lbzip2: failed to generate delta
Package: pristine-tar Version: 1.37 Severity: normal Hi, importing lbzip2 fails with: $ gbp import-dsc apt://lbzip2/sid gbp:info: Downloading 'lbzip2/sid' using 'apt-get'... gbp:info: No git repository found, creating one. gbp:info: Tag upstream/2.5 not found, importing Upstream tarball gbp:debug: /usr/bin/pristine-tar [] ['commit', '/tmp/tmpbShE_a/lbzip2_2.5.orig.tar.bz2', '21bccd0fc0f8ee51e2a1b7d19949a319ee62a978'] gbp:error: Couldn't commit to 'pristine-tar' with upstream '21bccd0fc0f8ee51e2a1b7d19949a319ee62a978': pristine-bz2 failed to reproduce build of /tmp/tmp4R3hbq/lbzip2_2.5.orig.tar.bz2 (Please file a bug report.) pristine-tar: command failed: pristine-bz2 --no-verbose --no-debug --no-keep gendelta /tmp/tmp4R3hbq/lbzip2_2.5.orig.tar.bz2 /tmp/pristine-tar.Oq_RoNQ9gp/wrapper pristine-tar: failed to generate delta I've compared the tree genrated by gbp with the one downloaded via apt-get source without any delta so the gbp end looks sane to me. Cheers, -- Guido -- System Information: Debian Release: stretch/sid APT prefers testing APT policy: (990, 'testing'), (500, 'unstable-debug'), (500, 'testing-debug'), (500, 'stable-updates'), (500, 'unstable'), (500, 'stable'), (500, 'oldstable'), (1, 'experimental') Architecture: amd64 (x86_64) Foreign Architectures: i386 Kernel: Linux 4.9.0-rc8-amd64 (SMP w/4 CPU cores) Locale: LANG=de_DE.UTF-8, LC_CTYPE=de_DE.UTF-8 (charmap=UTF-8) Shell: /bin/sh linked to /bin/dash Init: systemd (via /run/systemd/system) Versions of packages pristine-tar depends on: ii libbz2-1.0 1.0.6-8 ii libc6 2.24-8 ii perl5.24.1~rc4-1 ii tar 1.29b-1.1 ii xdelta 1.1.3-9.1 ii xdelta3 3.0.11-dfsg-1 ii zlib1g 1:1.2.8.dfsg-4 Versions of packages pristine-tar recommends: ii bzip2 1.0.6-8 ii pbzip21.1.9-1 ii xz-utils 5.2.2-1.2 pristine-tar suggests no packages. -- no debconf information