Bug#737057: debmirror: Please add xz support
On 11/08/2015 08:03 PM, Thaddeus H. Black wrote: > Sorry, friends. Obviously too busy, I am not keeping up with Debmirror > work at the moment. You have my blessing to help, to upload, to > hijack, as you think best. I regret having impeded your work in the > meantime. The problem with my patch is that it makes debmirror ignore the translation files which are still compressed with .bz2. Thus, I will have to improve it to support both .bz2 and .xz. Once I have verified that to work, I will NMU the package with my patch to close this bug. Adrian -- .''`. John Paul Adrian Glaubitz : :' : Debian Developer - glaub...@debian.org `. `' Freie Universitaet Berlin - glaub...@physik.fu-berlin.de `-GPG: 62FF 8A75 84E0 2956 9546 0006 7426 3B37 F5B5 F913
Bug#737057: debmirror: Please add xz support
Sorry, friends. Obviously too busy, I am not keeping up with Debmirror work at the moment. You have my blessing to help, to upload, to hijack, as you think best. I regret having impeded your work in the meantime. I do not even have an up-to-date sid installation at the moment against which to compile, so I cannot help right now. Should you absolutely require my prompt attention in the matter (though I doubt that I can help), please copy your messages to thaddeus.h.bl...@gmail.com; or better, phone me after 13:00 UTC but before 01:00 UTC at +1 540 921 1759. On Sat, Sep 12, 2015 at 11:10 AM, John Paul Adrian Glaubitz wrote: > -BEGIN PGP SIGNED MESSAGE- > Hash: SHA256 > > Control: tags -1 patch > > Hi! > > On 09/11/2015 12:08 PM, Cyril Brulebois wrote: >> That doesn't seem unreasonable; helping the new maintainer with a >> few patches would probably be appreciated as well, especially if >> you're a regular user for your local mirror. > > I have patched my local installation of debmirror now to always try to > download xz files instead of bz2 files. This is basically a lazy fix > which solves the issue I currently had with debmirror but introduces > a new regression by not being able to download the Translation files > anymore which, to my big surprise, are still bz2-compressed [1]. > > Since I don't really care about the translations, I just disabled > those in /etc/apt/apt.conf on the clients and I can now continue > to use debmirror normally. For anyone who doesn't know, adding > Acquire::Languages "none"; to the apt configuration is enough. > > Feel free to merge my patch, but be aware that it removes bz2 support > from debmirror completely. However, I expect that the translation > files will move away from bz2 to xz in the future as well, so > this might solve itself then. > > Cheers, > Adrian > >> [1] ftp://ftp.debian.org/debian/dists/sid/main/i18n/ > > - -- > .''`. John Paul Adrian Glaubitz > : :' : Debian Developer - glaub...@debian.org > `. `' Freie Universitaet Berlin - glaub...@physik.fu-berlin.de > `-GPG: 62FF 8A75 84E0 2956 9546 0006 7426 3B37 F5B5 F913 > -BEGIN PGP SIGNATURE- > Version: GnuPG v2 > > iQIcBAEBCAAGBQJV9AgfAAoJEHQmOzf1tfkTm4YP/jIg686bDmDpkcPEA5p/UcR2 > EDAoeqRBMIF3OjPMyDkcOHAr/HsA7xehSLIBtro65kP458BwLWOYisPDhT79N0UE > BfZnHXEcDSZtVz8J9wQFrBB0N9cvLiKd7CDYbbmfiSQJANST7wovu9bkyUs1A8CU > qQgK1BGPGfTxpjcLSNaFCNVMpnEKKqm3LM7NTzYwpOkHTu9vuATW3FM+aklDVvI0 > 49aIPY1yT+vlgoRRfrAwEDHtvAw2dWFDf3XCqpU4gfyebVe84E4noEn4ztCigJ6W > VtrFYz5J5nXzof8Od25uc0q2whigUf8Tx1/RiGGkGR4uoGRZMJY+erqhzEgRNuby > G1Ct1qSK04IO7evTo9rywYy1n/FjsIXXt4aeEAWW+ncb14il10HyCxHT0qnXcfo3 > pYWBNdcBYVaYS/m4gUkro7Ps0Irac0VqV3xkU3XoPpzVRyU9PBQpNpOi+vuNU4Mf > X1IP4fNf+VG2NsA7Nb4Ydv6n63PzEZWDo6zeDJPPgknOutLUZ+GQeEyGLRTu1epG > Vo+2+ekL80NyI4RbyV8xOe3I3k5ccpzmFh5WANjaEea2zTbX0ES6gtyj3BmB2sMn > seJsJO4LyzF49Z3ucBY8Zd6YPGLcRY72PcMfmNrqV6sMpVXkpzmXxc3gXOALyW8H > SvtIwWav/Cj5D6i+K+6l > =LRQZ > -END PGP SIGNATURE-
Bug#737057: debmirror: Please add xz support
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA256
Control: tags -1 patch
Hi!
On 09/11/2015 12:08 PM, Cyril Brulebois wrote:
> That doesn't seem unreasonable; helping the new maintainer with a
> few patches would probably be appreciated as well, especially if
> you're a regular user for your local mirror.
I have patched my local installation of debmirror now to always try to
download xz files instead of bz2 files. This is basically a lazy fix
which solves the issue I currently had with debmirror but introduces
a new regression by not being able to download the Translation files
anymore which, to my big surprise, are still bz2-compressed [1].
Since I don't really care about the translations, I just disabled
those in /etc/apt/apt.conf on the clients and I can now continue
to use debmirror normally. For anyone who doesn't know, adding
Acquire::Languages "none"; to the apt configuration is enough.
Feel free to merge my patch, but be aware that it removes bz2 support
from debmirror completely. However, I expect that the translation
files will move away from bz2 to xz in the future as well, so
this might solve itself then.
Cheers,
Adrian
> [1] ftp://ftp.debian.org/debian/dists/sid/main/i18n/
- --
.''`. John Paul Adrian Glaubitz
: :' : Debian Developer - glaub...@debian.org
`. `' Freie Universitaet Berlin - glaub...@physik.fu-berlin.de
`-GPG: 62FF 8A75 84E0 2956 9546 0006 7426 3B37 F5B5 F913
-BEGIN PGP SIGNATURE-
Version: GnuPG v2
iQIcBAEBCAAGBQJV9AgfAAoJEHQmOzf1tfkTm4YP/jIg686bDmDpkcPEA5p/UcR2
EDAoeqRBMIF3OjPMyDkcOHAr/HsA7xehSLIBtro65kP458BwLWOYisPDhT79N0UE
BfZnHXEcDSZtVz8J9wQFrBB0N9cvLiKd7CDYbbmfiSQJANST7wovu9bkyUs1A8CU
qQgK1BGPGfTxpjcLSNaFCNVMpnEKKqm3LM7NTzYwpOkHTu9vuATW3FM+aklDVvI0
49aIPY1yT+vlgoRRfrAwEDHtvAw2dWFDf3XCqpU4gfyebVe84E4noEn4ztCigJ6W
VtrFYz5J5nXzof8Od25uc0q2whigUf8Tx1/RiGGkGR4uoGRZMJY+erqhzEgRNuby
G1Ct1qSK04IO7evTo9rywYy1n/FjsIXXt4aeEAWW+ncb14il10HyCxHT0qnXcfo3
pYWBNdcBYVaYS/m4gUkro7Ps0Irac0VqV3xkU3XoPpzVRyU9PBQpNpOi+vuNU4Mf
X1IP4fNf+VG2NsA7Nb4Ydv6n63PzEZWDo6zeDJPPgknOutLUZ+GQeEyGLRTu1epG
Vo+2+ekL80NyI4RbyV8xOe3I3k5ccpzmFh5WANjaEea2zTbX0ES6gtyj3BmB2sMn
seJsJO4LyzF49Z3ucBY8Zd6YPGLcRY72PcMfmNrqV6sMpVXkpzmXxc3gXOALyW8H
SvtIwWav/Cj5D6i+K+6l
=LRQZ
-END PGP SIGNATURE-
--- /usr/bin/debmirror~ 2013-06-23 18:34:26.0 +0200
+++ /usr/bin/debmirror 2015-09-11 12:22:44.329419806 +0200
@@ -1029,7 +1029,7 @@
foreach my $arch (@arches) {
add_bytes("dists/$dist/$section/binary-$arch/Packages");
add_bytes("dists/$dist/$section/binary-$arch/Packages.gz");
- add_bytes("dists/$dist/$section/binary-$arch/Packages.bz2");
+ add_bytes("dists/$dist/$section/binary-$arch/Packages.xz");
add_bytes("dists/$dist/$section/binary-$arch/Release");
add_bytes("dists/$dist/$section/binary-$arch/Packages.diff/Index") unless ($diff_mode eq "none");
}
@@ -1037,7 +1037,7 @@
if ($do_source && $section !~ /debian-installer/) {
add_bytes("dists/$dist/$section/source/Sources");
add_bytes("dists/$dist/$section/source/Sources.gz");
- add_bytes("dists/$dist/$section/source/Sources.bz2");
+ add_bytes("dists/$dist/$section/source/Sources.xz");
add_bytes("dists/$dist/$section/source/Release");
add_bytes("dists/$dist/$section/source/Sources.diff/Index") unless ($diff_mode eq "none");
}
@@ -2180,7 +2180,7 @@
if (check_lists("$tempdir/$subdir/$file")) {
if (! $slow_cpu) {
system_redirect_io("gzip $gzip_options", "$tempdir/$subdir/$file", "$tempdir/$subdir/$file.gz");
- system_redirect_io("bzip2", "$tempdir/$subdir/$file", "$tempdir/$subdir/$file.bz2");
+ system_redirect_io("xz", "$tempdir/$subdir/$file", "$tempdir/$subdir/$file.xz");
}
}
}
@@ -2190,7 +2190,7 @@
if (check_lists("$tempdir/$subdir/$file")) {
if (! $slow_cpu) {
system_redirect_io("gzip $gzip_options", "$tempdir/$subdir/$file", "$tempdir/$subdir/$file.gz");
- system_redirect_io("bzip2", "$tempdir/$subdir/$file", "$tempdir/$subdir/$file.bz2");
+ system_redirect_io("xz", "$tempdir/$subdir/$file", "$tempdir/$subdir/$file.xz");
}
}
}
@@ -2203,7 +2203,7 @@
if (remote_get("$subdir/$file.gz")) {
system_redirect_io("gzip -d", "$tempdir/$subdir/$file.gz", "$tempdir/$subdir/$file");
if (! $slow_cpu) {
- system_redirect_io("bzip2", "$tempdir/$subdir/$file", "$tempdir/$subdir/$file.bz2");
+ system_redirect_io("xz", "$tempdir/$subdir/$file", "$tempdir/$subdir/$file.xz");
}
} else {
push (@errlog,"$subdir/$file.gz failed checksum verification\n");
@@ -2230,7 +2230,7 @@
if (!check_lists("$tempdir/$subdir/$file")) {
if (remote_get("$subdir/$file")) {
if (! $slow_cpu) {
- system_redirect_io("bzip2", "$tempdir/$subdir/$file", "$tempdir/$subdir/$file.bz2");
+ system_redirect_io("xz", "$tempdir/$subdir/$file", "$tempdir/$subdir/$file.xz");
}
} else {
push (@errlog,"$subdir/$file failed checksum verification\n");
@@ -2240,13 +2240,13 @@
$bytes_gotte
Bug#737057: debmirror: Please add xz support
Hi, John Paul Adrian Glaubitz (2015-09-11): > This seems to have broken debmirror now as it's no longer > being able to retrieve the Sources.bz2 and therefore generates > it themselves which results in an invalid Sources file. > > We are using debmirror for our local Debian mirror and have > run into problem with the validation of the Sources file: > > root@jessie64:~> apt-get update > Ign file: jessie InRelease > Ign file: jessie/updates InRelease > Ign file: jessie-backports InRelease > (...) > W: Failed to fetch > file:/net/deb/debian/dists/jessie-backports/non-free/source/Sources > Hash Sum mismatch > > E: Some index files failed to download. They have been ignored, or old > ones used instead. > root@jessie64:~> > > I assume the priority of this bug should be raised unless there > is some workaround to fix this. That doesn't seem unreasonable; helping the new maintainer with a few patches would probably be appreciated as well, especially if you're a regular user for your local mirror. I've moved to using archvsync to follow what mirrors are using (so that I can stop caring), so feel free to steal this bug report away from me with a submitter -1 ! call. Mraw, KiBi. signature.asc Description: Digital signature
Bug#737057: debmirror: Please add xz support
Hi! > (Apparently bzip2 might go away, too. > See Ansgar's bug report: http://bugs.debian.org/737037) This seems to have broken debmirror now as it's no longer being able to retrieve the Sources.bz2 and therefore generates it themselves which results in an invalid Sources file. We are using debmirror for our local Debian mirror and have run into problem with the validation of the Sources file: root@jessie64:~> apt-get update Ign file: jessie InRelease Ign file: jessie/updates InRelease Ign file: jessie-backports InRelease (...) W: Failed to fetch file:/net/deb/debian/dists/jessie-backports/non-free/source/Sources Hash Sum mismatch E: Some index files failed to download. They have been ignored, or old ones used instead. root@jessie64:~> I assume the priority of this bug should be raised unless there is some workaround to fix this. Adrian -- .''`. John Paul Adrian Glaubitz : :' : Debian Developer - glaub...@debian.org `. `' Freie Universitaet Berlin - glaub...@physik.fu-berlin.de `-GPG: 62FF 8A75 84E0 2956 9546 0006 7426 3B37 F5B5 F913
Bug#737057: debmirror: Please add xz support
Package: debmirror Version: 1:2.14 Severity: normal Hi, ftpmasters are progressively enabling xz support for Packages files: first done in experimental, then sid and jessie. It would be nice to have debmirror support it as well, which would help make sure net-retriever works as expected. (Apparently bzip2 might go away, too. See Ansgar's bug report: http://bugs.debian.org/737037) Thanks for considering. Mraw, KiBi. -- System Information: Debian Release: 7.3 APT prefers stable-updates APT policy: (500, 'stable-updates'), (500, 'proposed-updates'), (500, 'stable') Architecture: amd64 (x86_64) Kernel: Linux 3.2.0-4-amd64 (SMP w/8 CPU cores) Locale: LANG=en_GB.UTF-8, LC_CTYPE=en_GB.UTF-8 (charmap=UTF-8) Shell: /bin/sh linked to /bin/dash Versions of packages debmirror depends on: ii bzip2 1.0.6-4 pn libdigest-md5-perl ii liblockfile-simple-perl 0.208-1 ii libnet-inet6glue-perl 0.5-1 ii libwww-perl 6.04-1 ii perl [libdigest-sha-perl] 5.14.2-21+deb7u1 ii perl-modules [libnet-perl] 5.14.2-21+deb7u1 ii rsync 3.0.9-4 Versions of packages debmirror recommends: ii ed 1.6-2 ii gpgv 1.4.12-7+deb7u3 ii patch 2.6.1-3 Versions of packages debmirror suggests: ii gnupg 1.4.12-7+deb7u3 -- no debconf information -- To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org
