tags 628779 + patch thanks > Aha, I caught it in action. Looks like possibly content-autonegotiation
The cause is definitely an out of sync parent mirror (They've ignored my previous requests to run their mirror properly). However I get free access to it, so I'm prepared to live with the pain. The Package lists are failing verification against Release, but debmirror isn't checking the Packages lists after download, only before-hand. Thus it notices Packages.gz is invalid after downloading it, downloads Packages too, pulls bz2 due to autonegotiation, and then tries to parse this. Easy way to reproduce these issues: Put a mirror behind a normally configured apache (with content autonegotiation enabled) Break the Packages files. debmirror from it. Attached are two patches, both a bit hacky, to work-around this: 0001-Detect-errors-when-parsing-Packages.patch: Abort when running into a parse error, and refuse to clean up if there were errors. 0002-Before-download-Package-list-files-are-checked-with-.patch: Before download, Package list files are checked with prepended $tempdir. At download time, Package list files are verified without it. Store both forms in file_lists. SR -- Stefano Rivera http://tumbleweed.org.za/ H: +27 21 465 6908 C: +27 72 419 8559 UCT: x3127
From 5c6e9506e63f5f818e812e936a011a049cbef2c2 Mon Sep 17 00:00:00 2001 From: Stefano Rivera <[email protected]> Date: Mon, 6 Jun 2011 09:18:52 +0200 Subject: [PATCH 1/2] Detect errors when parsing Packages --- debmirror | 3 ++- 1 files changed, 2 insertions(+), 1 deletions(-) diff --git a/debmirror b/debmirror index ae50245..ce641c5 100755 --- a/debmirror +++ b/debmirror @@ -1071,6 +1071,7 @@ say("Parsing Packages and Sources files ..."); die "$file: $!" if $!; } my ($filename)=m/^Filename:\s+(.*)/im; + die "Error encountered while parsing $file." if !defined($filename); $filename=~s:/+:/:; # remove redundant slashes in paths my ($deb_section)=m/^Section:\s+(.*)/im; my ($deb_priority)=m/^Priority:\s+(.*)/im; @@ -1398,7 +1399,7 @@ if (! $do_dry_run) { } # Post mirror cleanup. -cleanup_unknown_files() if ($post_cleanup && ! $debmarshal); +cleanup_unknown_files() if ($post_cleanup && ! $debmarshal && $num_errors == 0); # Mirror cleanup for directories. if (! $use_cache && ($pre_cleanup || $post_cleanup)) { -- 1.7.4.4
From c9c758a208cba6e42023136a60a3d46cdb4df629 Mon Sep 17 00:00:00 2001 From: Stefano Rivera <[email protected]> Date: Mon, 6 Jun 2011 12:04:21 +0200 Subject: [PATCH 2/2] Before download, Package list files are checked with prepended $tempdir. At download time, Package list files are verified without it. Store both forms in file_lists. --- debmirror | 5 ++++- 1 files changed, 4 insertions(+), 1 deletions(-) diff --git a/debmirror b/debmirror index ce641c5..520083e 100755 --- a/debmirror +++ b/debmirror @@ -906,9 +906,12 @@ foreach my $dist (keys %distset) { $checksum_type=$1; } elsif (/^ / && defined $checksum_type) { - my ($checksum, $size, $filename) = /^ +([a-z0-9]+) +(\d+) +(.*)$/; + my ($checksum, $size, $filename) = /^ +([a-z0-9]+) +(\d+) +(.*)$/; $file_lists{"$tempdir/dists/$dist/$filename"}{$checksum_type} = $checksum; $file_lists{"$tempdir/dists/$dist/$filename"}{size} = $size; + # At download time, files are verified without $tempdir + $file_lists{"dists/$dist/$filename"}{$checksum_type} = $checksum; + $file_lists{"dists/$dist/$filename"}{size} = $size; } } close RELEASE; -- 1.7.4.4
