Okay, here is a revised patch implementing the feature. This time,
apt-cacher-cleanup.pl and the checksum database are onboard. I also
tweaked the long-filename convention to use "::" right before the actual
package name, to make the names easier to parse in Perl. (Colons can
also appear in the filename as part of a host:port combination, so this
kills any ambiguity.)
The caveat, however, is that I no longer believe mixing long- and short-
filename package files is workable. It's simple if all you're worried
about is the filename lookup in handle_connection(), but it gets very
awkward at the point of doing cache cleanup and checksum validation. And
you can't punt on these, because then the two-step lookup means that a
short-named package file is never superceded by a newer one from the
repository.
So I think a new upgrade/import script will be needed. The general
procedure I'm thinking of is along these lines:
1. Read in all the package index files, like apt-cacher-cleanup.pl
already does.
2. From the big %valid hash, build up an MD5-sum-to-long-filename lookup
hash. (The mapping might be one-to-many, in case we have the exact
same package in multiple distros, so structure the hash accordingly.)
3. For each *.deb/*.udeb/*.diff.gz/etc. file, get the MD5 sum, and check
to see if there is a corresponding long filename(s).
4. If yes, then rename/copy/etc. the package file to the long
filename(s). (If more than one, use hardlinks to conserve inodes.)
5. If no, complain that the package can't be imported.
Does this sound reasonable? #5 would be a new behavior for the import
script, but these would be files that apt-cacher-cleanup.pl would blow
away anyway.
P.S.: The patch includes a few "bonus" fixes that I stumbled upon. Most
of them are self-explanatory; the change to read_config() allows
the user= and group= directives in the config file to be commented
out (as the comments in that file imply that this is supported).
diff -rbu apt-cacher-1.6.8.orig/apt-cacher-cleanup.pl apt-cacher-1.6.8/apt-cacher-cleanup.pl
--- apt-cacher-1.6.8.orig/apt-cacher-cleanup.pl 2009-02-22 15:18:52.000000000 -0500
+++ apt-cacher-1.6.8/apt-cacher-cleanup.pl 2009-08-18 16:51:19.000000000 -0400
@@ -700,7 +700,7 @@
chdir "$cfg->{cache_dir}/headers" && -w "." || die "Could not enter the cache dir";
# headers for previously expired files
-for(<*.deb>, <*.bz2>, <*.gz>, <*.dsc>) {
+for(<*.deb>, <*.udeb>, <*.bz2>, <*.gz>, <*.dsc>) {
if(! defined($valid{$_})) {
unlink $_, "../private/$_.complete" unless $sim_mode;
printmsg "Removing expired headers: $_ and company...\n";
@@ -709,7 +709,7 @@
# also remove void .complete files, created by broken versions of apt-cacher in rare conditions
chdir "$cfg->{cache_dir}/private" && -w "." || die "Could not enter the cache dir";
-for(<*.deb.complete>, <*.bz2.complete>, <*.gz.complete>, <*.dsc.complete>) {
+for(<*.deb.complete>, <*.udeb.complete>, <*.bz2.complete>, <*.gz.complete>, <*.dsc.complete>) {
s/.complete$//;
if(! (defined($valid{$_}) && -e "../packages/$_" && -e "../headers/$_") ) {
printmsg "Removing: $_.complete\n";
diff -rbu apt-cacher-1.6.8.orig/apt-cacher-lib.pl apt-cacher-1.6.8/apt-cacher-lib.pl
--- apt-cacher-1.6.8.orig/apt-cacher-lib.pl 2009-02-22 15:18:52.000000000 -0500
+++ apt-cacher-1.6.8/apt-cacher-lib.pl 2009-08-19 13:49:01.000000000 -0400
@@ -21,6 +21,8 @@
cache_dir => '/var/cache/apt-cacher',
logdir => '/var/log/apt-cacher',
admin_email => 'r...@localhost',
+ user => $>,
+ group => (sub { my $g = $); $g =~ s/\s.*$//; $g; })->(),
generate_reports => 0,
expire_hours => 0,
http_proxy => '',
@@ -102,7 +104,7 @@
# Die if we have not been configured correctly
die "$0: No cache_dir directory!\n" if (!-d $cfg->{cache_dir});
- my $uid = $cfg->{user}=~/^\d+$/ ? $cfg->{user} : getpwnam($cfg->{group});
+ my $uid = $cfg->{user}=~/^\d+$/ ? $cfg->{user} : getpwnam($cfg->{user});
my $gid = $cfg->{group}=~/^\d+$/ ? $cfg->{group} : getgrnam($cfg->{group});
if (!defined ($uid || $gid)) {
@@ -206,7 +208,7 @@
# Stores data flattened for use in tied hashes
sub extract_sums {
my ($name, $hashref) = @_;
- my ($cat, $listpipe, $indexbase);
+ my ($cat, $listpipe, $repobase, $indexbase);
# If arg is filehandle, fileno is undef
if (fileno $name) {
@@ -216,6 +218,11 @@
else {
$cat = ($name=~/bz2$/ ? "bzcat" : ($name=~/gz$/ ? "zcat" : "cat"));
+ if($name=~/^(.*?_)dists_/) {
+ $repobase=$1;
+ $repobase=~s!.*/!!g;
+ }
+
if($name=~/^(.*_)(?:Index|Release)$/) {
$indexbase=$1;
$indexbase=~s!.*/!!g;
@@ -223,7 +230,7 @@
open($listpipe, "-|", $cat, $name);
}
- my ($file,$complete,$sha1,$sha256,$md5,$size);
+ my ($dir,$file,$complete,$sha1,$sha256,$md5,$size);
while(<$listpipe>) {
if(/^\s(\w{40})\s+(\d+)\s(\S+)\n/ && $indexbase) { # diff_Index/Release SHA1
$sha1=$1;
@@ -248,7 +255,8 @@
elsif(/^\s(\w{32})\s+(\d+)\s(\S+)\n/) { # Sources
$md5=$1;
$size=$2;
- $file=$3;
+ $file="$repobase${dir}::$3";
+ $file=~s!/!_!g;
$complete=1;
}
elsif(/^MD5sum:\s+(.*)$/) { # Packages
@@ -264,8 +272,12 @@
$size=$1;
}
elsif(/^Filename:\s+(.*)$/) {
- $file=$1;
- $file=~s/.*\///;
+ $file=$repobase.$1;
+ $file=~s!/([^/]+)$!::$1!;
+ $file=~s!/!_!g;
+ }
+ elsif(/^Directory:\s+(.*)$/) {
+ $dir=$1;
}
elsif(/^Description:.*$/) {
$complete=1;
diff -rbu apt-cacher-1.6.8.orig/apt-cacher2 apt-cacher-1.6.8/apt-cacher2
--- apt-cacher-1.6.8.orig/apt-cacher2 2009-02-22 15:30:20.000000000 -0500
+++ apt-cacher-1.6.8/apt-cacher2 2009-08-19 13:04:58.000000000 -0400
@@ -529,9 +529,13 @@
if (&is_package_file($filename)){
# We must be fetching a .deb or a .rpm or some other recognised
- # file, so let's cache it.
- # Place the file in the cache with just its basename
- $new_filename = $filename;
+ # file, so let's cache it. Make a unique filename so that we
+ # can cache packages from multiple distributions (e.g. Debian,
+ # Ubuntu) without name collisions, but if we only have a
+ # package with a non-unique name in the cache, then use that.
+ $new_filename = "$host$uri";
+ $new_filename =~ s,/([^/]+)$,::$1,; # makes demunging easier
+ $new_filename =~ s,/,_,g;
debug_message("new base file: $new_filename");
}
elsif ($filename =~ /2\d\d\d-\d\d-\d\d.*\.gz$/) {
