Processed: Re: Bug#562398: anna: fails if multiple versions of a udeb in Packages file
Processing commands for cont...@bugs.debian.org: severity 562398 important Bug #562398 [libdebian-installer4-udeb] libdebian-installer: strange behavior with more than one version of a package in a Packages file Severity set to 'important' from 'serious' Thanks Stopping processing here. Please contact me if you need assistance. -- 562398: http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=562398 Debian Bug Tracking System Contact ow...@bugs.debian.org with problems -- To UNSUBSCRIBE, email to debian-bugs-rc-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Bug#562398: anna: fails if multiple versions of a udeb in Packages file
reassign 562398 libdebian-installer4-udeb
found 562398 0.69
retitle 562398 libdebian-installer: strange behavior with more than one version
of a package in a Packages file
affects 562398 + anna
tags 562398 + patch
thanks
On Sun, May 02, 2010 at 03:08:08PM -0600, John Wright wrote:
Specifically, what happens is that anna unpacks all the packages in one
batch, and then it configures all of them. But while unpacking another
version of a package while another vesrion is in an
unpacked-but-not-configured state is ok, it's not ok to configure a
package that's already in the configured state. So if a package is in
the list twice, it fails at the second configure for that package.
Here's what's happing in libd-i: Upon encountering a Packages stanza
with the same Package field as one it's previously seen,
di_packages_parser_read_name sets the data pointer the rest of the
parsing functions will use to the previously-used di_packages pointer.
At first glance, this would appear simply to prefer packages that appear
later in the Packages file, irrespective of version. Unfortunately, it
appends the di_package to the package list
(parser-data-packages-list) whether it's a newly allocated one or an
old one. So while only one actual di_package exists for a particular
package name, it might appear multiple times in the list.
The simple way to fix the anna issue is to make sure we only append
newly allocated di_package objects to the list. But it would be nice to
favor new versions rather than whatever shows up latest in the Packages
file (for example, to facilitate #389430 or LP#234486).
I've attached a quick reproducer to demonstrate the issue, and a patch.
I would prefer if the version comparison could happen during the
Packages file parsing, rather than after the fact (since this way
requires creating a temporary hash table and traversing the list a
couple of extra times), but that change would seem to be very invasive.
In any case, after pounding my head for a couple of hours, I couldn't
figure out how to do it any better with the current parsing
infrastructure. :)
--
John Wright j...@debian.org
#include stdio.h
#include debian-installer/system/packages.h
#include debian-installer/slist.h
int main(int argc, char **argv)
{
di_packages_allocator *allocator = di_system_packages_allocator_alloc();
di_packages *packages;
di_package *package, *last_package = NULL;
di_slist_node *node;
const char *packages_file;
if (argc == 2)
packages_file = argv[1];
else {
printf(Usage: %s packages-file\n, argv[0]);
return 1;
}
packages = di_system_packages_read_file(packages_file, allocator);
for (node = packages-list.head; node; node = node-next) {
package = node-data;
if (!package)
printf(package == NULL\n);
if (package == last_package)
printf(Eek! package == last_package\n);
printf(Package: %s\n, package-package);
printf(Version: %s\n, package-version);
printf(\n);
last_package = package;
}
return 0;
}
Package: baz
Version: 0.0~hpde1
Package: bang
Version: 0.133
Package: baz
Version: 0.0hpde1
Package: foo
Version: 1.2.3
Package: foo
Version: 1.2.3hpde1
Package: bar
Version: 2.3.4hpde1
Package: bar
Version: 2.3.4
Package: baz
Version: 0.1
From 64d06247e4fdb35fd7f33eb7020ec84584e23b42 Mon Sep 17 00:00:00 2001
From: John Wright j...@johnwright.org
Date: Sun, 2 May 2010 22:15:11 -0600
Subject: [PATCH] Filter out old versions after parsing a Packages file
The previous behavior favored packages showing up later in the Packages
file rather than packages with later versions, and wound up putting
multiple entries for the same package in the di_packages list. This
patch ensures a new di_package is created for each stanza, and then
filters out all but the latest versions before returning the di_packages
pointer.
There's a small memory leak for each di_slist_node corresponding to an
old package version. It can't be helped, as far as I can tell, because
of how the memory for those are allocated (using mem_chunk). The bulk
of the memory used for those is freed, just not the di_slist_node glue.
This patch also fixes a bug in di_hash_table_insert, where the key was
not being reset after potentially destroying the old key and changing
the value for a key (which may contain the key itself).
---
include/debian-installer/packages.h |4 ++
src/hash.c |1 +
src/packages.c | 69 +++
src/packages_parser.c |8 +++-
4 files changed, 80 insertions(+), 2 deletions(-)
diff --git a/include/debian-installer/packages.h b/include/debian-installer/packages.h
index c5e4576..0d7dd1c 100644
--- a/include/debian-installer/packages.h
+++ b/include/debian-installer/packages.h
@@ -71,6 +71,10 @@ void di_packages_append_package (di_packages *packages, di_package *package, di_
Processed: Re: Bug#562398: anna: fails if multiple versions of a udeb in Packages file
Processing commands for cont...@bugs.debian.org: reassign 562398 libdebian-installer4-udeb Bug #562398 [anna] anna: fails if multiple versions of a udeb in Packages file Bug reassigned from package 'anna' to 'libdebian-installer4-udeb'. found 562398 0.69 Bug #562398 [libdebian-installer4-udeb] anna: fails if multiple versions of a udeb in Packages file Bug Marked as found in versions libdebian-installer/0.69. retitle 562398 libdebian-installer: strange behavior with more than one version of a package in a Packages file Bug #562398 [libdebian-installer4-udeb] anna: fails if multiple versions of a udeb in Packages file Changed Bug title to 'libdebian-installer: strange behavior with more than one version of a package in a Packages file' from 'anna: fails if multiple versions of a udeb in Packages file' affects 562398 + anna Bug #562398 [libdebian-installer4-udeb] libdebian-installer: strange behavior with more than one version of a package in a Packages file Added indication that 562398 affects anna tags 562398 + patch Bug #562398 [libdebian-installer4-udeb] libdebian-installer: strange behavior with more than one version of a package in a Packages file Added tag(s) patch. thanks Stopping processing here. Please contact me if you need assistance. Debian bug tracking system administrator (administrator, Debian Bugs database) -- To UNSUBSCRIBE, email to debian-bugs-rc-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Bug#562398: anna: fails if multiple versions of a udeb in Packages file
On Thu, Dec 24, 2009 at 01:30:04PM +0100, Frans Pop wrote: On Thursday 24 December 2009, Frans Pop wrote: So the error is probably the result of anna trying to install the same package twice. The exit code of anna was 8, which matches an error during unpack or configure. I've run into this with a custom installer suite with udebs overriding those in lenny's main/debian-installer. My solution at the time was just to filter the Packages file in net-retriever so that anna was only given one version of any particular package. (Ubuntu did something similar, see [1].) However, it should probably be fixed in anna or libdebian-installer itself. :) Specifically, what happens is that anna unpacks all the packages in one batch, and then it configures all of them. But while unpacking another version of a package while another vesrion is in an unpacked-but-not-configured state is ok, it's not ok to configure a package that's already in the configured state. So if a package is in the list twice, it fails at the second configure for that package. I'm trying to better understand anna and libd-i to come up with a suitable patch... [1]: https://bugs.launchpad.net/ubuntu/+source/net-retriever/+bug/234486 -- John Wright j...@debian.org -- To UNSUBSCRIBE, email to debian-bugs-rc-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Bug#562398: anna: fails if multiple versions of a udeb in Packages file
Package: anna Severity: serious Yesterday I uploaded some packages, including base-installer and hw-detect. Today during a test install from a local build this resulted in: Failed to load installer component loading base-installer failed for unknown reasons. Aborting. The syslog does not show much info, but this is interesting: [...] anna: DEBUG: retrieving base-installer 1.104 anna: DEBUG: retrieving base-installer 1.104 anna: DEBUG: retrieving bootstrap-base 1.104 [...] anna: DEBUG: retrieving disk-detect 1.74 anna: DEBUG: retrieving disk-detect 1.74 [...] So the error is probably the result of anna trying to install the same package twice. I ran anna a second time and that finished without errors and the rest of the installation also completed without problems, so the impact is limited. The packages file on my local mirror shows the cause of the problem. It contains two versions of both base-installer (1.103, 1.104) and disk-detect (1.73, 1.74). Both are arch:all packages, so this is probably the simple result of the recent changes in the archive and the packages not yet being built for all arches. We need to at least make sure anna does not try to process the same package twice. It already seems to return the correct (highest) version. The actual fix is probably in libd-i rather than anna itself. This issue will only manifest for packages that have both arch:any and arch:all udebs, and only while it has not been built for all arches. So the problem will be fairly rare [1], but IMO it should still be fixed as it looks extremely fatal. -- To UNSUBSCRIBE, email to debian-bugs-rc-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Bug#562398: anna: fails if multiple versions of a udeb in Packages file
On Thursday 24 December 2009, Frans Pop wrote: So the error is probably the result of anna trying to install the same package twice. The exit code of anna was 8, which matches an error during unpack or configure. -- To UNSUBSCRIBE, email to debian-bugs-rc-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
