On Thu, 16 Jun 2011 11:22:21 +0200 David Kuehling <[email protected]> wrote:
> >>>>> "Neil" == Neil Williams <[email protected]> writes: > > Thought I'd just point out where my paranoia about proper source version > matching comes from: > > * we're using architecture powerpcspe from debian-ports. debian-ports > doesn't carry sources, and is out of sync with normal debian mirrors, > which makes it pretty difficult to satisfy the GPL. pool-powerpcspe/main/e/eglibc/eglibc-source_2.11.1-2+powerpcspe1_all.deb That's from: http://ftp.debian-ports.org/debian/dists/unreleased/main/binary-powerpcspe/Packages http://ftp.debian-ports.org/debian/pool-powerpcspe/main/e/eglibc/ The problem is this: http://ftp.debian-ports.org/debian/dists/unreleased/main/source/ Sources is a zero length file. There is nothing apt or multistrap can do to help you here. The problem is that the source DOES exist, even for unreleased, but in a non-standard pool location which is NOT listed in the Sources file. This could be a bug in debian-ports. Hector will try and have a look at that. > * we're using our own, custom mirroring tool [1] to overcome the > limitations of debian-ports. this tool is named 'debparanoia' for a > reason, as it double checks that matching source packages are present > for all .debs. Use reprepro which is what Emdebian uses to twist the debian-ports stuff into some kind of standard shape, albeit that the problem of unreleased sources cannot be fixed that way. reprepro updates are atomic. > * Since debparanoia is used for mirroring as well as for > "license-checking" our images, I dont't really care if multistrap > does the source package version check. I just thought it would be > good anyways to eliminate the slightest chance of 'apt-get source' > not satisfying the GPL by getting wrong package versions, without the > user noticing. If the Sources file was not zero bytes, that could work. > >> * Concurrently I run 'multistrap' which runs 'apt-get update', > >> fetching dists/sid/main/binary-amd64/Packages.bz2 and > >> sid/main/source/Sources.bz2 . > >> > >> * Now I get Packages.bz2 from before the mirror update, and > >> Sources.bz2 from after the mirror update. > > > No you don't. You get Packages.bz2 and Sources.bz2 in sync at the same > > time in the same apt-get update call. Indeed in most cases, as apt is > > using parallel connections, Packages and Sources are downloaded at the > > same time over multiple sockets. What happens afterwards is that > > apt-get source uses that cached data to get the sources. > > Well this is the part that only works if you cross your fingers. > Nothing guarantees that 'apt-get update' schedules the packages.bz2 and > sources.bz2 for synchronous download. In fact typing in 'apt-get > update' on my pc, it first downloads 4 Sources files, then 4 Packages > files for me. Check the http-method carefully, it opens sockets for each of these files BEFORE indicating that the download has started and will commonly show parallel downloads if your connection is slow enough or your system far enough out of date from the last update. > This race can be detected by checking Packages.bz2 and Sources.bz2 with > the checksums present in the Release. Not sure whether that's > implemented. For me 'multistrap' happily uses repositories without > checksums in the Release file. Bad mirror. > I do not contest that debian archives carry source packages for all > binary packages in the pool. I only contest that the .deb packages > referenced by apt's cache do not neccessarily match the source packages > referenced the cache. This condition would result in license violation > for people who rely on 'apt-get source' to satisfy the GPL. That doesn't follow. > A version mismatch will occur exacty when a mirror update ocurred in > between the download of sources.bz2 and packages.bz2. You want to tell > me that this is not possible, however your description of the process > makes it look like it is possible, though unlikely. How? apt opens the sockets and then starts the download. If the file changes, the download will abort. Both sockets are open before the download starts. Therefore, if the files download successfully, the files must be in the same state as when the sockets were originally opened. Are you trying to say that files change in the microsecond between the creation of one socket and the creation of the next socket on a multi-core server?? This isn't a race in apt, it's an inherent problem of using unofficial ports with unreleased patches and a broken Source listing. If you can prove such a race, report it as a bug in apt. It has nothing to do with multistrap. > The only way to prevent such a race would be to either (a) prevent > mirror updates (i.e. 'lock' the archive) during 'apt-get update' > sessions, or (b) to guarantee that Sources.bz2 and Packages.bz2 download > starts at exactly the same point in time. Which, apart from the time which elapses between the opening of one socket and the opening of the next is already implemented. > I think neither (a) nor (b) can be implemented. You can only implement > (c): ensure consistency with checksums in the release file, and retry > download ad infinitum, until checksums match. False. > Note that I wasn't talking about packages in the pool, but only about > the indices that were "snapshotted" by 'apt-get update'. ... which are downloaded using parallel sockets from the mirror which uses an atomic process to create them... > >> Of course I guess the error rate will not be too high, at least not > >> over a normal high-rate internet connection. > > > It's nothing to do with the download speeds. > > Well, try 'apt-get update' over a modem line and notice how much time > passes between fetching sources.bz2 and fetching the packages.bz2. > Pretty long time for a mirror update occuring in between?! You are being misled by the output messages of apt which only start listing stuff when *data is sent over the open socket*, not when the socket itself is opened. I used apt over 28k modems for a long, long time. -- Neil Williams ============= http://www.linux.codehelp.co.uk/
pgpYVSvp2I3xL.pgp
Description: PGP signature
