Re: Bug#93612: Support for new archive structure
On Sat, Apr 14, 2001 at 01:08:11PM +0200, Santiago Vila wrote: I use this too, and I even have a burner. A loopback ISO mount is always faster than the real CD. If this is not going to be possible anymore, we lose *valuable* functionality. Hope I'm not too late here, but I use this functionality also. Sometimes even over NFS (yucky, I know...). -- Nate Duehr [EMAIL PROTECTED] GPG Key fingerprint = DCAF 2B9D CC9B 96FA 7A6D AAF4 2D61 77C5 7ECE C1D2 Public Key available upon request, or at wwwkeys.pgp.net and others. -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]
Re: Bug#93612: Support for new archive structure
Hello, - works with dpkg-multicd and apt-cdrom It's difficult to check automatically for those. But we'd need such a tool for sure. I think checking this could be done automatically and the integration of this step into debian-cd would be a very good thing... -- Attila Nagye-mail: [EMAIL PROTECTED] Budapest Polytechnic (BMF.HU) @work: +361 210 1415 (194) H-1084 Budapest, Tavaszmezo u. 15-17. cell.: +3630 306 6758 -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]
Re: Bug#93612: Support for new archive structure
On Sat, 14 Apr 2001, Jason Gunthorpe wrote: On Sat, 14 Apr 2001, Philip Charles wrote: A CD (or iso image) is essentially one file and the integrity of this can be verified by a single signed checksum. No, that is such an oversimplification and what you have described of the HURD CD's prooves that. CD's may in fact contain content from the Debian site and it should be possible to validate that content was part of a Debian release without having to jump through any special hopps. That is independent of anything else on the CDs. It is not a simplification. It is drawing boundaries. Is Debian responsible for Libranet, Corel, Stormix, and Progeny? I may be the only person in NZ that is offering vendor versions of Debian to the public, but there are others here doing this as consultants and in an in-house role. I think that you would be surprised just how much of this is happening world wide. Is Debian responsible for these? I suggest that Debian's responsibility for the integrity of CDs stops with the Official CD images. I take great care to ensure the integrity Debian packages included on my custom CDs, but in the last analysis the responsibility is mine. I would welcome any additional tools to check what I do. CDs are best viewed as an entity of their own and not an extension of the Official archive. Those responsible for the the creation of the Official discs keep close to the Official archive structure as possible. However, debian-cd and boot-floppies are very flexible and coupled with the present installation tools great and wondrous installation CDs can be built and used in peculiar ways. The ability to build these specialised CDs is one of the great strengths of Debian. People need to be involved in the Debian CD field to understand what can be done. Some of the less standard ways of using CDs have been mentioned in the discussion so far, and there are many more. Do not mess with this flexibility. If you do then you are in danger of destroying one of Debian's great strengths. only extends to Official CDs. The parallel here is some of the rather awful in-house Debian archives, Debian cannot take responsibility for these. A verification process may be available, but people may choose not to use it. Debian may not be responsible, but it does support them. There is no reason they should loose out on verification too. Look at apt-cdrom someday, it jumps through an unbelievable number of hoops specifically to support non-official CD's that may have not been made according to our standards. So don't do anything to limit any of its present functions. While the integrity issue for CD images can be solved by signing them, I recognise that the problem for mirrors is more complex and we have a solution to hand, great. What I would like to see is some kind of mechanism by which I could check my Debian archive using the Release file. Have a good Easter. Phil. - Philip Charles; 39a Paterson St., Dunedin, New Zealand; +64 3 4882818 Mobile 025 267 9420. I sell GNU/Linux CDs. See http://www.copyleft.co.nz [EMAIL PROTECTED] - preferred. [EMAIL PROTECTED] -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]
Re: Bug#93612: Support for new archive structure
On Sun, 15 Apr 2001, Attila Nagy wrote: Hello, I have used this myself. It is a good way to test an image. It is also used by people who have borrowed CDs and do not have a burner. I think a validity checking part in debian-cd would be very nice. BTW, what's the correct way to check an ISO made with the debian-cd script? And without any debian specific tools like apt? :) If it is going to be an Official image, then rsync it against an Official image and check the md5sum. If it is unoffical, loop mount it and compare the image's /md5sums.txt with the archives ./indices/md5sums.gz Phil. - Philip Charles; 39a Paterson St., Dunedin, New Zealand; +64 3 4882818 Mobile 025 267 9420. I sell GNU/Linux CDs. See http://www.copyleft.co.nz [EMAIL PROTECTED] - preferred. [EMAIL PROTECTED] -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]
Re: Bug#93612: Support for new archive structure
Le Sat, Apr 14, 2001 at 08:15:49PM -0600, Jason Gunthorpe écrivait: It will add them both and it becomes trivial for someone to defeat the security mechanisms. Why ? What do you mean 'Why?' Put the bad files in the insecure space and let-er-rip. I can't understand that. apt know what has been signed and what packages can be trusted. I guess that apt would warn the user if he had to use a file which is only part of the insecured tree and on which there's no valid trust path. The insecure one must be ignored for things to work correctly. That sounds ok to me. I don't see a problem here. *snort* Sure, I'll just wave a magic wand and it will all go away. Anyway you'll have to do something like that. When you use for example a Ximian tree and a Debian tree, some packages overlap, the Debian tree will be signed and the Ximian may not be (or with a key not accepted by the admin) and you will have to manage that. I don't see how it is harder for the CD. As I said, I can think of no good way to eliminate the redundent tree, continue to support current CDs and provide protection against ways to defeat the validation mechanism. Maybe some other apt hackers may think about it ? Because I can't believe that it is not doable. *you* are assuming that just because it is easy for the CD scripts to do that it should be done and leaving the problem of supporting it up to *me* No. I'm not willing to create problem for the sake of it, I just want to do something that satisfies my view of what a debian cd should be. 1) The distinction between 'the user who wants secured' and not is rediculous and should be dropped Sure, but why should it be up to me make that disappear ? You can make this disappear too by managing correctly the CD. 2) There are no other tools. Switching to the pool layout pretty much foobared every other option besides APT, and dpkg-multicd already needed funny files. Do you really think that all the tools were written by dumb people not able to read Packages files and use the Filename field ? A single CD can be used by any standard "file" access method. Be it one from dselect or one from apt. Bullocks. Their should be a consistent set of functionality and nobody should ever be forced to pick one over the other. Agreed. For the FTP/HTTP method, everything is ok. For mirror accessed by files URI, it's ok. For CD accessed by apt-cdrom, it may be ok if apt is able to give precedence to trusted packages over non trusted packages. For CD accessed by file URI, it's not ok actually but can be if the user who really need security are aware of the woody-secured tree and if he uses modern apt. What a big deal. We loose if it becomes hard to enable and use the validation mechanisms because people won't use them, and we are back to where we started. It's not hard for 99,9% of our user. Like you said, the 0,01% person who uses CD via loopback or anything like that either don't care or know about woody-secured. A single CD tests nothing, you have to do a set and do testing to make sure that the swapping mechanism won't freak out. Yup, I'm sure someone can afford that. I may even when I get back to college. Non-apt stuff was broke the instant we decided to put pool/ on a CD, you Why ? Old tools overcomed the non-US move without much troubles and they'll overcome the pool move again. updated in 1997! We better not do something or it might break it!' You're exagerating too. I never said that. I'm willing to make changes to offer that, but I'm not willing to break things for that when there's a possibility of not breaking it. Cheers, -- Raphaël Hertzog -+- http://strasbourg.linuxfr.org/~raphael/ Le bouche à oreille du Net : http://www.beetell.com Naviguez sans se fatiguer à chercher : http://www.deenoo.com Formation Linux et logiciel libre : http://www.logidee.com -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]
Re: Bug#93612: Support for new archive structure
Le Sun, Apr 15, 2001 at 12:18:26AM -0600, Jason Gunthorpe écrivait: 1) Make the empty file dists/woody/aptignr - This tells apt-cdrom that the CD is foobar'd below this directory and it should just ignore it. Does this feature already exist ? If not, please consider calling it ".aptignr" instead. 2) In dists/woody-secured/ put Release, Release.gpg Already ok. and files.list (.gz ?) I have troubles understanding why you need that file. 3) In dists/woody-secured/.. put the normal package files. Ok. [If you forget step #1 new apt-cdrom will likely refuse to use the disc without some serious prodding..] Ok. The format of files.list should be 1 line per file listing the entire contents of the CD relative to its 'root' (the same place Filename: fields are relative to) this should include all .debs, source items, boot floppies, etc. Everything referenced directly, or indirectly from the Release file. Just file names. Do you need that file for performance issue ? Because you could figure out yourself what's available and what's not. BTW, all the files on the CD are aleady listed in /md5sum.txt. Maybe you could use that file, or did you create that file so that it can also be used for other methods than the cdrom one ? BTW md5sum.txt will also list documentation files and so on that may not be referenced from the Release file. APT in 'partial' mode will fetch and use the files.list file to figure out what to ignore, etc. Compress if you want to be able to mount CDs and use them over HTTP. What is that partial mode, is that only for partial mirror like single CD with secured setup ? Why isn't it the standard mode ? ie will there be an option to apt-get to switch from one mode to the other ? I wouldn't like that. If apt-get detects it automatically by checking the presence of such a file then it's ok. A further refinement would be to use the files.list file to fingerprint the CD and pre-load the file listings from all CD's in the related set. That's not so nice. One of the big feature from apt-cdrom is that it works well with incomplete CD set and even several CD set from different sources. Relying on this would break that. Cheers, -- Raphaël Hertzog -+- http://strasbourg.linuxfr.org/~raphael/ Le bouche à oreille du Net : http://www.beetell.com Naviguez sans se fatiguer à chercher : http://www.deenoo.com Formation Linux et logiciel libre : http://www.logidee.com -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]
Re: Bug#93612: Support for new archive structure
On Fri, 13 Apr 2001, Philip Charles wrote: On Fri, 13 Apr 2001, Jason Gunthorpe wrote: No more than all the other ugly things that have been suggested, and this only affects the 3 people silly enough to loopback mount ISO's and try to use APT on them. I have used this myself. It is a good way to test an image. It is also used by people who have borrowed CDs and do not have a burner. I use this too, and I even have a burner. A loopback ISO mount is always faster than the real CD. If this is not going to be possible anymore, we lose *valuable* functionality. -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]
Re: Bug#93612: Support for new archive structure
On Sat, 14 Apr 2001, Santiago Vila wrote: On Fri, 13 Apr 2001, Philip Charles wrote: On Fri, 13 Apr 2001, Jason Gunthorpe wrote: No more than all the other ugly things that have been suggested, and this only affects the 3 people silly enough to loopback mount ISO's and try to use APT on them. I have used this myself. It is a good way to test an image. It is also used by people who have borrowed CDs and do not have a burner. I use this too, and I even have a burner. A loopback ISO mount is always faster than the real CD. If this is not going to be possible anymore, we lose *valuable* functionality. I also (relatively) often use it. Cheers, Cristian -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]
Re: Bug#93612: Support for new archive structure
On Fri, 13 Apr 2001, Jason Gunthorpe wrote: On Sat, 14 Apr 2001, J.A. Bezemer wrote: b) Use verbatim package files and call them 'Packages.something' - Everyone can make CD set, and we still have end-to-end security - apt file:/../ does not work properly on those discs e) The Packages of the FTP archive is copied verbatim to CD as Packages.complete That's b) - apt file:/../ works properly on those discs Nope, packages fails verification and APT will stop without using the file, ditto for ftp, http, etc. EVERY access method up to current potato APT will work nicely with this. If woody/sid APT suddenly stops working correctly, thats a grave bug in APT and nothing else. That's because APT (and ONLY apt) is effectively changing the very definition of "Packages file". Up to this moment: Packages = file that is an index to all packages (virtually) available inunder current directory But you want to change that into: Packages = file that contains security information on packages that either are, or are not, available inunder current directory You'll have to fiddle around and switch it off via some-means-not-yet-determined. You haven't even thought of that aspect, have you? I'm starting to think that you didn't think of _any_ aspect carefully enough. This is Debian, not Red Hat or M$. We are known for supplying our users with the options they need or want, not for taking options away because of uncareful management decisions. But, of course, that's why we're discussing things now: to create a solution that doesn't take away any functionality for any one of our users. And I think that's very well possible. Doing what you described, but swapping the names around is best. Then the name in the release file matches the name in the filesystem and you can get authentication with very little hassle. Is there really any more "hassle" when names (that, as I stated, essentially do not matter at all) are kept the way I suggested? Use an APT line like: deb-partial ftp:// ... For instance. Good, now you're thinking constructively. But instead of deb and deb-partial call them deb-complete and deb Then you can just forget about the deb-complete because the "new" deb can handle anything. I *really* don't like that we suddenly have to start special casing all the tools that work with the Release file to work on CD's, thats really lame. (see below) A "deb-partial" is _more_ a special case than a "deb" that handles authentication via either a one-step or two-step process without the user noticing it. Besides, the scheme I proposed also seamlessly allows for partial http and ftp mirrors that 1) can be authenticated if the access method has authentication support and 2) work with APT and all other access methods you can think about. Simply because it doesn't change the definition of "Packages file". Completely backward and forward compatible. Generalizing: I think the autentication sequence should be like this for _any_ medium (cdrom,file,ftp,http,whatever): Release/Release.gpg | Packages.complete | Packages | .deb file with the special case that Packages.complete may be omitted if it is exactly the same as Packages. I know this is a "radically" different way to view things, but if you had looked at it this way from the very beginning, there wouldn't have been any problem, would there? And it isn't too late to change things, because this solution is even back- and forward compatible with the current authentication structure. (Except for the filenames, that as I said, don't matter. I even wonder why the Packages.gz-s are mentioned in the Release file, because it's the _contents_ that matter and not the unzipped/gzip/bzip2/Xzip -1...-9 format it's distributed in.) And lastly, anyone with hostile intentions can easily make/ship a CD which contains a modified apt that doesn't check signatures at all. End-to-end There is a fairly direct means to validate the CD against the web-of-trust Yes of course there is. What I said was that a piece (or collection) of software fundamentally CANNOT authenticate itself. It must be authenticated by the user doing every algorithm manually, or using trusted programs that do NOT come from the software(collection) that is to be checked. This essentially means that APT checking authentication provides a _false_ sence of security. Of course, it's another barrier that an impostor has to take, and that's a good thing, but to anyone _really_ caring about security it's of no use at all. Regards, Anne Bezemer -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]
Re: Bug#93612: Support for new archive structure
I think that we are trying to turn apples into oranges. The security of CDs is relatively simple, that of a mirror more complex and they need to be approached differently. A mirror can have one corrupt or sabotaged file amongst 2-3 and there needs to be a way of detecting this. The proposed scheme tackles this problem and I am grateful. I have come across mirrors that I do not trust. A CD (or iso image) is essentially one file and the integrity of this can be verified by a single signed checksum. However, there has to be a process which ensures a secure transfer from a reliable archive to the CD image. The process needs to be such that the end user can understand how this was done and so have confidence in the images checked by this process. IMHO, we should be spending our energies on developing this transfer process. Debian's responsibility has to end somewhere. It cannot take responsibility for my Hurd CDs for example, and I would suggest that it only extends to Official CDs. The parallel here is some of the rather awful in-house Debian archives, Debian cannot take responsibility for these. A verification process may be available, but people may choose not to use it. Phil. - Philip Charles; 39a Paterson St., Dunedin, New Zealand; +64 3 4882818 Mobile 025 267 9420. I sell GNU/Linux CDs. See http://www.copyleft.co.nz [EMAIL PROTECTED] - preferred. [EMAIL PROTECTED] -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]
Re: Bug#93612: Support for new archive structure
Le Sun, Apr 15, 2001 at 12:27:19AM +1000, Anthony Towns écrivait: Instead of changing the name of the Packages file, how about we try something completely different? dists/ woody-secured/ [...] That should be pretty easy to do right now, and shouldn't cause any problems to anyone, right? All you're doing is creating a new directory that no one has to look at, and adding a Release file in dists/woody/ that'll make debootstrap work. That's the best solution at the moment. I'm going to implement it. So anyway, how does the above idea (two woody trees in dists), sound? Workable, or are there other problems? I don't see any apart from the fact that most tools will use stable symlink which still points to woody. That means that apt-cdrom must learn about woody-secured (give it precedence over woody) when looking in the CDs. I can create symlinks "stable-secured" and so on if required. Cheers, -- Raphaël Hertzog -+- http://strasbourg.linuxfr.org/~raphael/ Le bouche à oreille du Net : http://www.beetell.com Naviguez sans se fatiguer à chercher : http://www.deenoo.com Formation Linux et logiciel libre : http://www.logidee.com -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]
Re: Bug#93612: Support for new archive structure
On Sat, Apr 14, 2001 at 06:05:04PM +0200, Raphael Hertzog wrote: Le Sun, Apr 15, 2001 at 12:27:19AM +1000, Anthony Towns ?crivait: Instead of changing the name of the Packages file, how about we try something completely different? dists/ woody-secured/ [...] That should be pretty easy to do right now, and shouldn't cause any problems to anyone, right? All you're doing is creating a new directory that no one has to look at, and adding a Release file in dists/woody/ that'll make debootstrap work. That's the best solution at the moment. I'm going to implement it. Agreed, gets my vote. Simple solution that doesn't break old/existing tools - well done AJ! -- Steve McIntyre, Cambridge, UK. [EMAIL PROTECTED] "It's actually quite entertaining to watch ag129 prop his foot up on the desk so he can get a better aim." [ seen in ucam.chat ] Finger for PGP key -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]
Re: Bug#93612: Support for new archive structure
On Fri, Apr 13, 2001 at 11:14:30PM +, Philip Charles wrote: On Fri, 13 Apr 2001, Jason Gunthorpe wrote: you can tell it to read Packages.cd directly I don't want that, it's a hack. :) No more than all the other ugly things that have been suggested, and this only affects the 3 people silly enough to loopback mount ISO's and try to use APT on them. I have used this myself. It is a good way to test an image. It is also used by people who have borrowed CDs and do not have a burner. I can add another `weirdo' method of using cds... I have a cdrom-less workstation that can nfs mount a server cdrom drive. I tried to apt-cdrom add the 3cd potato set with the nfs mount, worked great, but when I try to install a package apt-get fails to recognize the cd - it keeps asking for it whether the nfs share has be pre-mounted on the workstation or not. Regards, Filip -- Did you know that if you play a Windows 2000 cd backwards, you will hear the voice of Satan? That's nothing! If you play it forward, it'll install Windows 2000. -- Robert Guthrie PGP signature
Re: Bug#93612: Support for new archive structure
On Sat, 14 Apr 2001, J.A. Bezemer wrote: Nope, packages fails verification and APT will stop without using the file, ditto for ftp, http, etc. EVERY access method up to current potato APT will work nicely with this. If woody/sid APT suddenly stops working correctly, thats a grave bug in APT and nothing else. Bullocks. The user will have the option to invoke a difficult mechanism to turn it off for exactly such infrequent cases [For which 99% of the time should not be ignored!] . To actually get reasonable security in automated tools we need to be as paranoid and as strict as possible. The tool should not quitely disable checking and continue in any situation! Next you are going to be insisting that since some tools don't check MD5's APT must have a grave bug if refuses to use files with bad checksums! In this case it is very simple, if you put a new-style release file down beside a set of Packages files *you* (the archive creator) have made a statement that APT 5 should check this new information. If you then proceed to screw up the contents of the Package files then it is entirely your fault that it doesn't work. You'll have to fiddle around and switch it off via some-means-not-yet-determined. You haven't even thought of that aspect, have you? I'm starting to think that you didn't think of _any_ aspect carefully enough. This is Debian, not Red Hat Yes, I haven't decided the name of the option. Clearly that means nothing has been thought through. Geeze, take a chill pill :P I *really* don't like that we suddenly have to start special casing all the tools that work with the Release file to work on CD's, thats really lame. (see below) A "deb-partial" is _more_ a special case than a "deb" that handles authentication via either a one-step or two-step process without the user noticing it. How exactly is adding a new feature to APT that lets it directly read certian well-formed CD's for the 1% of the population that uses such CD's as loop back mounts _more_ of a special, undesirable case than fixing every tool that anyone ever writes to understand that the file names in the Release file are in fact meaningless! That is not an acceptable trade off, please stop maintaing that it is! Generalizing: I think the autentication sequence should be like this for _any_ medium (cdrom,file,ftp,http,whatever): Release/Release.gpg | Packages.complete | Packages | .deb file with the special case that Packages.complete may be omitted if it is exactly the same as Packages. I know this is a "radically" different way to Yes, it will wave it's magic wand and determine that Packages.complete and Packages are in fact, the same. If they aren't then it does some equally magical thing that has all kinds of oppurtunity to introduce flaws from the user's perspective - 'It is listed in Packages but APT wont use it!! Its a bug!!!' Sheeze! Make everything operate worse just so that a handful of people don't have to change the way they are doing things! This essentially means that APT checking authentication provides a _false_ sence of security. Of course, it's another barrier that an impostor has to take, and that's a good thing, but to anyone _really_ caring about security it's of no use at all. Anyone who really cares about security is not operating in a void that expects to have self-authenticating software! They will validate their CD, against the FTP site and/or the web of trust and they will validate the keys they give to APT. Jason -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]
Re: Bug#93612: Support for new archive structure
On Sat, 14 Apr 2001, Philip Charles wrote: A CD (or iso image) is essentially one file and the integrity of this can be verified by a single signed checksum. No, that is such an oversimplification and what you have described of the HURD CD's prooves that. CD's may in fact contain content from the Debian site and it should be possible to validate that content was part of a Debian release without having to jump through any special hopps. That is independent of anything else on the CDs. only extends to Official CDs. The parallel here is some of the rather awful in-house Debian archives, Debian cannot take responsibility for these. A verification process may be available, but people may choose not to use it. Debian may not be responsible, but it does support them. There is no reason they should loose out on verification too. Look at apt-cdrom someday, it jumps through an unbelivable number of hoops specifically to support non-official CD's that may have not been made according to our standards. Jason -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]
Re: Bug#93612: Support for new archive structure
Le Sat, Apr 14, 2001 at 02:05:35PM -0600, Jason Gunthorpe écrivait: How exactly do you propose that apt-cdrom determine that these two random trees of stuff are actually the same tree of stuff? Current apt-cdrom will not properly handle CD's made like this, so you've broke that. Explain us how apt-cdrom works then ... what is it looking for ? And it's what I'd call an exception to the general rule, no ? BTW, you could use the information in the "Release" file to know that they are the same, no ? Frankly, I have no idea how I'd make it able to detect this that doesn't end up being excessively difficult. And what happens if it doesn't detect it ? What's the problem of having both tree discovered by apt-cdrom ? One will be signed by a public key and will therefore be trusted and would logically take precedence over the non-signed tree. Doesn't that sound logical ? ;-) Cheers, -- Raphaël Hertzog -+- http://strasbourg.linuxfr.org/~raphael/ Le bouche à oreille du Net : http://www.beetell.com Naviguez sans se fatiguer à chercher : http://www.deenoo.com Formation Linux et logiciel libre : http://www.logidee.com -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]
Re: Bug#93612: Support for new archive structure
Hello, I have used this myself. It is a good way to test an image. It is also used by people who have borrowed CDs and do not have a burner. I think a validity checking part in debian-cd would be very nice. BTW, what's the correct way to check an ISO made with the debian-cd script? And without any debian specific tools like apt? :) -- Attila Nagye-mail: [EMAIL PROTECTED] Budapest Polytechnic (BMF.HU) @work: +361 210 1415 (194) H-1084 Budapest, Tavaszmezo u. 15-17. cell.: +3630 306 6758 -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]
Re: Bug#93612: Support for new archive structure
On Sat, 14 Apr 2001, Raphael Hertzog wrote: Le Sat, Apr 14, 2001 at 02:05:35PM -0600, Jason Gunthorpe crivait: How exactly do you propose that apt-cdrom determine that these two random trees of stuff are actually the same tree of stuff? Current apt-cdrom will not properly handle CD's made like this, so you've broke that. Explain us how apt-cdrom works then ... what is it looking for ? It looks for all files names Packages or Packages.gz on the CD, uses inodes to infer which are refered to by symlinks, then probes the Packages file and attempts to infer any necessary corrections to the Filename fields and removes any records that don't exist, then it uses various algorithms to determine the most compact 'deb' line(s) for the disc. Having more than one tree means it will be detected more than once and that certianly is not desirable, any may cause problems, like it asking for the disks in a non-ideal order, or something equally lame. And it's what I'd call an exception to the general rule, no ? What? That you want to make discs that don't work with any apt-cdrom that exists? That sounds like a pretty bad thing to do. BTW, you could use the information in the "Release" file to know that they are the same, no ? No. It is entirely reasonable for someone to make a Release file for a non-debian component that has the same characteristics as the Debian one. This would then let the various apt pinning mechanisms work consistantly over the entire CD. Frankly, I have no idea how I'd make it able to detect this that doesn't end up being excessively difficult. And what happens if it doesn't detect it ? What's the problem of having both tree discovered by apt-cdrom ? One will be signed by a public key and will therefore be trusted and would logically take precedence over the non-signed tree. Doesn't that sound logical ? ;-) It will add them both and it becomes trivial for someone to defeat the security mechanisms. The insecure one must be ignored for things to work correctly. I *really* don't see why this is necessary. How is writing: deb file:/.../ woody-secured main any better than writing deb-partial file:/.../ woody main ? Even with this scheme you still need to have the 'deb-partial' feature! Consider with AJ's case that you want to use security, and file/http URI's, you *still* need to have the original release file, the complete package file and the reduced file list and you still need APT support to combine all that information. All this proposal does is break apt-cdrom, in a way that I probably can't fix, and that makes it far worse than everything else that has been proposed! Step back and look at why you are doing this - the only reason is so that people using old APTs can continue to use file: URIs and loopback mounts and not use -m! Those people are so few that it is not unreasonable to expect them to upgrade to a new APT and change their sources.list slightly. Nothing they already have stops working. We've done much, much worse things to our users, this is not a big deal! Jason -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]
Re: Bug#93612: Support for new archive structure
On Sun, 15 Apr 2001, Raphael Hertzog wrote: Having more than one tree means it will be detected more than once and that certianly is not desirable, any may cause problems, like it asking for the disks in a non-ideal order, or something equally lame. May or will cause problem ? I can't predict that. It might work OK in some cases, probably not in all. It is certainly not something I designed or tested for. It will add them both and it becomes trivial for someone to defeat the security mechanisms. Why ? What do you mean 'Why?' Put the bad files in the insecure space and let-er-rip. This scheme lets people make valid 'woody-secured' areas and hide nastly little bombs in the normal 'woody' area. The insecure one must be ignored for things to work correctly. That sounds ok to me. I don't see a problem here. *snort* Sure, I'll just wave a magic wand and it will all go away. As I said, I can think of no good way to eliminate the redundent tree, continue to support current CDs and provide protection against ways to defeat the validation mechanism. *you* are assuming that just because it is easy for the CD scripts to do that it should be done and leaving the problem of supporting it up to *me* You don't see the point ? With what aj proposed, the standard directory uses the standard Packages which works ok with all old tools (including file URI with apt, but also dselect and so on) and the people who requires a secured set of files will use woody-secured. 1) The distinction between 'the user who wants secured' and not is rediculous and should be dropped 2) There are no other tools. Switching to the pool layout pretty much foobared every other option besides APT, and dpkg-multicd already needed funny files. True, however people willing security over a loopbak mount are silly. The main point was not about having security on a file URI but rather about not breaking what already existed. Bullocks. Their should be a consistent set of functionality and nobody should ever be forced to pick one over the other. We loose if it becomes hard to enable and use the validation mechanisms because people won't use them, and we are back to where we started. Anyway I already commited the code to debian-cd CVS, someone please burn a CD with this new tree and check if apt-cdrom does really as bad as Jason says. A single CD tests nothing, you have to do a set and do testing to make sure that the swapping mechanism won't freak out. Step back and look at why you are doing this - the only reason is so that people using old APTs can continue to use file: URIs and loopback mounts It's a bit over exagerated, it's also for people not using apt. :) Non-apt stuff was broke the instant we decided to put pool/ on a CD, you are over-exagerating the importance of maintaning this compatibility. I am *so* sick of this 'Oh! Somebody might be using a tool that was last updated in 1997! We better not do something or it might break it!' attitude. We over came that for the new FTP structure, and here it is again being used to defend a bad solution for the CDs. Wee. Jason -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]
Re: Bug#93612: Support for new archive structure
On Sat, Apr 14, 2001 at 08:15:49PM -0600, Jason Gunthorpe wrote:
On Sun, 15 Apr 2001, Raphael Hertzog wrote:
Having more than one tree means it will be detected more than once and
that certianly is not desirable, any may cause problems, like it asking
for the disks in a non-ideal order, or something equally lame.
May or will cause problem ?
I can't predict that. It might work OK in some cases, probably not in all.
It is certainly not something I designed or tested for.
Easiest way is to do it and find out. We're not going to be releasing
tomorrow, so it doesn't matter much if the first pass is wrong.
It will add them both and it becomes trivial for someone to defeat the
security mechanisms.
Why ?
What do you mean 'Why?' Put the bad files in the insecure space and
let-er-rip.
This scheme lets people make valid 'woody-secured' areas and hide nastly
little bombs in the normal 'woody' area.
Sure. The user has to ensure what they point at is signed appropriately,
and use methods that care about signatures if they want any security.
Having random insecure files, however tempting they may look, shouldn't
stop a user from at least knowing whether they're using something straight
from Debian or not.
True, however people willing security over a loopbak mount are silly.
The main point was not about having security on a file URI but rather
about not breaking what already existed.
That's not true at all: the point is to get end-end security, independent
of the media used to get Debian from us to the end user. Whether it's
by ftp, or CD, or downloaded ISO, or a dozen partial mirrors on a dozen
continents, or whatever.
In addition to this, it's also about not breaking other things, wherever
possible.
I am *so* sick of this 'Oh! Somebody might be using a tool that was last
updated in 1997! We better not do something or it might break it!'
attitude.
"Oh! Somebody might be using apt-cdrom that was last updated at least a few
hours ago! We better not do something or it might break it!" :)
Saying something like:
When upgrading using an old apt-cdrom, if you wish to ensure that
you're getting what you paid for do the following:
* Check debian/dists/woody/Release{,.gpg} match
* Run the following script to ensure that
debian/dists/woody/Release accurately hashes the Packages
files
* Run apt-cdrom over each CD
* Remove the lines referencing woody-cd#X -- these are not
secured.
probably isn't unreasonable, at worst.
(Checking the MD5 of the ISO may not be possible if it's already been burnt
or if it's not quite official)
Cheers,
aj
--
Anthony Towns [EMAIL PROTECTED] http://azure.humbug.org.au/~aj/
I don't speak for anyone save myself. GPG signed mail preferred.
``_Any_ increase in interface difficulty, in exchange for a benefit you
do not understand, cannot perceive, or don't care about, is too much.''
-- John S. Novak, III (The Humblest Man on the Net)
PGP signature
Re: Bug#93612: Support for new archive structure
On Thu, 12 Apr 2001, Jason Gunthorpe wrote: On Fri, 13 Apr 2001, Philip Charles wrote: Apt-cdrom does not work. dselect works if the file system is strangely modified. apt will work if the CD is copied to a separate partition. Hurd even had apt-cdrom? The ancient version that was packaged sure didn't include it, so I'm not amazed if it didn't work :P I should have said apt does not work with cdroms. Since hurd has a mordern APT now, I don't think this is a worry. If it still doesn't work, then there be a bug in there that should be fixed, and not used as a reason to butcher our CDs. Sure, but people have other priorities. The CDs work and this enables people to get on with other critical development work. Essential packages are not in the Debian archive, but at alpha.gnu.org and must be included. Need I go on? I build the CDs. I don't see any additions to the CD set from another site being a concern at all, as long as they are properly put in a seperate directory. If you tie the security too tightly to CD installation custom Debian CD builders will simply by-pass the security. This would be a "Bad Thing". I am not the only person building custom CDs. Phil. - Philip Charles; 39a Paterson St., Dunedin, New Zealand; +64 3 4882818 Mobile 025 267 9420. I sell GNU/Linux CDs. See http://www.copyleft.co.nz [EMAIL PROTECTED] - preferred. [EMAIL PROTECTED] -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]
Re: Bug#93612: Support for new archive structure
On Fri, Apr 13, 2001 at 06:28:24AM +, Philip Charles wrote: I don't see any additions to the CD set from another site being a concern at all, as long as they are properly put in a seperate directory. If you tie the security too tightly to CD installation custom Debian CD builders will simply by-pass the security. This would be a "Bad Thing". I am not the only person building custom CDs. If you're including .debs from outside Debian, there's no way you're going to get a Debian signature on it anyway. But you're not trying to call the hurd CDs "Official Debian CDs" anyway, so that's not a problem. That's not going to stop you from having your own signature on it though. All you'll need to do is create your own Release file and sign it yourself. Cheers, aj -- Anthony Towns [EMAIL PROTECTED] http://azure.humbug.org.au/~aj/ I don't speak for anyone save myself. GPG signed mail preferred. ``_Any_ increase in interface difficulty, in exchange for a benefit you do not understand, cannot perceive, or don't care about, is too much.'' -- John S. Novak, III (The Humblest Man on the Net) PGP signature
Re: Bug#93612: Support for new archive structure
Le Thu, Apr 12, 2001 at 10:42:33PM -0600, Jason Gunthorpe écrivait: It works because you got lucky, you had a CD that was fortunately constructed properly. It is not supported, and if it does not work, I totally don't care. Of course, the CDs are constructed properly ! I'm in charge of maintaining debian-cd so that it builds "properly constructed" CDs ... I don't see why I need to change it to something where CDs are no more properly constructed ! /me grumbles : I'm really beginning to think that the only valid alternative is to have a Release file and its signature for each CD. Because the release file says Packages, not Packages.cd - why should we make the release file inconsistant? The Release file has been introduced after Packages file, and they should have been thought of a bit more to avoid those problems. We can let Release file mentionning Packages file and have the necessary logic in APT to know that if Packages-signed exists that's the file to use instead of Packages for checking the validity. You can make it specific for cdrom BTW, I don't see the need for that for any other access method. BTW, those changes would be limited to the "update" part of apt-get I guess. Since you'll use the content of Packages-signed files to build the internal apt cache but with only the files mentionned in Packages. Or if you ask particularly nice I might extend the sources.list syntax so you can tell it to read Packages.cd directly I don't want that, it's a hack. :) Cheers, -- Raphaël Hertzog -+- http://strasbourg.linuxfr.org/~raphael/ Le bouche à oreille du Net : http://www.beetell.com Naviguez sans se fatiguer à chercher : http://www.deenoo.com Formation Linux et logiciel libre : http://www.logidee.com -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]
Re: Bug#93612: Support for new archive structure
On Fri, 13 Apr 2001, Raphael Hertzog wrote: Of course, the CDs are constructed properly ! I'm in charge of maintaining debian-cd so that it builds "properly constructed" CDs ... I don't see why I need to change it to something where CDs are no more properly constructed ! /me grumbles : I'm really beginning to think that the only valid alternative is to have a Release file and its signature for each CD. Why not do just this? Whoever builds the Official CDs signs a Release file for each CD. A builder of Unofficial CDs is responsible for signing the Release file for their own CDs. This could be incorporated into debian-cd. Phil. - Philip Charles; 39a Paterson St., Dunedin, New Zealand; +64 3 4882818 Mobile 025 267 9420. I sell GNU/Linux CDs. See http://www.copyleft.co.nz [EMAIL PROTECTED] - preferred. [EMAIL PROTECTED] -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]
Re: Bug#93612: Support for new archive structure
On Thu, Apr 12, 2001 at 10:42:33PM -0600, Jason Gunthorpe wrote: Or if you ask particularly nice I might extend the sources.list syntax so you can tell it to read Packages.cd directly What would happen if you made apt stat every .deb in a file:// url too, on apt-get update, say, and trimmed the Packages file appropriately? Cheers, aj -- Anthony Towns [EMAIL PROTECTED] http://azure.humbug.org.au/~aj/ I don't speak for anyone save myself. GPG signed mail preferred. ``_Any_ increase in interface difficulty, in exchange for a benefit you do not understand, cannot perceive, or don't care about, is too much.'' -- John S. Novak, III (The Humblest Man on the Net) -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]
Re: Bug#93612: Support for new archive structure
On Fri, 13 Apr 2001, Raphael Hertzog wrote: It works because you got lucky, you had a CD that was fortunately constructed properly. It is not supported, and if it does not work, I totally don't care. Of course, the CDs are constructed properly ! I'm in charge of maintaining debian-cd so that it builds "properly constructed" CDs ... I don't see why I need to change it to something where CDs are no more properly constructed ! Because no matter what you do your CD will be invalid in some form, and using a verbatim Packages file is the least pain. /me grumbles : I'm really beginning to think that the only valid alternative is to have a Release file and its signature for each CD. Absolutely not. The Release file has been introduced after Packages file, and they should have been thought of a bit more to avoid those problems. This was fully realized when we constructed the file and accepted as a necessary evil. instead of Packages for checking the validity. You can make it specific for cdrom BTW, I don't see the need for that for any other access method. Did you miss me mentioning that *still* breaks using file:/../ on CD's so why bother? Or if you ask particularly nice I might extend the sources.list syntax so you can tell it to read Packages.cd directly I don't want that, it's a hack. :) No more than all the other ugly things that have been suggested, and this only affects the 3 people silly enough to loopback mount ISO's and try to use APT on them. Jason -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]
Re: Bug#93612: Support for new archive structure
Le Fri, Apr 13, 2001 at 02:23:41PM -0600, Jason Gunthorpe écrivait: Because no matter what you do your CD will be invalid in some form, and using a verbatim Packages file is the least pain. That's the first time I see Debian willing to accept "invalid" CDs instead of designing cleanly the thing from scratch so that such problem don't exist. I'm really beginning to think that the only valid alternative is to have a Release file and its signature for each CD. Absolutely not. Why ? Of course, it's a pain for the Release manager to sign check all those files but I don't see why it wouldn't be an acceptable solution ... It seems a lot cleaner than to have Packages files not corresponding to what is on the CD. And it doesn't suffer from the problem "won't work when used with a file:/ config in apt". BTW, if we can't find an agreement, we can also leave CDs without signature. I'll produce a new and correct Release file but it will never be signed. The Release file has been introduced after Packages file, and they should have been thought of a bit more to avoid those problems. This was fully realized when we constructed the file and accepted as a necessary evil. I don't agree with the "necessary". It's not "necessary". It's just that you're too lazy (not a personal attack) to implement something better for CDs. Cheers, -- Raphaël Hertzog -+- http://strasbourg.linuxfr.org/~raphael/ Le bouche à oreille du Net : http://www.beetell.com Naviguez sans se fatiguer à chercher : http://www.deenoo.com Formation Linux et logiciel libre : http://www.logidee.com -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]
Re: Bug#93612: Support for new archive structure
On Fri, 13 Apr 2001, Jason Gunthorpe wrote: you can tell it to read Packages.cd directly I don't want that, it's a hack. :) No more than all the other ugly things that have been suggested, and this only affects the 3 people silly enough to loopback mount ISO's and try to use APT on them. I have used this myself. It is a good way to test an image. It is also used by people who have borrowed CDs and do not have a burner. Phil. - Philip Charles; 39a Paterson St., Dunedin, New Zealand; +64 3 4882818 Mobile 025 267 9420. I sell GNU/Linux CDs. See http://www.copyleft.co.nz [EMAIL PROTECTED] - preferred. [EMAIL PROTECTED] -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]
Re: Bug#93612: Support for new archive structure
Le Thu, Apr 12, 2001 at 03:20:07PM +1000, Anthony Towns écrivait: debootstrap uses the Release file to work out what Packages files and such to download Many of the files listed in the Release file simply don't exist on all CDs. (and was going to use the md5sums in it to ensure the Packages file wasn't corrupt, too :-/) Duh. Couldn't we generate new Release files and sign them again ? Wouldn't you trust the debian-cd build ? Cheers, -- Raphaël Hertzog -+- http://strasbourg.linuxfr.org/~raphael/ Le bouche à oreille du Net : http://www.beetell.com Naviguez sans se fatiguer à chercher : http://www.deenoo.com Formation Linux et logiciel libre : http://www.logidee.com -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]
Re: Bug#93612: Support for new archive structure
On Wed, 11 Apr 2001, Raphael Hertzog wrote: 2a) Check that the md5sums of the Packages-signed.gz and Sources-signed.gz files you have match the md5sums listed in the Release file 2b) Check that every package listed in each Packages.gz and Sources.gz exactly matches the corresponding entry in Package-signed.gz or Sources-signed.gz, and that there *is* a corresponding entry which is a fair bit more awkward. If you have to modify apt-cdrom at least you can make it manage this precise case. Use "Packages" file for knowing which packages are available Why? apt-cdrom already stats every file to make sure it is available, I have no desire or need to use a paired down file. All apt-cdroms will work with what aj is proposing, but you will get a warning that a lot of files are missing and that may mean the CD is bad. I think the best suggestion was to have a Packages.cd which could be used by non-apt tools that can't cope with extra file names (which are basically random folks's personal scripts). I don't want to remane the files referenced by the release file, that just gets ugly fast. I don't want to change the "standard" Packages files since those are used by all the old tools we have (including those that won't understand why There are no old tools that reliably speak to cds and rely on Packages. Jason -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]
Re: Bug#93612: Support for new archive structure
Le Thu, Apr 12, 2001 at 02:20:49AM -0600, Jason Gunthorpe écrivait: I think the best suggestion was to have a Packages.cd which could be used Packages.cd files exists but exists only for dpkg-multicd which is a dselect method. And it's quite ugly since those files lists all the packages available on the CD and on all the previous CDs ! Each entry in the Packages.cd file has a X-Medium field indicating on which CD it is. I don't want to change the "standard" Packages files since those are used by all the old tools we have (including those that won't understand why There are no old tools that reliably speak to cds and rely on Packages. By old tools, I meant standard tools, not tools dedicated to CDs. Actually many people use standalone Debian CD like a Debian mirror : deb file:/cdrom/debian woody main contrib non-free Other people simply copy cdroms on their harddrive (Gb on ide drives are cheap nowadays) ... And it'd be a pity that doing so wouldn't work as expected. Because apt-cache search would list packages not available, because apt-get would need --fix-missing to work most of the times and so on. Cheers, -- Raphaël Hertzog -+- http://strasbourg.linuxfr.org/~raphael/ Le bouche à oreille du Net : http://www.beetell.com Naviguez sans se fatiguer à chercher : http://www.deenoo.com Formation Linux et logiciel libre : http://www.logidee.com -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]
Re: Bug#93612: Support for new archive structure
On Thu, 12 Apr 2001, Jason Gunthorpe wrote: On Wed, 11 Apr 2001, Raphael Hertzog wrote: 2a) Check that the md5sums of the Packages-signed.gz and Sources-signed.gz files you have match the md5sums listed in the Release file 2b) Check that every package listed in each Packages.gz and Sources.gz exactly matches the corresponding entry in Package-signed.gz or Sources-signed.gz, and that there *is* a corresponding entry which is a fair bit more awkward. If you have to modify apt-cdrom at least you can make it manage this precise case. Use "Packages" file for knowing which packages are available Why? apt-cdrom already stats every file to make sure it is available, I have no desire or need to use a paired down file. All apt-cdroms will work with what aj is proposing, but you will get a warning that a lot of files are missing and that may mean the CD is bad. I think the best suggestion was to have a Packages.cd which could be used by non-apt tools that can't cope with extra file names (which are basically random folks's personal scripts). I don't want to remane the files referenced by the release file, that just gets ugly fast. I don't want to change the "standard" Packages files since those are used by all the old tools we have (including those that won't understand why There are no old tools that reliably speak to cds and rely on Packages. Until yesterday, I did agree because I didn't think any further. However one of Raphaels earlier mails changed that. Namely: people do not use a CD as CD at all times! I know many people that buy a CD set (because they don't have the bandwidth) only to put the contents on their harddisks (because they do have the space). Currently this is very easy, just copy each CD to a separate directory, create the appropriate entries in sources.list and that's it. With a complete Packages file on each CD, this needs apt's file: method to also stat every possible Filename: in each of those CD-directories. (No, there is no alternative. Copying the contents of all CDs to one single directory changes nothing because we can't be sure that _all_ CDs are copied -- possibly there's only space for one or two copied CDs and the rest keeps going via apt-cdrom. And having every user run dpkg-scanpackages is really very bad.) But that's not all. Those copied dirs, or a set of mounted CDs (or both) may be exported via NFS to some other machine (which is quite common in LANs). Even if this were only one CD, there would be about 6000 stat requests going over the network. This is bad. Not simply stat-ing but "find -name '*.deb'" would probably be better, but doesn't really solve things either. And it gets even worse. The dirs mounted CDs may be exported over HTTP (maybe even more common these days). You can't trust the webserver on generating any useful directory index, so you've got to send HEAD requests for all 6000 files in the complete Packages file. Either you can do this parallelly which will cause a spectacular load on the server, or serially which will take enormous amounts of time. Furthermore since the Packages file does not any longer describe the exact files present on the server, you need to check everything every time you run apt-get update. Of course, this has to be implemented in apt's file:, ftp: and http: methods. And dselect's `mounted' and `http' method. And dpkg-ftp. And... So I think we should continue to generate _correct_ Packages files for each CD, and solve the "signing issue" using some other method. Regards, Anne Bezemer -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]
Re: Bug#93612: Support for new archive structure
On Thu, Apr 12, 2001 at 08:58:30AM +0200, Raphael Hertzog wrote: (and was going to use the md5sums in it to ensure the Packages file wasn't corrupt, too :-/) Duh. Couldn't we generate new Release files and sign them again ? Wouldn't you trust the debian-cd build ? Not if I can avoid it. If we can avoid trusting the debian-cd build, and still have the CD themselves be trustworthy, then that's a win. It means we don't have to worry about the security of the machine doing the builds, or worry who's building it, or anything similar. Signing a different file on every CD means there are around around 30 different files to check the validity of and get people to sign; assuming we want the CDs to be signed by the RM and the security team (which seems sensible to me) then they have to assure themselves that all those files are valid. Further, people like debian-jr can't just use the same scripts with appropriate tweaks and end up with a CD#1'ish subset of Debian that's equally "secure" as the official CD#1. Cheers, aj -- Anthony Towns [EMAIL PROTECTED] http://azure.humbug.org.au/~aj/ I don't speak for anyone save myself. GPG signed mail preferred. ``_Any_ increase in interface difficulty, in exchange for a benefit you do not understand, cannot perceive, or don't care about, is too much.'' -- John S. Novak, III (The Humblest Man on the Net) -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]
Re: Bug#93612: Support for new archive structure
On Thu, 12 Apr 2001, Raphael Hertzog wrote: Le Thu, Apr 12, 2001 at 02:20:49AM -0600, Jason Gunthorpe crivait: I think the best suggestion was to have a Packages.cd which could be used Packages.cd files exists but exists only for dpkg-multicd which is a dselect method. And it's quite ugly since those files lists all the packages available on the CD and on all the previous CDs ! Each entry in the Packages.cd file has a X-Medium field indicating on which CD it is. Kill that, and use a normal Packages.cd then, or name it something else, whatever. By old tools, I meant standard tools, not tools dedicated to CDs. Actually many people use standalone Debian CD like a Debian mirror : APT does not support that, if it does not work then too bad, I don't care. :P Other people simply copy cdroms on their harddrive (Gb on ide drives are cheap nowadays) ... cp Packages.cd Packages No Problem (tm) People who do that are more likely to copy the entire CD set to the hard disk so they do in fact have a complete mirror, and should have normal signed package files in the right place. Jason -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]
Re: Bug#93612: Support for new archive structure
On Thu, 12 Apr 2001, Philip Charles wrote: On Thu, 12 Apr 2001, J.A. Bezemer wrote: So I think we should continue to generate _correct_ Packages files for each CD, and solve the "signing issue" using some other method. I repeat my earlier suggestion. Sign md5sums.gz, this is supposed to be the accuracy and security file. A script could be provided for the Insufficiant, and prone to the same problems AJ described. The Release signature covers the pre-scanned package meta data and that is important too. Jason -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]
Re: Bug#93612: Support for new archive structure
On Thu, 12 Apr 2001, Raphael Hertzog wrote: ** Why should the normal Packages file be named differently and not the new files that we are introducing ? People who do that are more likely to copy the entire CD set to the hard disk so they do in fact have a complete mirror, and should have normal signed package files in the right place. Yes, but they won't always copy them in the same place, they copy them in different directories and use several lines like "deb file:/tmp/CD1 .." and so on. I for one use several directories like that when I mount (with the loopback) several ISO images : # mount -o loop /debian-cd/debian-woody-1.raw /mnt/cd1 and so on ... With the Hurd CDs it is even worse. The file structure on the CDs has only a passing resemblance to any file system found on a Debian CD or mirror past or present. It has to be like that or it will not work. If a signed md5sums.gz is not secure enough then create another signed security file. Packages files are for installations, leave it that way or destroy their usefulness and flexibility. Phil. - Philip Charles; 39a Paterson St., Dunedin, New Zealand; +64 3 4882818 Mobile 025 267 9420. I sell GNU/Linux CDs. See http://www.copyleft.co.nz [EMAIL PROTECTED] - preferred. [EMAIL PROTECTED] -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]
Re: Bug#93612: Support for new archive structure
On Thu, 12 Apr 2001, Raphael Hertzog wrote: APT does not support that, if it does not work then too bad, I don't care. :P Apt always supported that. If I put my CD and mount it on /cdrom and use "deb file:/cdrom/debian woody main contrib non-free" in sources.list, it does work ! It works because you got lucky, you had a CD that was fortunately constructed properly. It is not supported, and if it does not work, I totally don't care. Why should the normal Packages file be named differently and not the new files that we are introducing ? Because the release file says Packages, not Packages.cd - why should we make the release file inconsistant? If you point apt at a CD with a valid release file and a mangled package file it won't work! It will look at it, tell you it's hacked and then you'll have to muck around to get it to stop doing that. I for one use several directories like that when I mount (with the loopback) several ISO images : # mount -o loop /debian-cd/debian-woody-1.raw /mnt/cd1 and so on ... cd /.../jnk ln -s /mnt/cd* . apt-ftparchive packages . Packages deb file:/.../jnk ./ Or if you ask particularly nice I might extend the sources.list syntax so you can tell it to read Packages.cd directly Jason -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]
Re: Bug#93612: Support for new archive structure
On Fri, 13 Apr 2001, Philip Charles wrote: With the Hurd CDs it is even worse. The file structure on the CDs has only a passing resemblance to any file system found on a Debian CD or mirror past or present. It has to be like that or it will not work. I don't even think I want to know.. I'm going to have a really hard time accepting that hurd can't read a CD that is properly structured, even if the ones they make now are weird. Particularly since it is APT that is reading the CD :P Jason -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]
Re: Bug#93612: Support for new archive structure
On Thu, 12 Apr 2001, Jason Gunthorpe wrote: On Fri, 13 Apr 2001, Philip Charles wrote: With the Hurd CDs it is even worse. The file structure on the CDs has only a passing resemblance to any file system found on a Debian CD or mirror past or present. It has to be like that or it will not work. I don't even think I want to know.. I'm going to have a really hard time accepting that hurd can't read a CD that is properly structured, even if the ones they make now are weird. Particularly since it is APT that is reading the CD :P Apt-cdrom does not work. dselect works if the file system is strangely modified. apt will work if the CD is copied to a separate partition. Essential packages are not in the Debian archive, but at alpha.gnu.org and must be included. Need I go on? I build the CDs. Phil. - Philip Charles; 39a Paterson St., Dunedin, New Zealand; +64 3 4882818 Mobile 025 267 9420. I sell GNU/Linux CDs. See http://www.copyleft.co.nz [EMAIL PROTECTED] - preferred. [EMAIL PROTECTED] -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]
Re: Bug#93612: Support for new archive structure
On Fri, 13 Apr 2001, Philip Charles wrote: Apt-cdrom does not work. dselect works if the file system is strangely modified. apt will work if the CD is copied to a separate partition. Hurd even had apt-cdrom? The ancient version that was packaged sure didn't include it, so I'm not amazed if it didn't work :P Since hurd has a mordern APT now, I don't think this is a worry. If it still doesn't work, then there be a bug in there that should be fixed, and not used as a reason to butcher our CDs. Essential packages are not in the Debian archive, but at alpha.gnu.org and must be included. Need I go on? I build the CDs. I don't see any additions to the CD set from another site being a concern at all, as long as they are properly put in a seperate directory. Jason -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]
Processed: Re: Bug#93612: Support for new archive structure
Processing commands for [EMAIL PROTECTED]: reassign 93612 debian-cd Bug#93612: Support for new archive structure Bug reassigned from package `cdimage.debian.org' to `debian-cd'. thanks Stopping processing here. Please contact me if you need assistance. Darren Benham (administrator, Debian Bugs database) -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]
Re: Bug#93612: Support for new archive structure
Jason: wassup with apt-cdrom and dists/woody/Release and such? On Wed, Apr 11, 2001 at 09:45:58AM +0200, Raphael Hertzog wrote: Le Wed, Apr 11, 2001 at 01:05:41PM +1000, Anthony Towns crivait: For this to work, the Release and Release.gpg files should be verbatim This is not a problem, we just need to copy Release.gpg as well. Note that the two files: dists/woody/main/binary-i386/Release dists/woody/Release are quite different. Are you already copying dists/woody/Release or just dists/woody/main/binary-i386/Release? from the archive. For that to work, the Packages and Sources files also must be verbatim from the archive. If you do this, then the verifying a mirror or a CD looks like: 0) Check that Release describes the distribution you think it's meant to 1) Check that Release.gpg is a detached signature for Release, signed with the right key 2) Check that the md5sums of the Packages.gz and Sources.gz files you have match the md5sums listed in the Release file 3) Check the md5sums of the debs you have match the md5sums listed in the Packages files (0) is a user interface issue, (3) is already done by apt, and (1) and (2) are reasonably straightforward additions. This is a problem. I *really* don't like having Packages and Sources files mentionning files that are not available. It goes against some principles I always tried to follow. debian-cd has been written in order to be able to generate CD which contains subset of Debian and I don't want to have to put the complete Packages file for each CD set we'll create with debian-cd. OTOH, if you *don't* do this, verfication becomes much harder. An acceptable alternative would be to provide Packages.signed and Sources.signed that could be checked against Release.gpg and a check for a package "validity" would be to compare if the 2 or 3 informations do match (Packages, Packages.signed and the package itself). For example, if you have separate files, you'd need to change step (2) to be: 2a) Check that the md5sums of the Packages-signed.gz and Sources-signed.gz files you have match the md5sums listed in the Release file 2b) Check that every package listed in each Packages.gz and Sources.gz exactly matches the corresponding entry in Package-signed.gz or Sources-signed.gz, and that there *is* a corresponding entry which is a fair bit more awkward. ...should be the sort of thing to do. Unfortunately debian-cd is a bit more complicated. :) Well, naturally... trigger a bunch of apt-cdrom warnings still when people try to install from it, but those are things that need to be fixed in apt-cdrom... I'm not sure that this is really the way to go. apt-cdrom has been designed to be able to use different CDs from different CD set, it will build a list of the files mentionned on each CD, so it's a big win that each CD only mentions what it does really have ! Well, another way of handling that would be to maintain a separate list of which packages are on the CD. This'd be pretty easy to do, and wouldn't have any impact on security issues. It'd need support from apt-cdrom, perhaps. Something like: dists/woody/ Release main/binary-i386/ Packages.gz Packages-Present.gz where Packages-Present.gz is just a list like: anacron apt dpkg libc6 without any additional information. Cheers, aj -- Anthony Towns [EMAIL PROTECTED] http://azure.humbug.org.au/~aj/ I don't speak for anyone save myself. GPG signed mail preferred. ``_Any_ increase in interface difficulty, in exchange for a benefit you do not understand, cannot perceive, or don't care about, is too much.'' -- John S. Novak, III (The Humblest Man on the Net) PGP signature
Re: Bug#93612: Support for new archive structure
IMHO is would be easier to include a signed version of md5sums.gz on each CD. This would still mean the the integrity of the packages could be checked with confidence and would enable the detection of foreign packages. Someone might want to write a script that would automate the process. Phil. On Wed, 11 Apr 2001, Raphael Hertzog wrote: * This is a problem. I *really* don't like having Packages and Sources files mentionning files that are not available. It goes against some principles I always tried to follow. debian-cd has been written in order to be able to generate CD which contains subset of Debian and I don't want to have to put the complete Packages file for each CD set we'll create with debian-cd. An acceptable alternative would be to provide Packages.signed and Sources.signed that could be checked against Release.gpg and a check for a package "validity" would be to compare if the 2 or 3 informations do match (Packages, Packages.signed and the package itself). * Unfortunately debian-cd is a bit more complicated. :) I'm not sure that this is really the way to go. apt-cdrom has been designed to be able to use different CDs from different CD set, it will build a list of the files mentionned on each CD, so it's a big win that each CD only mentions what it does really have ! - Philip Charles; 39a Paterson St., Dunedin, New Zealand; +64 3 4882818 Mobile 025 267 9420. I sell GNU/Linux CDs. See http://www.copyleft.co.nz [EMAIL PROTECTED] - preferred. [EMAIL PROTECTED] -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]
Re: Bug#93612: Support for new archive structure
Le Wed, Apr 11, 2001 at 11:15:34AM -0400, Adam Di Carlo écrivait: If you all on the debian-cd list could prioritize the issues which are preventing debootstrap from working (the one I know about is the dists/woody/Release file) that would be great. Right now I can't do testing using woody CDs. We are currently unaware of any issues preventing deboostrap from working. We can change whatever is needed. The Release stuff will be worked out soon but if you know any other problem please tell us. Cheers, -- Raphaël Hertzog -+- http://strasbourg.linuxfr.org/~raphael/ Le bouche à oreille du Net : http://www.beetell.com Naviguez sans se fatiguer à chercher : http://www.deenoo.com Formation Linux et logiciel libre : http://www.logidee.com -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]
Re: Bug#93612: Support for new archive structure
On Wed, Apr 11, 2001 at 06:01:30PM +0200, Raphael Hertzog wrote: Le Wed, Apr 11, 2001 at 11:15:34AM -0400, Adam Di Carlo crivait: If you all on the debian-cd list could prioritize the issues which are preventing debootstrap from working (the one I know about is the dists/woody/Release file) that would be great. Right now I can't do testing using woody CDs. We are currently unaware of any issues preventing deboostrap from working. We can change whatever is needed. The Release stuff will be worked out soon but if you know any other problem please tell us. debootstrap uses the Release file to work out what Packages files and such to download (and was going to use the md5sums in it to ensure the Packages file wasn't corrupt, too :-/) Cheers, aj -- Anthony Towns [EMAIL PROTECTED] http://azure.humbug.org.au/~aj/ I don't speak for anyone save myself. GPG signed mail preferred. ``_Any_ increase in interface difficulty, in exchange for a benefit you do not understand, cannot perceive, or don't care about, is too much.'' -- John S. Novak, III (The Humblest Man on the Net) -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]
