Bug#456332: dpkg could use an elevated pre-depends or depends on lzma
Raphael Hertzog writes (Re: Bug#456332: dpkg could use an elevated pre-depends or depends on lzma): I'm still not convinced that this is the right approach, Pre-Depends are supposed to not be used lightly. Does the package need to be configured, isn't a simple dependency enough ? Yes, the decompressor needs to be configured because its dependencies need to have been installed. And it has to be a predependency because the _unpack_ requires the decompressor. Depends is not checked until after unpacking. And we're speaking of: $ dpkg -s lzma | grep Installed Installed-Size: 292 $ dpkg -s bzip2 | grep Installed Installed-Size: 124 Between them that's nearly half a megabyte. (That's not to mention the couple more we're bound to acquire if we make this actually work well.) The downsides of a Pre-Depends is that it makes installation ordering much more tricky - but this Pre-Depends is in fact an accurate reflection of the situation. If the Pre-Depends is absent and installation operations are done that it would have prevented, those operations will fail. Ian. -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Bug#456332: dpkg could use an elevated pre-depends or depends on lzma
[ CCed Mohammed for the lzma package size reduction part. ] Hi, On Fri, 25 Jan 2008, Ian Jackson wrote: Raphael Hertzog writes (Bug#456332: dpkg could use an elevated pre-depends or depends on lzma): The debian/control field is the only viable option IMO. It would be somewhat similar to the Package-Type: header which has no real use except influencing the behaviour of another tool during the build. Having slept on it I think this is the right answer. Compression-Type: gzip|bzip2|lzma|... converted into Pre-Depends: deb-decompressor-lzma by dpkg-gencontrol The problem with that is when unpacking and repacking a .deb with a different compression, dpkg-deb would have to munge the Pre-Depends. Even if that should be safe with such virtual package, I don't feel comfortable doing that. The other problem is that all current packages using bzip2 or lzma would have to gain such dependency (supposedly gzip ones as well?!), there should not be many using bzip2 or lzma, though. On Fri, 2008-01-25 at 18:54:10 +, Ian Jackson wrote: Raphael Hertzog writes (Re: Bug#456332: dpkg could use an elevated pre-depends or depends on lzma): I'm still not convinced that this is the right approach, Pre-Depends are supposed to not be used lightly. Does the package need to be configured, isn't a simple dependency enough ? Yes, I'm not convinced either that using a dependency is the way to go (although yes, I agree that if we'd have to use one, that should be a Pre-Depends). And we're speaking of: $ dpkg -s lzma | grep Installed Installed-Size: 292 $ dpkg -s bzip2 | grep Installed Installed-Size: 124 Between them that's nearly half a megabyte. (That's not to mention the couple more we're bound to acquire if we make this actually work well.) Probably lzma_alone (104 KiB) could be removed at some point from the lzma package, and few docs moved to lzma-dev where they belong (20 KiB). Also probably only that one package in base switched to lzma we'd recover that space anyway. The downsides of a Pre-Depends is that it makes installation ordering much more tricky - but this Pre-Depends is in fact an accurate reflection of the situation. If the Pre-Depends is absent and installation operations are done that it would have prevented, those operations will fail. Ian, I can see your point, about the correctness (depends on demand), and not encoding the archive state in the metadata of all .deb (virtual package). But on the other hand it's dpkg who is calling the lzma binary, it's an implementation detail as Chris said. Also dpkg either supports it or it does not, regardless of any Pre-Depends any package might have. And we do something similar with Essential packages. The only concern I can see with dpkg having the Pre-Depends is that if we have a .deb with an unsupported compression, installation will fail too late. But I don't think that's that important as this should not happen on the archive anyway (those would get rejected). Also just for reference, the main reason I didn't add the Pre-Depends to dpkg in etch was because it was near a freeze (or already frozen, don't remember now) and didn't want to destabilize d-i at that time. That was probably a mistake, but I don't regret having played safe, there's always time in the future to fix things. regards, guillem -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Bug#456332: dpkg could use an elevated pre-depends or depends on lzma
Ian Jackson writes (Re: Bug#456332: dpkg could use an elevated pre-depends or depends on lzma): by dpkg-gencontrol and into an appropriate -Z option by dpkg-buildpackage, then ? Oh, no, not dpkg-buildpackage. Hrm. It really has to be in dpkg-deb, which ought to strip out the special header too. Ian. -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Bug#456332: dpkg could use an elevated pre-depends or depends on lzma
On Fri, 25 Jan 2008, Ian Jackson wrote: Raphael Hertzog writes (Bug#456332: dpkg could use an elevated pre-depends or depends on lzma): The debian/control field is the only viable option IMO. It would be somewhat similar to the Package-Type: header which has no real use except influencing the behaviour of another tool during the build. Having slept on it I think this is the right answer. Compression-Type: gzip|bzip2|lzma|... converted into Pre-Depends: deb-decompressor-lzma by dpkg-gencontrol Ok. and into an appropriate -Z option by dpkg-buildpackage, then ? Well, no. dpkg-buildpackage doesn't call dpkg-deb --build. dpkg-gencontrol must push the field in DEBIAN/control and dpkb-deb --build must read that field there and use the corresponding compressor. I'm still not convinced that this is the right approach, Pre-Depends are supposed to not be used lightly. Does the package need to be configured, isn't a simple dependency enough ? And furthermore, while I can understand the need to not add the dependency to dpkg itself, in practice most systems will have all the compressors installed anyway. And we're speaking of: $ dpkg -s lzma | grep Installed Installed-Size: 292 $ dpkg -s bzip2 | grep Installed Installed-Size: 124 Cheers, -- Raphaël Hertzog Le best-seller français mis à jour pour Debian Etch : http://www.ouaza.com/livre/admin-debian/
Bug#456332: dpkg could use an elevated pre-depends or depends on lzma
Raphael Hertzog writes (Bug#456332: dpkg could use an elevated pre-depends or depends on lzma): The debian/control field is the only viable option IMO. It would be somewhat similar to the Package-Type: header which has no real use except influencing the behaviour of another tool during the build. Having slept on it I think this is the right answer. Compression-Type: gzip|bzip2|lzma|... converted into Pre-Depends: deb-decompressor-lzma by dpkg-gencontrol and into an appropriate -Z option by dpkg-buildpackage, then ? Ian. -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Bug#456332: dpkg could use an elevated pre-depends or depends on lzma
On Thu, 24 Jan 2008, Ian Jackson wrote: Yes, but if the decompressor packages are reorganised at all, or we decide to change again how it is implemented in dpkg, all of those packages will become broken. Given that this will be a pre-depends and will be in a large number of packages, I think it is important that it be very stable. Ok, makes sense. I would only be in favor of that if we modify dpkg to auto-generate that dependency, unfortunately right now we only know how to compress when dpkg-deb -b -Zsomething is called and this is after dpkg-gencontrol obviously. How about if we arrange to pass -Zsomething to dpkg-gencontrol, and have a table in dpkg-dev to map something to deb-decompressor-something for values where it is needed ? The problematic part is arrange to pass -Zsomething to dpkg-gencontrol ... the -Z of dpkg-buildpackage is for dpkg-source and not dpkg-deb -b. And even if we knew right from the start what compressor to use, the dpkg-gencontrol call is hidden in debian/rules and we have no way to pass anything (except maybe an environment variable which isn't really nice). Cheers, -- Raphaël Hertzog Le best-seller français mis à jour pour Debian Etch : http://www.ouaza.com/livre/admin-debian/
Bug#456332: dpkg could use an elevated pre-depends or depends on lzma
Raphael Hertzog writes (Re: Bug#456332: dpkg could use an elevated pre-depends or depends on lzma): On Thu, 24 Jan 2008, Ian Jackson wrote: How about if we arrange to pass -Zsomething to dpkg-gencontrol, and have a table in dpkg-dev to map something to deb-decompressor-something for values where it is needed ? The problematic part is arrange to pass -Zsomething to dpkg-gencontrol ... the -Z of dpkg-buildpackage is for dpkg-source and not dpkg-deb -b. And even if we knew right from the start what compressor to use, the dpkg-gencontrol call is hidden in debian/rules and we have no way to pass anything (except maybe an environment variable which isn't really nice). Yes, I obviously hadn't thought about this too deeply. How do we expect to choose which packages will use which compressors ? Ideally in the end the maintainer would decide, so it ought to be in the source package somewhere. But we want to override it too. I think for overriding it, an environment variable is fine. Perhaps we could just tell maintainers to put export DPKG_DEB_COMPRESSOR ?= ... in their rules. Hrm, that doesn't work so well if different packages want different compression schemes. A debian/control file field ? Ian. -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Bug#456332: dpkg could use an elevated pre-depends or depends on lzma
On Thu, 24 Jan 2008, Ian Jackson wrote: How do we expect to choose which packages will use which compressors ? Right now, maintainers choose that by adding the right -Z option in the dh_builddeb call (which forwards the option to dpkg-deb -b). So they put it directly in the rules file. Ideally in the end the maintainer would decide, so it ought to be in the source package somewhere. But we want to override it too. I think for overriding it, an environment variable is fine. Perhaps we could just tell maintainers to put export DPKG_DEB_COMPRESSOR ?= ... in their rules. Hrm, that doesn't work so well if different packages want different compression schemes. A debian/control file field ? The debian/control field is the only viable option IMO. It would be somewhat similar to the Package-Type: header which has no real use except influencing the behaviour of another tool during the build. Cheers, -- Raphaël Hertzog Le best-seller français mis à jour pour Debian Etch : http://www.ouaza.com/livre/admin-debian/
Bug#456332: dpkg could use an elevated pre-depends or depends on lzma
Raphael Hertzog writes (Bug#456332: dpkg could use an elevated pre-depends or depends on lzma): On Tue, 18 Dec 2007, Ian Jackson wrote: IMO the lzma binary package should Provide a new virtual package name, lzma-deb-support or some such. Packages could Pre-Depend on that. What does it bring? I assume you mean why do it like that. The answer is that you don't need the lzma support in cases where it's not needed. The dependencies are more accurate. Also, see my previous comments: that dpkg should execute decompressors as subprocesses (the way it used to before doogie changed it). Decompression also takes more memory than bzip2 but with 32M for the bigger packages, I think it's okay. I have machines still in service with as little as 20Mby of RAM. Obviously they don't have the biggest packages but such memory use during decompression can be a problem. Ian. -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Bug#456332: dpkg could use an elevated pre-depends or depends on lzma
On Tue, 22 Jan 2008, Ian Jackson wrote: Raphael Hertzog writes (Bug#456332: dpkg could use an elevated pre-depends or depends on lzma): On Tue, 18 Dec 2007, Ian Jackson wrote: IMO the lzma binary package should Provide a new virtual package name, lzma-deb-support or some such. Packages could Pre-Depend on that. What does it bring? I assume you mean why do it like that. The answer is that you don't need the lzma support in cases where it's not needed. The dependencies are more accurate. You don't need a virtual package for that. You just need to have each package depend on the right decompressor. I would only be in favor of that if we modify dpkg to auto-generate that dependency, unfortunately right now we only know how to compress when dpkg-deb -b -Zsomething is called and this is after dpkg-gencontrol obviously. Cheers, -- Raphaël Hertzog Le best-seller français mis à jour pour Debian Etch : http://www.ouaza.com/livre/admin-debian/
Bug#456332: dpkg could use an elevated pre-depends or depends on lzma
Chris Cheney writes (Bug#456332: dpkg could use an elevated pre-depends or depends on lzma): dpkg supports using lzma for compression of debs however it only suggests the lzma binary package which is what is _currently_ used for decompressing the debs. In the future it would be preferable if a liblzma package existed that dpkg could use similarly to how it uses libbz2. IMHO packages should not pre-depend on the lzma binary package since it is an implementation level detail of how dpkg supports lzma binaries currently. So dpkg needs to pre-depends/depends on the lzma binary package so that packages can use lzma compression. However packages using lzma compression will need a pre-depends on the correct version of dpkg (the one with the depends/pre-depends on lzma). IMO the lzma binary package should Provide a new virtual package name, lzma-deb-support or some such. Packages could Pre-Depend on that. (How valuable is lzma compression?) I disagree that dpkg ought to support lzma the same way it supports bzip2. Rather I think it ought to support gzip and bzip2 the way it supports lzma: by calling an external program, as I explained in my mail [EMAIL PROTECTED] to debian-devel on Tue, 16 Oct 2007 14:34:36 +0100. Ian. -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Bug#456332: dpkg could use an elevated pre-depends or depends on lzma
Package: dpkg dpkg supports using lzma for compression of debs however it only suggests the lzma binary package which is what is _currently_ used for decompressing the debs. In the future it would be preferable if a liblzma package existed that dpkg could use similarly to how it uses libbz2. IMHO packages should not pre-depend on the lzma binary package since it is an implementation level detail of how dpkg supports lzma binaries currently. So dpkg needs to pre-depends/depends on the lzma binary package so that packages can use lzma compression. However packages using lzma compression will need a pre-depends on the correct version of dpkg (the one with the depends/pre-depends on lzma). FWIW - Ubuntu already does the above so that it can have lzma compressed debs. There is more information about what Ubuntu has done and is planning on doing and how much lzma helps at: https://wiki.ubuntu.com/dpkg-lzma There is also a patch in Ubuntu for apt-utils apt-ftparchive to support lzma compressed debs that I wrote that someone may want to pull into Debian. The above page has the patch along with including support for Packages files being compressed with lzma except for one pkgAcqIndex::Failed function that needs to be corrected. If the Packages file support isn't wanted then it will need to be split out of the diff. Thanks, Chris Cheney -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
