Bug#456332: dpkg could use an elevated pre-depends or depends on lzma

2008-01-25 Thread Ian Jackson
Raphael Hertzog writes (Re: Bug#456332: dpkg could use an elevated pre-depends 
or depends  on lzma):
 I'm still not convinced that this is the right approach, Pre-Depends are
 supposed to not be used lightly. Does the package need to be configured,
 isn't a simple dependency enough ?

Yes, the decompressor needs to be configured because its dependencies
need to have been installed.  And it has to be a predependency because
the _unpack_ requires the decompressor.  Depends is not checked until
after unpacking.

 And we're speaking of:
 $ dpkg -s lzma | grep Installed
 Installed-Size: 292
 $ dpkg -s bzip2 | grep Installed
 Installed-Size: 124

Between them that's nearly half a megabyte.  (That's not to mention
the couple more we're bound to acquire if we make this actually work
well.)

The downsides of a Pre-Depends is that it makes installation ordering
much more tricky - but this Pre-Depends is in fact an accurate
reflection of the situation.  If the Pre-Depends is absent and
installation operations are done that it would have prevented, those
operations will fail.

Ian.




-- 
To UNSUBSCRIBE, email to [EMAIL PROTECTED]
with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]



Bug#456332: dpkg could use an elevated pre-depends or depends on lzma

2008-01-25 Thread Guillem Jover
[ CCed Mohammed for the lzma package size reduction part. ]

Hi,

On Fri, 25 Jan 2008, Ian Jackson wrote:
 Raphael Hertzog writes (Bug#456332: dpkg could use an elevated pre-depends 
 or depends on lzma):
  The debian/control field is the only viable option IMO. It would be
  somewhat similar to the Package-Type: header which has no real use except
  influencing the behaviour of another tool during the build.

 Having slept on it I think this is the right answer.
   Compression-Type: gzip|bzip2|lzma|...
 converted into
   Pre-Depends: deb-decompressor-lzma
 by dpkg-gencontrol

The problem with that is when unpacking and repacking a .deb with a
different compression, dpkg-deb would have to munge the Pre-Depends.
Even if that should be safe with such virtual package, I don't feel
comfortable doing that.

The other problem is that all current packages using bzip2 or lzma
would have to gain such dependency (supposedly gzip ones as well?!),
there should not be many using bzip2 or lzma, though.

On Fri, 2008-01-25 at 18:54:10 +, Ian Jackson wrote:
 Raphael Hertzog writes (Re: Bug#456332: dpkg could use an elevated 
 pre-depends or depends on lzma):
  I'm still not convinced that this is the right approach, Pre-Depends are
  supposed to not be used lightly. Does the package need to be configured,
  isn't a simple dependency enough ?

Yes, I'm not convinced either that using a dependency is the way to go
(although yes, I agree that if we'd have to use one, that should be a
Pre-Depends).

  And we're speaking of:
  $ dpkg -s lzma | grep Installed
  Installed-Size: 292
  $ dpkg -s bzip2 | grep Installed
  Installed-Size: 124
 
 Between them that's nearly half a megabyte.  (That's not to mention
 the couple more we're bound to acquire if we make this actually work
 well.)

Probably lzma_alone (104 KiB) could be removed at some point from the
lzma package, and few docs moved to lzma-dev where they belong (20 KiB).
Also probably only that one package in base switched to lzma we'd recover
that space anyway.

 The downsides of a Pre-Depends is that it makes installation ordering
 much more tricky - but this Pre-Depends is in fact an accurate
 reflection of the situation.  If the Pre-Depends is absent and
 installation operations are done that it would have prevented, those
 operations will fail.

Ian, I can see your point, about the correctness (depends on demand),
and not encoding the archive state in the metadata of all .deb (virtual
package). But on the other hand it's dpkg who is calling the lzma
binary, it's an implementation detail as Chris said. Also dpkg either
supports it or it does not, regardless of any Pre-Depends any package
might have. And we do something similar with Essential packages.

The only concern I can see with dpkg having the Pre-Depends is that
if we have a .deb with an unsupported compression, installation will
fail too late. But I don't think that's that important as this should
not happen on the archive anyway (those would get rejected).

Also just for reference, the main reason I didn't add the Pre-Depends to
dpkg in etch was because it was near a freeze (or already frozen, don't
remember now) and didn't want to destabilize d-i at that time. That was
probably a mistake, but I don't regret having played safe, there's
always time in the future to fix things.

regards,
guillem




-- 
To UNSUBSCRIBE, email to [EMAIL PROTECTED]
with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]



Bug#456332: dpkg could use an elevated pre-depends or depends on lzma

2008-01-25 Thread Ian Jackson
Ian Jackson writes (Re: Bug#456332: dpkg could use an elevated pre-depends or 
depends on lzma):
 by dpkg-gencontrol and into an appropriate -Z option by
 dpkg-buildpackage, then ?

Oh, no, not dpkg-buildpackage.  Hrm.  It really has to be in dpkg-deb,
which ought to strip out the special header too.

Ian.




-- 
To UNSUBSCRIBE, email to [EMAIL PROTECTED]
with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]



Bug#456332: dpkg could use an elevated pre-depends or depends on lzma

2008-01-25 Thread Raphael Hertzog
On Fri, 25 Jan 2008, Ian Jackson wrote:
 Raphael Hertzog writes (Bug#456332: dpkg could use an elevated pre-depends 
 or depends on lzma):
  The debian/control field is the only viable option IMO. It would be
  somewhat similar to the Package-Type: header which has no real use except
  influencing the behaviour of another tool during the build.
 
 Having slept on it I think this is the right answer.
   Compression-Type: gzip|bzip2|lzma|...
 converted into
   Pre-Depends: deb-decompressor-lzma
 by dpkg-gencontrol 

Ok.

 and into an appropriate -Z option by dpkg-buildpackage, then ?

Well, no. dpkg-buildpackage doesn't call dpkg-deb --build. dpkg-gencontrol
must push the field in DEBIAN/control and dpkb-deb --build must read
that field there and use the corresponding compressor.

I'm still not convinced that this is the right approach, Pre-Depends are
supposed to not be used lightly. Does the package need to be configured,
isn't a simple dependency enough ?

And furthermore, while I can understand the need to not add the dependency
to dpkg itself, in practice most systems will have all the compressors
installed anyway.

And we're speaking of:
$ dpkg -s lzma | grep Installed
Installed-Size: 292
$ dpkg -s bzip2 | grep Installed
Installed-Size: 124

Cheers,
-- 
Raphaël Hertzog

Le best-seller français mis à jour pour Debian Etch :
http://www.ouaza.com/livre/admin-debian/





Bug#456332: dpkg could use an elevated pre-depends or depends on lzma

2008-01-25 Thread Ian Jackson
Raphael Hertzog writes (Bug#456332: dpkg could use an elevated pre-depends or 
depends on lzma):
 The debian/control field is the only viable option IMO. It would be
 somewhat similar to the Package-Type: header which has no real use except
 influencing the behaviour of another tool during the build.

Having slept on it I think this is the right answer.
  Compression-Type: gzip|bzip2|lzma|...
converted into
  Pre-Depends: deb-decompressor-lzma
by dpkg-gencontrol and into an appropriate -Z option by
dpkg-buildpackage, then ?

Ian.




-- 
To UNSUBSCRIBE, email to [EMAIL PROTECTED]
with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]



Bug#456332: dpkg could use an elevated pre-depends or depends on lzma

2008-01-24 Thread Raphael Hertzog
On Thu, 24 Jan 2008, Ian Jackson wrote:
 Yes, but if the decompressor packages are reorganised at all, or we
 decide to change again how it is implemented in dpkg, all of those
 packages will become broken.
 
 Given that this will be a pre-depends and will be in a large number of
 packages, I think it is important that it be very stable.

Ok, makes sense.

  I would only be in favor of that if we modify dpkg to auto-generate that
  dependency, unfortunately right now we only know how to compress when
  dpkg-deb -b -Zsomething is called and this is after dpkg-gencontrol
  obviously.
 
 How about if we arrange to pass -Zsomething to dpkg-gencontrol, and
 have a table in dpkg-dev to map something to
 deb-decompressor-something for values where it is needed ?

The problematic part is arrange to pass -Zsomething to
dpkg-gencontrol ... the -Z of dpkg-buildpackage is for dpkg-source and
not dpkg-deb -b. And even if we knew right from the start what compressor
to use, the dpkg-gencontrol call is hidden in debian/rules and we have no
way to pass anything (except maybe an environment variable which isn't
really nice).

Cheers,
-- 
Raphaël Hertzog

Le best-seller français mis à jour pour Debian Etch :
http://www.ouaza.com/livre/admin-debian/





Bug#456332: dpkg could use an elevated pre-depends or depends on lzma

2008-01-24 Thread Ian Jackson
Raphael Hertzog writes (Re: Bug#456332: dpkg could use an elevated pre-depends 
or depends  on lzma):
 On Thu, 24 Jan 2008, Ian Jackson wrote:
  How about if we arrange to pass -Zsomething to dpkg-gencontrol, and
  have a table in dpkg-dev to map something to
  deb-decompressor-something for values where it is needed ?
 
 The problematic part is arrange to pass -Zsomething to
 dpkg-gencontrol ... the -Z of dpkg-buildpackage is for dpkg-source and
 not dpkg-deb -b. And even if we knew right from the start what compressor
 to use, the dpkg-gencontrol call is hidden in debian/rules and we have no
 way to pass anything (except maybe an environment variable which isn't
 really nice).

Yes, I obviously hadn't thought about this too deeply.

How do we expect to choose which packages will use which compressors ?
Ideally in the end the maintainer would decide, so it ought to be in
the source package somewhere.  But we want to override it too.

I think for overriding it, an environment variable is fine.  Perhaps
we could just tell maintainers to put
   export DPKG_DEB_COMPRESSOR ?= ...
in their rules.  Hrm, that doesn't work so well if different packages
want different compression schemes.  A debian/control file field ?

Ian.




-- 
To UNSUBSCRIBE, email to [EMAIL PROTECTED]
with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]



Bug#456332: dpkg could use an elevated pre-depends or depends on lzma

2008-01-24 Thread Raphael Hertzog
On Thu, 24 Jan 2008, Ian Jackson wrote:
 How do we expect to choose which packages will use which compressors ?

Right now, maintainers choose that by adding the right -Z option in the
dh_builddeb call (which forwards the option to dpkg-deb -b). So they put
it directly in the rules file.

 Ideally in the end the maintainer would decide, so it ought to be in
 the source package somewhere.  But we want to override it too.
 
 I think for overriding it, an environment variable is fine.  Perhaps
 we could just tell maintainers to put
export DPKG_DEB_COMPRESSOR ?= ...
 in their rules.  Hrm, that doesn't work so well if different packages
 want different compression schemes.  A debian/control file field ?

The debian/control field is the only viable option IMO. It would be
somewhat similar to the Package-Type: header which has no real use except
influencing the behaviour of another tool during the build.

Cheers,
-- 
Raphaël Hertzog

Le best-seller français mis à jour pour Debian Etch :
http://www.ouaza.com/livre/admin-debian/





Bug#456332: dpkg could use an elevated pre-depends or depends on lzma

2008-01-22 Thread Ian Jackson
Raphael Hertzog writes (Bug#456332: dpkg could use an elevated pre-depends or 
depends on lzma):
 On Tue, 18 Dec 2007, Ian Jackson wrote:
  IMO the lzma binary package should Provide a new virtual package name,
  lzma-deb-support or some such.  Packages could Pre-Depend on that.
 
 What does it bring?

I assume you mean why do it like that.  The answer is that you don't
need the lzma support in cases where it's not needed.  The
dependencies are more accurate.

Also, see my previous comments: that dpkg should execute decompressors
as subprocesses (the way it used to before doogie changed it).

 Decompression also takes more memory than bzip2 but with 32M for
 the bigger packages, I think it's okay.

I have machines still in service with as little as 20Mby of RAM.
Obviously they don't have the biggest packages but such memory use
during decompression can be a problem.

Ian.




-- 
To UNSUBSCRIBE, email to [EMAIL PROTECTED]
with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]



Bug#456332: dpkg could use an elevated pre-depends or depends on lzma

2008-01-22 Thread Raphael Hertzog
On Tue, 22 Jan 2008, Ian Jackson wrote:
 Raphael Hertzog writes (Bug#456332: dpkg could use an elevated pre-depends 
 or depends on lzma):
  On Tue, 18 Dec 2007, Ian Jackson wrote:
   IMO the lzma binary package should Provide a new virtual package name,
   lzma-deb-support or some such.  Packages could Pre-Depend on that.
  
  What does it bring?
 
 I assume you mean why do it like that.  The answer is that you don't
 need the lzma support in cases where it's not needed.  The
 dependencies are more accurate.

You don't need a virtual package for that. You just need to have each
package depend on the right decompressor.

I would only be in favor of that if we modify dpkg to auto-generate that
dependency, unfortunately right now we only know how to compress when
dpkg-deb -b -Zsomething is called and this is after dpkg-gencontrol
obviously.

Cheers,
-- 
Raphaël Hertzog

Le best-seller français mis à jour pour Debian Etch :
http://www.ouaza.com/livre/admin-debian/





Bug#456332: dpkg could use an elevated pre-depends or depends on lzma

2007-12-18 Thread Ian Jackson
Chris Cheney writes (Bug#456332: dpkg could use an elevated pre-depends or 
depends on lzma):
 dpkg supports using lzma for compression of debs however it only
 suggests the lzma binary package which is what is _currently_ used for
 decompressing the debs. In the future it would be preferable if a
 liblzma package existed that dpkg could use similarly to how it uses
 libbz2. IMHO packages should not pre-depend on the lzma binary package
 since it is an implementation level detail of how dpkg supports lzma
 binaries currently. So dpkg needs to pre-depends/depends on the lzma
 binary package so that packages can use lzma compression. However
 packages using lzma compression will need a pre-depends on the correct
 version of dpkg (the one with the depends/pre-depends on lzma).

IMO the lzma binary package should Provide a new virtual package name,
lzma-deb-support or some such.  Packages could Pre-Depend on that.

(How valuable is lzma compression?)

I disagree that dpkg ought to support lzma the same way it supports
bzip2.  Rather I think it ought to support gzip and bzip2 the way it
supports lzma: by calling an external program, as I explained in my
mail [EMAIL PROTECTED]
to debian-devel on Tue, 16 Oct 2007 14:34:36 +0100.

Ian.




-- 
To UNSUBSCRIBE, email to [EMAIL PROTECTED]
with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]



Bug#456332: dpkg could use an elevated pre-depends or depends on lzma

2007-12-14 Thread Chris Cheney
Package: dpkg

dpkg supports using lzma for compression of debs however it only
suggests the lzma binary package which is what is _currently_ used for
decompressing the debs. In the future it would be preferable if a
liblzma package existed that dpkg could use similarly to how it uses
libbz2. IMHO packages should not pre-depend on the lzma binary package
since it is an implementation level detail of how dpkg supports lzma
binaries currently. So dpkg needs to pre-depends/depends on the lzma
binary package so that packages can use lzma compression. However
packages using lzma compression will need a pre-depends on the correct
version of dpkg (the one with the depends/pre-depends on lzma).

FWIW - Ubuntu already does the above so that it can have lzma compressed
debs.

There is more information about what Ubuntu has done and is planning on
doing and how much lzma helps at:

https://wiki.ubuntu.com/dpkg-lzma

There is also a patch in Ubuntu for apt-utils apt-ftparchive to support
lzma compressed debs that I wrote that someone may want to pull into
Debian. The above page has the patch along with including support for
Packages files being compressed with lzma except for one
pkgAcqIndex::Failed function that needs to be corrected. If the Packages
file support isn't wanted then it will need to be split out of the diff.

Thanks,

Chris Cheney




-- 
To UNSUBSCRIBE, email to [EMAIL PROTECTED]
with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]