Re: Proposal: convert all udebs to xz compression
On Sun, Nov 20, 2011 at 08:50:20AM -0500, John D. Hendrickson and Sara Darnell wrote: How is alteration from .gz to .xz going to do anything but create new upgrade or boot issues? what bug number is it? Could you please mind your own business instead of spamming people? udebs are the realm of the installer team and we're not interested in keeping any compatibility where it doesn't make sense. Thanks Philipp Kern -- .''`. Philipp KernDebian Developer : :' : http://philkern.de Stable Release Manager `. `' xmpp:p...@0x539.de Wanna-Build Admin `-finger pkern/k...@db.debian.org signature.asc Description: Digital signature
Re: Proposal: convert all udebs to xz compression
How is alteration from .gz to .xz going to do anything but create new upgrade or boot issues? what bug number is it? - Is lzh/ma xz built already built-in (to kernel or glibs)? What if kernel or libs changes or isn't yet installed? Use bin? Won't this mean all software and scripts that look into udebs will need to be re-written? Also that udebs will no longer be compatible with dpkg as they were? Just wondering what the upgrade is? --- Bob the Toucan — Copyright © 2005, 2006 Ville Koskinen ~ liblzma doesn't include any file I/O functions. A separate I/O library is planned, which would abstract handling of .gz Below is an incomplete and somewhat vague list of operating systems most linux XZ Utils is free general-purpose data compression software with high compression ratio. XZ Utils also work on some not-so-POSIX systems. XZ Utils are the successor ~ * liblzma is a compression library with API similar to that of zlib. * xz is a command line tool with syntax similar to that of gzip. isn't that a 360 while not yet finished ? Guillem Jover wrote: On Sat, 2011-11-19 at 17:32:49 -0200, Otavio Salvador wrote: On Sat, Nov 19, 2011 at 17:21, Philipp Kern pk...@debian.org wrote: We could start with -z0 of course, which would be supported by dpkg-dev and hence a trivial patch of debhelper would do. But it doesn't gain that much. If you mean dpkg-deb, then as mentioned on the bug report -z0 will switch the compressor to none. The closer thing you can get right now is -z1. Dpkg people, can you comment on that? I've pinged people on IRC already about this bug earlier today. I'm not on IRC anymore. In any case I've been making up my mind over this and playing with some possible options and some draft code, I'll reply to the bug report when I have something more concrete. regards, guillem -- To UNSUBSCRIBE, email to debian-boot-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/4ec9059c.6090...@cox.net
Re: Proposal: convert all udebs to xz compression
On Wed, Nov 2, 2011 at 23:14, Otavio Salvador ota...@ossystems.com.br wrote: On Sun, Oct 23, 2011 at 13:26, Philipp Kern pk...@debian.org wrote: ... So my proposal is to switch the udeb compression default in dpkg to xz for wheezy, when the busybox and udpkg changes have landed. Then most udebs will get a translation upload anyway, if not they can be binNMUed to pick up the right compression. ... From my side you have a go ahead but I'd like to hear from Colin and Joey if they can think about any con about doing it. Philipp, What is the current status of it? -- Otavio Salvador O.S. Systems E-mail: ota...@ossystems.com.br http://www.ossystems.com.br Mobile: +55 53 9981-7854 http://projetos.ossystems.com.br -- To UNSUBSCRIBE, email to debian-boot-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/cap9odkp_1cxyzsyj86fqebqguwynu9jushzrjopx3wr1fh5...@mail.gmail.com
Re: Proposal: convert all udebs to xz compression
Otavio, am Sat, Nov 19, 2011 at 04:55:12PM -0200 hast du folgendes geschrieben: On Wed, Nov 2, 2011 at 23:14, Otavio Salvador ota...@ossystems.com.br wrote: On Sun, Oct 23, 2011 at 13:26, Philipp Kern pk...@debian.org wrote: ... So my proposal is to switch the udeb compression default in dpkg to xz for wheezy, when the busybox and udpkg changes have landed. Then most udebs will get a translation upload anyway, if not they can be binNMUed to pick up the right compression. ... From my side you have a go ahead but I'd like to hear from Colin and Joey if they can think about any con about doing it. What is the current status of it? blocked on inconclusive feedback of the dpkg devs on using the extreme variant of xz (see #647915). We could start with -z0 of course, which would be supported by dpkg-dev and hence a trivial patch of debhelper would do. But it doesn't gain that much. In other news I didn't yet upload the udpkg change because I waited for a busybox upload to incorporate the already committed unxz inclusion. Kind regards, Philipp Kern -- .''`. Philipp KernDebian Developer : :' : http://philkern.de Stable Release Manager `. `' xmpp:p...@0x539.de Wanna-Build Admin `-finger pkern/k...@db.debian.org signature.asc Description: Digital signature
Re: Proposal: convert all udebs to xz compression
On Sat, Nov 19, 2011 at 17:21, Philipp Kern pk...@debian.org wrote: Otavio, am Sat, Nov 19, 2011 at 04:55:12PM -0200 hast du folgendes geschrieben: On Wed, Nov 2, 2011 at 23:14, Otavio Salvador ota...@ossystems.com.br wrote: On Sun, Oct 23, 2011 at 13:26, Philipp Kern pk...@debian.org wrote: ... So my proposal is to switch the udeb compression default in dpkg to xz for wheezy, when the busybox and udpkg changes have landed. Then most udebs will get a translation upload anyway, if not they can be binNMUed to pick up the right compression. ... From my side you have a go ahead but I'd like to hear from Colin and Joey if they can think about any con about doing it. What is the current status of it? blocked on inconclusive feedback of the dpkg devs on using the extreme variant of xz (see #647915). We could start with -z0 of course, which would be supported by dpkg-dev and hence a trivial patch of debhelper would do. But it doesn't gain that much. Dpkg people, can you comment on that? I've pinged people on IRC already about this bug earlier today. In other news I didn't yet upload the udpkg change because I waited for a busybox upload to incorporate the already committed unxz inclusion. This has already been done. Please go ahead with udpkg then :-) -- Otavio Salvador O.S. Systems E-mail: ota...@ossystems.com.br http://www.ossystems.com.br Mobile: +55 53 9981-7854 http://projetos.ossystems.com.br -- To UNSUBSCRIBE, email to debian-boot-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/cap9odkr2m9qyvrapbsvsnzsl3zcz3wl0vw68_pscpjayqq8...@mail.gmail.com
Re: Proposal: convert all udebs to xz compression
On Sat, 2011-11-19 at 17:32:49 -0200, Otavio Salvador wrote: On Sat, Nov 19, 2011 at 17:21, Philipp Kern pk...@debian.org wrote: We could start with -z0 of course, which would be supported by dpkg-dev and hence a trivial patch of debhelper would do. But it doesn't gain that much. If you mean dpkg-deb, then as mentioned on the bug report -z0 will switch the compressor to none. The closer thing you can get right now is -z1. Dpkg people, can you comment on that? I've pinged people on IRC already about this bug earlier today. I'm not on IRC anymore. In any case I've been making up my mind over this and playing with some possible options and some draft code, I'll reply to the bug report when I have something more concrete. regards, guillem -- To UNSUBSCRIBE, email to debian-boot-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/2020011931.ga21...@gaara.hadrons.org
Re: Proposal: convert all udebs to xz compression
Philipp Kern wrote: True. OTOH udebs are a very restricted environment. Should I look out for additional dots? That might break compression formats with multiple components. ;-) Why not loop until a known format or EOF is reached? -- see shy jo signature.asc Description: Digital signature
Re: Proposal: convert all udebs to xz compression
Otavio Salvador wrote:
From my side you have a go ahead but I'd like to hear from Colin and
Joey if they can think about any con about doing it.
No objection to the concept.
Code review:
+ while (fgets(buf, sizeof(buf), infp)) {
+ if (strncmp(buf, data.tar., 9) == 0) {
+ compression_type = buf + 9;
+ break;
+ }
+ }
There is no point in looping here, because if the ar -t output is somehow
longer than sizeof(buf), the data.tar.* output could be split across buffers
and so you've already lost. Anyway, with a buffer of 8192, that won't
happen unless the deb format changes in a very unexpected way.
FWIW, this code would fail if the dpkg format got a new member named something
like data.tar.gz.sign.
+ puts(buf);
leftover debugging?
--
see shy jo
signature.asc
Description: Digital signature
Re: Proposal: convert all udebs to xz compression
Hi,
On Sat, Nov 05, 2011 at 11:17:05AM -0400, Joey Hess wrote:
Code review:
+ while (fgets(buf, sizeof(buf), infp)) {
+ if (strncmp(buf, data.tar., 9) == 0) {
+ compression_type = buf + 9;
+ break;
+ }
+ }
There is no point in looping here, because if the ar -t output is somehow
longer than sizeof(buf), the data.tar.* output could be split across buffers
and so you've already lost. Anyway, with a buffer of 8192, that won't
happen unless the deb format changes in a very unexpected way.
it's looping over the lines, as fgets stops at newlines.
FWIW, this code would fail if the dpkg format got a new member named something
like data.tar.gz.sign.
True. OTOH udebs are a very restricted environment. Should I look out for
additional dots? That might break compression formats with multiple
components. ;-)
+ puts(buf);
leftover debugging?
Yep, thanks for spotting it.
Kind regards,
Philipp Kern
--
.''`. Philipp KernDebian Developer
: :' : http://philkern.de Stable Release Manager
`. `' xmpp:p...@0x539.de Wanna-Build Admin
`-finger pkern/k...@db.debian.org
signature.asc
Description: Digital signature
Re: Proposal: convert all udebs to xz compression
On Sun, Oct 23, 2011 at 13:26, Philipp Kern pk...@debian.org wrote: ... So my proposal is to switch the udeb compression default in dpkg to xz for wheezy, when the busybox and udpkg changes have landed. Then most udebs will get a translation upload anyway, if not they can be binNMUed to pick up the right compression. ... From your detailed analisys it does seem it is safe to move this way. The increase in busybox size is quite small and it seems to be the right time to do that change as we'll have enough time to have feedback about it. From my side you have a go ahead but I'd like to hear from Colin and Joey if they can think about any con about doing it. -- Otavio Salvador O.S. Systems E-mail: ota...@ossystems.com.br http://www.ossystems.com.br Mobile: +55 53 9981-7854 http://projetos.ossystems.com.br -- To UNSUBSCRIBE, email to debian-boot-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/cap9odkpetpvmrrxwcqyycsh9glfjwtkfi3gc4yep4_oysdb...@mail.gmail.com
