Re: CD sizes again (and BoF reminder!)

2012-07-30 Thread Jakub Wilk
* Adam Borowski kilob...@angband.pl, 2012-07-22, 23:51: having an option to disable fsync in dpkg without unreliable LD_PRELOAD tricks would be great. http://bugs.debian.org/613428 -- Jakub Wilk -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe.

xz (Re: CD sizes again (and BoF reminder!)

2012-07-27 Thread Hideki Yamane
Hi, On Sat, 21 Jul 2012 19:13:45 -0400 Joey Hess jo...@debian.org wrote: Which is why I asked for actual, real-world benchmarks... As I said in my presentation, I just tested a few case, fonts-horai-umefont, poppler-data and openclipart-png. Install fonts- and poppler-data is almost same

Re: CD sizes again (and BoF reminder!)

2012-07-22 Thread Uoti Urpala
Simon Paillard wrote: On Sat, Jul 21, 2012 at 09:25:50PM +0300, Uoti Urpala wrote: So at least in this case the biggest performance problem by far is the inappropriate use of fsync() or other disk synchronization primitives, and CPU use for unpacking is pretty much irrelevant. Though the

Re: CD sizes again (and BoF reminder!)

2012-07-22 Thread Philipp Kern
On Sun, Jul 22, 2012 at 01:58:59AM +0200, Adam Borowski wrote: BTW, when we switched to building udebx with xz, Philipp Kern benchmarked it using little or no additional CPU to decompress xz produced with -Zxz -z1 -Sextreme http://lists.debian.org/debian-boot/2011/10/msg00247.html Per the

Re: CD sizes again (and BoF reminder!)

2012-07-22 Thread Adam Borowski
On Sun, Jul 22, 2012 at 07:58:42PM +0300, Uoti Urpala wrote: Simon Paillard wrote: , I understand debian-installer ask dpkg not to fsync: - Run dpkg with --force-unsafe-io during installation; syncing is This only affects one particular instance of syncing (which I think may be

Re: CD sizes again (and BoF reminder!)

2012-07-22 Thread Russell Coker
On Mon, 23 Jul 2012, Adam Borowski kilob...@angband.pl wrote: You tested ext4. On btrfs, dpkg is around an order of magnitude slower, making using it without eatmydata a laughable idea. And that's on a filesystem whose features include: * transactions (so all dpkg processing could be done

Re: CD sizes again (and BoF reminder!)

2012-07-21 Thread Joey Hess
Hideki Yamane wrote: On Sun, 8 Jul 2012 17:58:16 +0200 Adam Borowski kilob...@angband.pl wrote: • xz -6 (the default) is a lot slower when compressing, fast when decompressing, needs only 10MB memory, 58% size • xz -9 has very slow compression, takes gobs of memory, 56% size

Re: CD sizes again (and BoF reminder!)

2012-07-21 Thread Mike Hommey
On Sat, Jul 21, 2012 at 11:42:03AM -0400, Joey Hess wrote: Hideki Yamane wrote: On Sun, 8 Jul 2012 17:58:16 +0200 Adam Borowski kilob...@angband.pl wrote: • xz -6 (the default) is a lot slower when compressing, fast when decompressing, needs only 10MB memory, 58% size • xz -9

Re: CD sizes again (and BoF reminder!)

2012-07-21 Thread Uoti Urpala
Joey Hess wrote: Hideki Yamane wrote: I tested as well, and sometimes decompression with xz is so slw, it takes 6-8 times than default gz. I was just watching your DebConf presentation Lets shrink Debian package archive and I think there you said decompression with xz was between

Re: CD sizes again (and BoF reminder!)

2012-07-21 Thread brian m. carlson
On Sat, Jul 21, 2012 at 09:25:50PM +0300, Uoti Urpala wrote: Most of the time taken by cdebootstrap is wasted by dpkg on doing useless file syncs: cdebootstrap --arch=amd64 unstable debian-tree/ from local package cache on ext4: 138 seconds on tmpfs where dpkg can't waste time on

Re: CD sizes again (and BoF reminder!)

2012-07-21 Thread Uoti Urpala
brian m. carlson wrote: On Sat, Jul 21, 2012 at 09:25:50PM +0300, Uoti Urpala wrote: So at least in this case the biggest performance problem by far is the inappropriate use of fsync() or other disk synchronization primitives, and CPU use for unpacking is pretty much irrelevant. My

Re: CD sizes again (and BoF reminder!)

2012-07-21 Thread Simon Paillard
Hi, On Sat, Jul 21, 2012 at 09:25:50PM +0300, Uoti Urpala wrote: Joey Hess wrote: Hideki Yamane wrote: I tested as well, and sometimes decompression with xz is so slw, it takes 6-8 times than default gz. I was just watching your DebConf presentation Lets shrink Debian

Re: CD sizes again (and BoF reminder!)

2012-07-21 Thread Joey Hess
Mike Hommey wrote: Note that slower decompression doesn't necessarily mean longer installation time. I/O is still more time consuming than CPU. Which is why I asked for actual, real-world benchmarks... -- see shy jo signature.asc Description: Digital signature

Re: CD sizes again (and BoF reminder!)

2012-07-21 Thread Adam Borowski
On Sat, Jul 21, 2012 at 11:42:03AM -0400, Joey Hess wrote: Hideki Yamane wrote: On Sun, 8 Jul 2012 17:58:16 +0200 Adam Borowski kilob...@angband.pl wrote: • xz -6 (the default) is a lot slower when compressing, fast when decompressing, needs only 10MB memory, 58% size • xz -9

Re: CD sizes again (and BoF reminder!)

2012-07-09 Thread Ivan Shmakov
Ben Hutchings b...@decadent.org.uk writes: [...] - twm: no-one should have to suffer this And, exactly, why not? Before I've switched to Openbox, it was one of the two WM's I've used, along with FVWM. And they say [1] that it still can be handy at times.

Re: CD sizes again (and BoF reminder!)

2012-07-09 Thread Charles Plessy
Le Sun, Jul 08, 2012 at 02:12:44PM -0600, Joey Hess a écrit : grub-legacy is still used for multipath and sataraid. Something was going to be done to make grub2 support those, but I don't know the status. Hi, Grub-legacy is also useful for booting virtual machine images with pv-grub (such

Re: CD sizes again (and BoF reminder!)

2012-07-09 Thread Neil McGovern
On Sat, Jul 07, 2012 at 04:22:58PM -0600, Stefano Zacchiroli wrote: Ansgar has been experimenting with .deb sizes to make the packages needed for a minimal desktop installation fit in the first CD. It looks like that's doable by switching to xz compression for the involved binaries. Would

Re: CD sizes again (and BoF reminder!)

2012-07-09 Thread Hideki Yamane
Hi, On Sun, 8 Jul 2012 17:58:16 +0200 Adam Borowski kilob...@angband.pl wrote: • xz -6 (the default) is a lot slower when compressing, fast when decompressing, needs only 10MB memory, 58% size • xz -9 has very slow compression, takes gobs of memory, 56% size (Obviously, the size

Re: CD sizes again (and BoF reminder!)

2012-07-08 Thread Adam Borowski
On Sat, Jul 07, 2012 at 04:22:58PM -0600, Stefano Zacchiroli wrote: On Sun, Jul 08, 2012 at 12:03:44AM +0200, Cyril Brulebois wrote: Ansgar Burchardt ans...@debian.org (07/07/2012): Another question is if the release team would consider unblocking updated packages (with just the switch to

Re: CD sizes again (and BoF reminder!)

2012-07-08 Thread Adam Borowski
On Sat, Jul 07, 2012 at 08:42:07PM +0100, Ben Hutchings wrote: On Sat, 2012-07-07 at 16:14 +0200, Bastian Blank wrote: - ftp, telnet: mostly redundant with wget and nc, unless you really like cleartext authentication I don't get why anyone would talk about authentication in the context of ftp

Re: CD sizes again (and BoF reminder!)

2012-07-08 Thread Hideki Yamane
Hi, On Sat, 7 Jul 2012 04:47:35 +0100 Steve McIntyre st...@einval.com wrote: So, yes - looks like xz will make a difference here for the wheezy release, for amd64 at least. It's enough that we'd probably even have space for the installation manual and release notes to fit \o/. BTW, I'll talk

Re: CD sizes again (and BoF reminder!)

2012-07-08 Thread Wouter Verhelst
On Sun, Jul 08, 2012 at 01:17:32AM +0200, Bastian Blank wrote: On Sat, Jul 07, 2012 at 04:36:34PM -0600, Joey Hess wrote: [...] - grub-legacy These are all installed by d-i in various situations. Even grub-legacy? Yes; d-i in expert mode still has the ability to explicitly choose for

Re: CD sizes again (and BoF reminder!)

2012-07-08 Thread Ansgar Burchardt
Hi Stefano Zacchiroli z...@debian.org writes: Ansgar has been experimenting with .deb sizes to make the packages needed for a minimal desktop installation fit in the first CD. It looks like that's doable by switching to xz compression for the involved binaries. Would you grant freeze

Re: CD sizes again (and BoF reminder!)

2012-07-08 Thread Cyril Brulebois
Wouter Verhelst wou...@debian.org (08/07/2012): On Sun, Jul 08, 2012 at 01:17:32AM +0200, Bastian Blank wrote: Even grub-legacy? Yes; d-i in expert mode still has the ability to explicitly choose for grub legacy, if you really want to. It looks to me like a current debian-installer build

Re: CD sizes again (and BoF reminder!)

2012-07-08 Thread Joey Hess
Cyril Brulebois wrote: It looks to me like a current debian-installer build installs grub2, with no option for grub-legacy, even in expert mode. grub-legacy is still used for multipath and sataraid. Something was going to be done to make grub2 support those, but I don't know the status. --

Re: CD sizes again (and BoF reminder!)

2012-07-08 Thread Jonathan McDowell
On Sun, Jul 08, 2012 at 02:12:44PM -0600, Joey Hess wrote: Cyril Brulebois wrote: It looks to me like a current debian-installer build installs grub2, with no option for grub-legacy, even in expert mode. grub-legacy is still used for multipath and sataraid. Something was going to be done

Re: CD sizes again (and BoF reminder!)

2012-07-07 Thread Ansgar Burchardt
Steve McIntyre st...@einval.com writes: On Fri, Jul 06, 2012 at 09:00:32PM -0600, Ansgar Burchardt wrote: I tried recompressing all packages in wheezy with xz. The total size for all amd64+all packages was reduced from 42GB to 33 GB (about 20%). A per-package listing is available from [1] [1]

Re: CD sizes again (and BoF reminder!)

2012-07-07 Thread Bastian Blank
On Fri, Jul 06, 2012 at 10:10:04PM +0100, Steve McIntyre wrote: http://cdimage.debian.org/cdimage/tmp/new-tasks/gnome-cd.list.gz Why does this contain debug packages? Bastian -- I've already got a female to worry about. Her name is the Enterprise. -- Kirk, The Corbomite

Re: CD sizes again (and BoF reminder!)

2012-07-07 Thread Steve McIntyre
On Sat, Jul 07, 2012 at 03:10:15PM +0200, Bastian Blank wrote: On Fri, Jul 06, 2012 at 10:10:04PM +0100, Steve McIntyre wrote: http://cdimage.debian.org/cdimage/tmp/new-tasks/gnome-cd.list.gz Why does this contain debug packages? The list posted there is the full sorted list of *all*

Re: CD sizes again (and BoF reminder!)

2012-07-07 Thread Bastian Blank
On Sat, Jul 07, 2012 at 02:34:44PM +0100, Steve McIntyre wrote: The list posted there is the full sorted list of *all* packages, as applied to the full set of CDs. The last one on CD#1 is gnome-packagekit-data, as I said, and I don't see any debug packages above that in the list. Ups, I did

Re: CD sizes again (and BoF reminder!)

2012-07-07 Thread Ben Hutchings
On Sat, 2012-07-07 at 16:14 +0200, Bastian Blank wrote: On Sat, Jul 07, 2012 at 02:34:44PM +0100, Steve McIntyre wrote: The list posted there is the full sorted list of *all* packages, as applied to the full set of CDs. The last one on CD#1 is gnome-packagekit-data, as I said, and I don't

Re: CD sizes again (and BoF reminder!)

2012-07-07 Thread Cyril Brulebois
Ansgar Burchardt ans...@debian.org (07/07/2012): Another question is if the release team would consider unblocking updated packages (with just the switch to xz for binaries). I think it would be nice to be able to get a useful desktop system using just the first CD, but I'm not sure if they

Re: CD sizes again (and BoF reminder!)

2012-07-07 Thread Stefano Zacchiroli
On Sun, Jul 08, 2012 at 12:03:44AM +0200, Cyril Brulebois wrote: Ansgar Burchardt ans...@debian.org (07/07/2012): Another question is if the release team would consider unblocking updated packages (with just the switch to xz for binaries). I think it would be nice to be able to get a

Re: CD sizes again (and BoF reminder!)

2012-07-07 Thread Joey Hess
- ftp, telnet: mostly redundant with wget and nc, unless you really like cleartext authentication - procmail: server These are priority standard. - pcmciautils: PCMCIA is dead - jfsutils, reiserfsprogs, ufsutils: obscure - discover - openssh-server: server, not desktop - grub-legacy

Re: CD sizes again (and BoF reminder!)

2012-07-07 Thread Bastian Blank
On Sat, Jul 07, 2012 at 04:36:34PM -0600, Joey Hess wrote: - ftp, telnet: mostly redundant with wget and nc, unless you really like cleartext authentication - procmail: server These are priority standard. This is fixeable. - pcmciautils: PCMCIA is dead - jfsutils, reiserfsprogs,

Re: CD sizes again (and BoF reminder!)

2012-07-07 Thread Ben Hutchings
On Sat, 2012-07-07 at 16:36 -0600, Joey Hess wrote: - ftp, telnet: mostly redundant with wget and nc, unless you really like cleartext authentication - procmail: server These are priority standard. Which is not the same as being important to include in a desktop installation. -

Re: CD sizes again (and BoF reminder!)

2012-07-07 Thread Christian PERRIER
Quoting Ben Hutchings (b...@decadent.org.uk): partman-crypto still installs this. Seems like a bug...? The package is orphaned and dm-crypt has support for a compatible encryption mode. Given the level of attention which partman-crypto got during the squeeze-wheezy release, I'd bet this

CD sizes again (and BoF reminder!)

2012-07-06 Thread Steve McIntyre
Hey people, Following up on this again... Back in May I warned about CD sizes[1] for the Wheezy release, pointing out that CD#1 isn't big enough any more to provide usable Gnome or KDE installations. There was some discussion about what to do about that (change compression to xz, switch to the

Re: CD sizes again (and BoF reminder!)

2012-07-06 Thread Timo Juhani Lindfors
Steve McIntyre st...@einval.com writes: Back in May I warned about CD sizes[1] for the Wheezy release, pointing out that CD#1 isn't big enough any more to provide usable Gnome or KDE installations. Indeed. CD1 was really problematic in squeeze too:

Re: CD sizes again (and BoF reminder!)

2012-07-06 Thread Steve McIntyre
As suggested by Ansgar IRL, here's a summary of what compression types are showing up on each CD (by looking at data.tar.$EXT for all the .debs and .udebs): Gnome = The last package on amd64 CD#1 is gnome-packagekit-data. task-desktop fits on CD#1, but task-gnome-desktop is ~110 packages

Re: CD sizes again (and BoF reminder!)

2012-07-06 Thread Ansgar Burchardt
Hi, Steve McIntyre st...@einval.com writes: Back in May I warned about CD sizes[1] for the Wheezy release, pointing out that CD#1 isn't big enough any more to provide usable Gnome or KDE installations. There was some discussion about what to do about that (change compression to xz, switch to

Re: CD sizes again (and BoF reminder!)

2012-07-06 Thread Steve McIntyre
On Fri, Jul 06, 2012 at 09:00:32PM -0600, Ansgar Burchardt wrote: Hi, Steve McIntyre st...@einval.com writes: Back in May I warned about CD sizes[1] for the Wheezy release, pointing out that CD#1 isn't big enough any more to provide usable Gnome or KDE installations. There was some discussion