Re: Using a better compression than .gz for one's CPAN modules

2010-11-28 Thread Aristotle Pagaltzis
* Shlomi Fish shlo...@iglu.org.il [2010-11-26 22:05]: In any case, regardless of its age, xz does tend to compress better than bz2 and should also be faster. I know. I heard of it quite early and switched from bzip2 to xz for my database dumps and mail archives. That’s not the point of the

Re: Using a better compression than .gz for one's CPAN modules

2010-11-28 Thread David Golden
On Sun, Nov 28, 2010 at 4:22 PM, Aristotle Pagaltzis pagalt...@gmx.de wrote: I agree with the notion. But let me ask how much pressure changing the compression format on CPAN would exert on the world to adapt itself to it. Note too the quote is written from the perspective of the world: no

Re: Using a better compression than .gz for one's CPAN modules

2010-11-28 Thread Aristotle Pagaltzis
* David Golden xda...@gmail.com [2010-11-28 22:45]: On the other hand, if someone wants to recompress all of CPAN into XYZ compression scheme and release their own client that deals with it (or patch cpanm or whatever), then they can have the benefits (and any resulting hassles) themselves.

Re: Using a better compression than .gz for one's CPAN modules

2010-11-26 Thread Aristotle Pagaltzis
* Shlomi Fish shlo...@iglu.org.il [2010-11-24 21:05]: Welcome to 2010. There are two kinds of fool. One says, “This is old, and therefore good.” And one says, “This is new, and therefore better.” —John Brunner Regards, -- Aristotle Pagaltzis

Re: Using a better compression than .gz for one's CPAN modules

2010-11-26 Thread David Golden
On Fri, Nov 26, 2010 at 3:59 PM, Shlomi Fish shlo...@iglu.org.il wrote:     There are two kinds of fool. One says,     “This is old, and therefore good.” And one says,     “This is new, and therefore better.” That put aside sticking with an older solution may be preferable due to the better

Re: Using a better compression than .gz for one's CPAN modules

2010-11-25 Thread David Cantrell
On Wed, Nov 24, 2010 at 09:59:59PM +0200, Shlomi Fish wrote: On Friday 19 November 2010 22:02:48 David Cantrell wrote: Even if it does, there's not much point. bzip2 support is nowhere near universal, and preventing lots of users from using your code would seem to be a poor trade-off for

Re: Using a better compression than .gz for one's CPAN modules

2010-11-22 Thread Andreas J. Koenig
On Sat, 20 Nov 2010 23:22:52 +0100, Aristotle Pagaltzis pagalt...@gmx.de said: It’s gonna be a lot of work to iron out the entire tool chain to support the newer formats; then it will take a lot of time until the work trickles out far enough that people could start relying on it.

Re: Using a better compression than .gz for one's CPAN modules

2010-11-22 Thread Aristotle Pagaltzis
* Andreas J. Koenig andreas.koenig.7os6v...@franz.ak.mind.de [2010-11-22 09:20]: Agreed, but since bzip2 support is already done we can welcome it when people actually use it. I am unwilling to encourage it but I won’t argue if someone wants to use it. And it can be a win for distributions

Reducing rsync cost (was: Re: Using a better compression than .gz for one's CPAN modules)

2010-11-22 Thread David Landgren
On 19/11/2010 20:57, dhu...@hudes.org wrote: source code, even 100KLOC? Once you go to .gz you're already at better than 2:1. What are you going to save by going to even 3:1, 10Kbytes? compared to the nuisance inflicted, it's nothing. Over the entire CPAN archive, it'd be significant... I

Re: Reducing rsync cost (was: Re: Using a better compression than .gz for one's CPAN modules)

2010-11-22 Thread David Nicol
On Mon, Nov 22, 2010 at 4:37 AM, David Landgren da...@landgren.net wrote: Yeah, this is the killer. In an ideal world, we would kill the symlinks such as authors/id/*, modules/by-category/*, modules/by-module/* and so on. These could be recreated via shell scripts locally on mirrors for people

Re: Using a better compression than .gz for one's CPAN modules

2010-11-20 Thread Aristotle Pagaltzis
* Shlomi Fish shlo...@gmail.com [2010-11-19 19:55]: here is a report on compressing Graph-Easy-0.70.tar with various compression methods: {{{ shlomif:~/progs/perl/cpan/Graph/Easy/trunk/Graph-Easy/TEMP$ ls -l total 3420 -rw-r--r-- 1 shlomif shlomif 2160640 Nov 14 22:20 Graph-Easy-0.70.tar

Re: Using a better compression than .gz for one's CPAN modules

2010-11-20 Thread Dana Hudes
While I completely agree with Aristotle I wish to clarify that Solaris 10 and 11 ship with bzip2. I can't recall about Solaris 9 and I am recalling this was not the case with 8 and earlier. Sent from my BlackBerry® smartphone with Nextel Direct Connect

Re: Using a better compression than .gz for one's CPAN modules

2010-11-19 Thread David Golden
On Fri, Nov 19, 2010 at 1:53 PM, Shlomi Fish shlo...@gmail.com wrote: 1. Will the CPAN testing and downloading toolchian will handle modules uploaded as .tar.bz2?  (Allow to install them, unpack them, etc.)  How about tar.xz. .bz2, yes. .xz, possibly, but not reliably. CPANPLUS uses

Re: Using a better compression than .gz for one's CPAN modules

2010-11-19 Thread dhudes
The savings for going to .bz2 over .gz for source code are fairly insignificant. We're talking about source code for a perl module. Is your stuff tens of megabytes in size? That's a lot of code if so. I could understand if you were distributing a sizable database with your code but source code,

Re: Using a better compression than .gz for one's CPAN modules

2010-11-19 Thread Daniel Staal
On Fri, November 19, 2010 2:18 pm, dhu...@hudes.org wrote: The savings for going to .bz2 over .gz for source code are fairly insignificant. We're talking about source code for a perl module. Is your stuff tens of megabytes in size? That's a lot of code if so. I could understand if you were

Re: Using a better compression than .gz for one's CPAN modules

2010-11-19 Thread dhudes
source code, even 100KLOC? Once you go to .gz you're already at better than 2:1. What are you going to save by going to even 3:1, 10Kbytes? compared to the nuisance inflicted, it's nothing. Over the entire CPAN archive, it'd be significant... I agree on the individual case it's probably not

Re: Using a better compression than .gz for one's CPAN modules

2010-11-19 Thread David Cantrell
On Fri, Nov 19, 2010 at 08:53:12PM +0200, Shlomi Fish wrote: here is a report on compressing Graph-Easy-0.70.tar with various compression methods: -rw-r--r-- 1 shlomif shlomif 416916 Nov 14 22:23 Graph-Easy-0.70.tar.gz -rw-r--r-- 1 shlomif shlomif 329197 Nov 5 12:24

Re: Using a better compression than .gz for one's CPAN modules

2010-11-19 Thread Curtis Jewell
On Fri, 19 Nov 2010 11:57 -0800, dhu...@hudes.org wrote: Disk space is cheap. Bandwidth is cheap. What's rough is the rsync between mirrors. Compressing to .bz2 won't help that: the stress is doing a stat on every single file in CPAN not the transfer. Work toward optimizing the mirror

Re: Using a better compression than .gz for one's CPAN modules

2010-11-19 Thread Daniel Staal
On Fri, November 19, 2010 2:57 pm, dhu...@hudes.org wrote: Disk space is cheap. Bandwidth is cheap. What's rough is the rsync between mirrors. Compressing to .bz2 won't help that: the stress is doing a stat on every single file in CPAN not the transfer. Work toward optimizing the mirror