On Sun, 4 Apr 2010, David Nicol wrote:
so the requirements for the Solution To The Problem Which Solves A
More General Problem Than The Immediate Problem And Will Therefore
Make Whoever Sets It Up A Hero include a replacement for the current
mirroring technology stack that is tailored to mirrori
> It hasn't been done because its outside of the scope of design for rsync.
> It's meant to sync arbitrary filesets in which many, if not all, changes are
> made out of band. It's decidely non-trivial to implement in that mode
> unless you're willing to accept a certain window in which your databa
On Apr 2, 2010, at 1:50, Arthur Corliss wrote:
> And my assertion has been that the excessive stats by the server are a bigger
> impediment to synchronization than the inode count.
Well, then one of us don't understand how file systems etc work. :-)
- ask
On Fri, 2 Apr 2010, Ask Bj?rn Hansen wrote:
On Apr 2, 2010, at 1:50, Arthur Corliss wrote:
And my assertion has been that the excessive stats by the server are a bigger
impediment to synchronization than the inode count.
Well, then one of us don't understand how file systems etc work. :-)
On Apr 1, 2010, at 19:49, Arthur Corliss wrote:
> I've made a viable suggestion, and offered some time to work on it. But
> you've made it abundantly clear that it's not welcome.
Talk = ZzZz.
Code = Interesting.
Deployment = Useful.
- ask
Much of this discussion is beyond my depth but in terms of keeping it
simple, and trying to limit the stat calls on the upstream servers,
what about DNS as a replication model? You could break up the tree at
logical divisions similar to zones and assign them serial numbers
(say a .serial file) and
On Apr 1, 2010, at 19:49, Arthur Corliss wrote:
I can't believe I'm doing this, but ...
>> The main point here is that we can't use 20 inodes per distribution. It's
>> Just Nuts. Sure, it's only something like 400k files/inodes now - but at
>> the rate it's going it'll be a lot more soon en
On Fri, 2 Apr 2010, Ask Bj?rn Hansen wrote:
Talk = ZzZz.
Code = Interesting.
Deployment = Useful.
Please. The talk serves to gauge interest before I waste any time
implementing a solution that's already been rejected out of hand. As I've
mentioned repeatedly I already use rsync, albeit on mu
On Fri, 2 Apr 2010, Ask Bj?rn Hansen wrote:
I can't believe I'm doing this, but ...
:-) All for entertainment's sake...
The main point here is that we can't use 20 inodes per distribution. It's Just
Nuts. Sure, it's only something like 400k files/inodes now - but at the rate
it's going
On Wed, 31 Mar 2010, Ask Bj?rn Hansen wrote:
Everyone who doesn't run mirrors says "oh, who cares - it doesn't bother me".
Some of us who does run mirrors say "actually, that sort of thing is important and
an actual issue.".
Others reply "then you're doing it wrong". But nobody came with
On Thursday 01 April 2010 05:39:27 David Nicol wrote:
> On Wed, Mar 31, 2010 at 7:43 AM, Ask Bjørn Hansen wrote:
> > The main point here is that we can't use 20 inodes per distribution.
>
> so don't. How much reengineering would be needed to keep CPAN in a
> database instead of a file system?
It
On Wed, Mar 31, 2010 at 7:43 AM, Ask Bjørn Hansen wrote:
> The main point here is that we can't use 20 inodes per distribution.
so don't. How much reengineering would be needed to keep CPAN in a
database instead of a file system?
Just to summarize (and this is going to be the last mail I send in this thread):
Old releases (more than a few releases back) are virtually useless. Just how
useless is up for debate, but BACKPAN is there.
We've always encouraged CPAN authors to purge old releases as appropriate.
Tim noticed (
On Mar 31, 2010, at 6:52, David Nicol wrote:
> new proposal: Make modules "pay rent" in order to remain on a mirror.
> Rent could be in the form of actual user interest, or good reviews.
How you are proposing purging useless stuff from CPAN -- that's a lot more
radical than Tim's proposal of j
On Wed, Mar 31, 2010 at 10:45 AM, David Landgren wrote:
> On 31/03/2010 06:52, David Nicol wrote:
>
>> new proposal: Make modules "pay rent" in order to remain on a mirror.
>> Rent could be in the form of actual user interest, or good reviews.
>>
>> Use as a dependency could count as rent.
>
> Put
On 31/03/2010 06:52, David Nicol wrote:
new proposal: Make modules "pay rent" in order to remain on a mirror.
Rent could be in the form of actual user interest, or good reviews.
Use as a dependency could count as rent.
Put a value tag on things and people will game the system to ensure
their
On 30/03/2010 20:33, Arthur Corliss wrote:
On Tue, 30 Mar 2010, Matija Grabnar wrote:
Er, not exactly. Read
http://www.cvsup.org/howsofast.html
I had read http://www.cvsup.org/faq.html#features item #3.
From what I can see, cvsup uses the rsync algorithm on a file-by-file
basis (it uses jus
Arthur Corliss wrote:
You had me excited at first, but then the home page said:
To update non-RCS files, CVSup uses the highly efficient rsync
algorithm,
developed by Andrew Tridgell and Paul Mackerras.
Looks like its speed benefits are due to knowledge of specific file types
(RCS and log
On Mon, 29 Mar 2010, Dana Hudes wrote:
Arthur your ignorance is apalling
Go look at what ORCA does
SAR doesn't give you the info
With ORCA i have any thing from kstat or iostat. It goes into roundrobin
database with rrdtool.
Procallaotr does for linux what
orcallator does for solaris where it
On Wed, Mar 31, 2010 at 01:03:51PM +1100, Adam Kennedy wrote:
> I've said nothing till now, because I figured more noise wouldn't help much.
>
> But I quite like the rsync daemon/proxy idea, and as it so happens I'm
> attending the OzLabs Unconference in 3 weeks time to hang out with
> Tridge, Rus
I've said nothing till now, because I figured more noise wouldn't help much.
But I quite like the rsync daemon/proxy idea, and as it so happens I'm
attending the OzLabs Unconference in 3 weeks time to hang out with
Tridge, Rusty and the other Australia C/Kernel/Samba/RSync elites.
So I'd be happy
toool
--Original Message--
From: Arthur Corliss
To: Dana Hudes
Cc: module-authors@perl.org
Sent: Mar 29, 2010 1:12 PM
Subject: Re: Trimming the CPAN - "Automatic Purging"
On Mon, 29 Mar 2010, Dana Hudes wrote:
> Orcallator, procallator and friends aren't shiny new toys
On Mon, 29 Mar 2010, Dana Hudes wrote:
Orcallator, procallator and friends aren't shiny new toys
Adrian Cockroft wrote initial version of orcallator in the early 90s for his book
"Solaris Performance Tuning. The 2nd edition is I think 1998.
The current version of ORCA (processes the collected d
David Nicol wrote:
On Sun, Mar 28, 2010 at 2:32 PM, Elaine Ashton wrote:
On Mar 28, 2010, at 12:48 PM, Randy Kobes wrote:
Has some sort of disk quota system for CPAN author accounts ever been
considered?
Not specifically, no, at least not that I'm aware of. That would have to be
implemente
Nicholas Clark wrote:
On Tue, Mar 30, 2010 at 10:08:57PM +0200, Rene Schickbauer wrote:
Now, if we where to put all files into mercurial, git or the like,
renaming the files so they don't have version numbers in their names but
storing them sequentially as commits so new versions update old on
On Tue, Mar 30, 2010 at 10:08:57PM +0200, Rene Schickbauer wrote:
> Now, if we where to put all files into mercurial, git or the like,
> renaming the files so they don't have version numbers in their names but
> storing them sequentially as commits so new versions update old ones.
Sort of like
On Sun, Mar 28, 2010 at 2:32 PM, Elaine Ashton wrote:
>
> On Mar 28, 2010, at 12:48 PM, Randy Kobes wrote:
>
>>
>> Has some sort of disk quota system for CPAN author accounts ever been
>> considered?
>
> Not specifically, no, at least not that I'm aware of. That would have to be
> implemented on
On Tue, 30 Mar 2010, Rene Schickbauer wrote:
This could work like any modern, distributed version control systems. That
way, the user would also be able to apply local patches and/or deciding which
changesets to pull in from the main server. Or have a complete, local mirror
and one for the p
Hi!
Now the key, as I see it, is that unlike all the other use cases where
rsync is used, large mirrors are likely to have their directories
directly transfered from another mirror. So, the client that pulled
the tree update down could store a list of changed files, and the
server could then
On Tue, 30 Mar 2010, Matija Grabnar wrote:
Er, not exactly. Read
http://www.cvsup.org/howsofast.html
I had read http://www.cvsup.org/faq.html#features item #3.
From what I can see, cvsup uses the rsync algorithm on a file-by-file basis
(it uses just the differential send part of the rsync
On Tue, 30 Mar 2010, David Landgren wrote:
I believe cvsup (FreeBSD's source distribution mechanism) knows how to avoid
this cost by serialising context between runs.
That may be an avenue worth exploring, since it should be a less risky
proposition for a mirror operator to download a tried a
On Sun, Mar 28, 2010 at 06:04:03PM -0400, David Golden wrote:
> As always with perl, "it depends". They are laid out just as a normal
> CPAN repository, so if you have one in your urllist, something
> specified as author/distribution.tar.gz might well resolve.
Not just "might well resolve". It
On Sun, Mar 28, 2010 at 07:28:48AM -0700, dhu...@hudes.org wrote:
> The danger in a CPAN::Mini and in removing old versions is that one is
> assuming that the latest and greatest is the one to use. This is false.
And this is why I run cp5.6.2an.barnyard.co.uk etc.
It wouldn't be difficult for
On 29/03/2010 09:39, Arthur Corliss wrote:
On Sun, 28 Mar 2010, dhu...@hudes.org wrote:
The entire point of rsync is to send only changes.
Therefore once your mirror initially syncs the old versions of modules is
not the issue. Indeed, removing the old versions would present additional
burden o
I think that Andreas's concept of treating these mirrors as a database is good.
Checkpoint logical log replay is better than a simple rsync for large numbers
of files.
The replication problem for databases is well-understood and open-source code
for it is available from at least Postgresql.
On Sun, 28 Mar 2010, Dana Hudes wrote:
Why is rsync a problem? Where is the bottleneck in the protocol or the code
implementing it?
Specifics!
SAR is antiquated doesn't give the info you really need. Using a linux system?
Use procallator and feed resulting collected data to ORCA. Better yet, u
r 28, 2010 11:45 PM
Subject: Re: Trimming the CPAN - "Automatic Purging"
* Dana Hudes [2010-03-29 04:30]:
> Using http for this is inefficient It makes for slower file
> transfer because you keep rerunning path mtu probes and tcp
> slow start It makes extra socket handles o
Hi Elaine,
Elaine Ashton wrote:
On Mar 28, 2010, at 12:48 PM, Randy Kobes wrote:
Jarkko and I were talking about it this morning - as he's not in
favour of pruning - while trying to think of a way around the size
problem and he reminded me of the idea that, if I recall correctly
was Adreas' sugg
2007 or so
www.orcaware.org i think it was
Sent from my BlackBerry® smartphone with Nextel Direct Connect
-Original Message-
From: Arthur Corliss
Date: Mon, 29 Mar 2010 00:31:50
To: Dana Hudes
Cc:
Subject: Re: Trimming the CPAN - "Automatic Purging"
On Sun, 28 Mar 2010, Dana Hu
ith the list of files to fetch -- via ftp.
.
--Original Message--
From: Aristotle Pagaltzis
To: module-authors@perl.org
Sent: Mar 28, 2010 10:13 PM
Subject: Re: Trimming the CPAN - "Automatic Purging"
* Nicholas Clark [2010-03-28 18:20]:
> I'm missing something here,
On Sun, 28 Mar 2010, Dana Hudes wrote:
I agree with Elaine
I can't get rsync through the firewall at work. Not even tunneled.
For CPAN I use CPAN::Mini. It uses http and it does the job though it does
force the local CPAN to blead. My local solution to other things we need such
as Blastwave (w
On Sun, 28 Mar 2010, Dana Hudes wrote:
Use of wget and http to download an entire site means numerous TCP opens and
HTTP GET requests. The entire point of rsync is that it knows there are
numerous downloads. It does ONE open. This allows TCP slow start to ramp up
That wasn't exactly what I w
On Sun, 28 Mar 2010, Andreas J. Koenig wrote:
Says the author of a module named Paranoid. A lovely coincidence.
:-) As they say, just because you may be paranoid, it doesn't mean that no
one's out to get you.
If you want to study the CPAN "checkpointed logs" solution running on
the very CPAN
On Sun, 28 Mar 2010, Andy Armstrong wrote:
We're nearly there if A == a CPAN::Mini style mirror, B == the current mirror
pruned and C == backpan.
So the actions to make that happen are:
* give the current clients specific support for this
* generate a master mini mirror that other mini mirror
On Sun, 28 Mar 2010, Elaine Ashton wrote:
I do very much like Tim's proposal for giving old modules a push to BackPAN
since, with proper communication of the changes to the authors along with a way
to mark exceptions, this would rid CPAN of a lot of cruft that should be on
BackPan anyway.
I
On Sun, 28 Mar 2010, Nicholas Clark wrote:
Are you running a large public mirror site, where you don't even have
knowledge of who is mirroring from you?
(Not even knowledge, let alone channels of communication with, let alone
control over)
Because (as I see it, not having done any of this) the
On Sun, 28 Mar 2010, dhu...@hudes.org wrote:
The entire point of rsync is to send only changes.
Therefore once your mirror initially syncs the old versions of modules is
not the issue. Indeed, removing the old versions would present additional
burden on synchronization! The ongoing burden is the
* Dana Hudes [2010-03-29 04:30]:
> Using http for this is inefficient It makes for slower file
> transfer because you keep rerunning path mtu probes and tcp
> slow start It makes extra socket handles opening and closing
Errm, you missed the last decade. (HTTP/1.1 has keep-alive and
pipelining an
* Nicholas Clark [2010-03-28 18:20]:
> I'm missing something here, I suspect.
Yes, you are.
> How can HTTP be more efficient than rsync? The only obvious
> method to me of mirroring a CPAN site by HTTP is to instruct
> a client (such as wget) to get it all.
As Arthur has repeatedly pointed this
* Graham Barr [2010-03-26 10:20]:
> On Mar 25, 2010, at 8:42 AM, Barbie wrote:
> >Lastly I would also personnally be annoyed if only the latest
> >versions were available, as I often make great use of the diff
> >tool on search.cpan.org. Having only the latest version
> >renders that great tool re
Why is rsync a problem? Where is the bottleneck in the protocol or the code
implementing it?
Specifics!
SAR is antiquated doesn't give the info you really need. Using a linux system?
Use procallator and feed resulting collected data to ORCA. Better yet, use
DTrace or at least truss. Compile rsy
On Sun, Mar 28, 2010 at 5:43 PM, Jonathan Yu wrote:
> On Sun, Mar 28, 2010 at 12:55 PM, Dana Hudes wrote:
>> But you can't use CPAN.pm on the Backpan.
> Can't you? It's just a mirror, so if you point CPAN.pm to the backpan,
> you should be able to install packages from there (though to get the
>
cify the author/package name
manually I think).
Of course, I've never done this myself, so I could be mistaken
>
> --Original Message--
> From: Shlomi Fish
> To: module-authors@perl.org
> Cc: dhu...@hudes.org
> Sent: Mar 28, 2010 11:31 AM
> Subject: Re: Trimming th
But you can't use CPAN.pm on the Backpan.
--Original Message--
From: Shlomi Fish
To: module-authors@perl.org
Cc: dhu...@hudes.org
Sent: Mar 28, 2010 11:31 AM
Subject: Re: Trimming the CPAN - "Automatic Purging"
On Sunday 28 Mar 2010 17:28:48 dhu...@hudes.org wrote:
> T
s for each site of
interest.
Sent from my BlackBerry® smartphone with Nextel Direct Connect
-Original Message-
From: Elaine Ashton
Date: Sat, 27 Mar 2010 21:38:16
To: Arthur Corliss
Cc: Elaine Ashton; ;
Subject: Re: Trimming the CPAN - "Automatic Purging"
On Mar 27, 2
t
for the first dependency. Repeat until entire dependency tree is completed
Sent from my BlackBerry® smartphone with Nextel Direct Connect
-Original Message-
From: Nicholas Clark
Date: Sun, 28 Mar 2010 17:20:34
To: Arthur Corliss
Cc: Elaine Ashton; ;
Subject: Re: Trimming the CPAN -
You are misunderstanding the problem of changing the mirroring mechanism.
Making new software is nice and good -- Andreas already has something that's
better for the PAUSE data.
Getting 1000s of mirrors to use your software (rather than rsync which they use
for ALL OTHER mirrors -- not so easy.
> On Sat, 27 Mar 2010 16:44:49 -0800 (AKDT), Arthur Corliss
> said:
> On Sat, 27 Mar 2010, Jarkko Hietaniemi wrote:
>> The time-honored tradition of many open source communities is to
>> talk. And talk. And talk. The problem is that this solves nothing.
>> To do, does.
>>
>> Yo
On 28 Mar 2010, at 19:32, Elaine Ashton wrote:
> Jarkko and I were talking about it this morning - as he's not in favour of
> pruning - while trying to think of a way around the size problem and he
> reminded me of the idea that, if I recall correctly was Adreas' suggestion a
> while back, there
On Mar 28, 2010, at 12:48 PM, Randy Kobes wrote:
>
> Has some sort of disk quota system for CPAN author accounts ever been
> considered?
Not specifically, no, at least not that I'm aware of. That would have to be
implemented on PAUSE and quotas frequently end up not solving the real problem
On Sun, 28 Mar 2010, Elaine Ashton wrote:
I'm not sending any barbs, only my reasonable opinion borne from years on the
reality-based operations side of this equation. As for who you are, it doesn't
matter as I work daily with those who wrote, and continue to write, large
chunks of operating
On Sun, 28 Mar 2010, Ask Bj?rn Hansen wrote:
You are misunderstanding the problem of changing the mirroring mechanism.
I am not misunderstanding, I'm just willing to accept the reality for what
it is. Rsync does not scale. Period.
Making new software is nice and good -- Andreas already has
On 2010-03-28, at 9:13 AM, Elaine Ashton wrote:
> On Mar 28, 2010, at 12:52 AM, Arthur Corliss wrote:
>
>> What you're overlooking is that CPAN has, and will, continue to grow. Even
>> if you remove the cruft now at some point it might grow to the same size
>> just with fresh files. When that
On Sat, Mar 27, 2010 at 08:52:22PM -0800, Arthur Corliss wrote:
> On Sat, 27 Mar 2010, Elaine Ashton wrote:
>
> >Actually, I thought I was merely offering my opinion both as the sysadmin
> >for the canonical CPAN mothership and as an end-user. If that makes me a
> >prick, well, I suppose I shoul
> -Original Message-
> From: Shlomi Fish [mailto:shlo...@iglu.org.il]
> Sent: Sunday, March 28, 2010 6:32 PM
> To: module-authors@perl.org
> Cc: dhu...@hudes.org
> Subject: Re: Trimming the CPAN - "Automatic Purging"
>
> On Sunday 28 Mar 2010 17:28:4
On Sunday 28 Mar 2010 17:28:48 dhu...@hudes.org wrote:
> The entire point of rsync is to send only changes.
> Therefore once your mirror initially syncs the old versions of modules is
> not the issue. Indeed, removing the old versions would present additional
> burden on synchronization! The ongoin
The entire point of rsync is to send only changes.
Therefore once your mirror initially syncs the old versions of modules is
not the issue. Indeed, removing the old versions would present additional
burden on synchronization! The ongoing burden is the ever-growing CPAN.
The danger in a CPAN::Mini
On Mar 28, 2010, at 12:52 AM, Arthur Corliss wrote:
>
> :-) You'll have to pardon my indiscriminate epithets. The barbs are coming
> from multiple directions. My point still stands, however. Your experience,
> however worthy, has zero bearing on whether or not my experience is
> just as worthy
# from Andreas J. Koenig
# on Saturday 27 March 2010 21:02:
>If you want to study the CPAN "checkpointed logs" solution running on
>the very CPAN for exactly one year now: File::Rsync::Mirror::Recent
>
>What needs to be done is really extremely trivial: rewrite it in C and
>convince the rsync peop
On Sat, 27 Mar 2010, Elaine Ashton wrote:
Actually, I thought I was merely offering my opinion both as the sysadmin for
the canonical CPAN mothership and as an end-user. If that makes me a prick,
well, I suppose I should go out and buy one :)
:-) You'll have to pardon my indiscriminate epith
On Mar 27, 2010, at 2:52 PM, Arthur Corliss wrote:
>
> Don't be such an arrogant prick. You guys made baseless assumptions about
> people's experience with storage management in an attempt to diregard their
> opinions. That's being a dick by any metric.
Actually, I thought I was merely offerin
> Oh, I understand that fully. And I'd be happy to lend some of my
time. But
you don't make people inclined to help when people are lobbing snarky
comments like "we'll wait breathlessly for you to do it."
The time-honored tradition of many open source communities is to talk.
And talk. And
On Sat, 27 Mar 2010, Jarkko Hietaniemi wrote:
The time-honored tradition of many open source communities is to talk. And
talk. And talk. The problem is that this solves nothing. To do, does.
You are free to decide to take this as a personal insult.
I didn't take it as an insult, I took it
# from Arthur Corliss
# on Saturday 27 March 2010 12:52:
>...should it appear that we have some kind of elitist cabal that will
>make their decision in isolation.
More likely there will not be some decision made because there will be
no action taken.
>If that's going to be the case then this sh
On Mar 26, 2010, at 16:02, Arthur Corliss wrote:
> Why use rsync, then? Why not have checkpointed logs on cpan with
> additions/removals logged by date so you can roll forward on the client,
> processing only those files? It would be trivial to set up and a lot more
> efficient.
I find it cur
On Sat, 27 Mar 2010, Nicholas Clark wrote:
"I"
You?
Or someone else?
I am quite happy to agree that your understanding and experience of storage
management is better than mine. But that's not the key question, in a
volunteer organisation. The questions I ask, repeating Jan's comments in
anot
On Sat, Mar 27, 2010 at 10:52:05AM -0800, Arthur Corliss wrote:
> I think I was quite explicit in saying that efficiencies should be pursued
> in multiple areas, but the predominant bitch I took away from your thread
> dealt with the burden of synchronizing mirrors. What's the easiest way to
> ad
On Fri, 26 Mar 2010, Elaine Ashton wrote:
Oh, don't be such a drama queen. I rebuilt and helped run nic.funet.fi for 2
years which is the canonical mirror for a large number of mirrors and the
perspective of having a few terabytes spinning in storage changes quite
dramatically when you are ac
On Friday-201003-26 19:02, Arthur Corliss wrote:
On Fri, 26 Mar 2010, Jarkko Hietaniemi wrote:
The total size is not the problem. The number of files is. Vanilla
rsync is horribly inefficient (not the protocol, which is genius, mind)
because a client coming by and asking for updates basically
On Friday-201003-26 13:20, Arthur Corliss wrote:
On Fri, 26 Mar 2010, Andy Lester wrote:
Absolutely. This factual info would ideally look like this:
"Of the 17,000 distros on CPAN, there are 8,000 that have versions more than a year
older than the most recent one. If those distros with vers
On Fri, 26 Mar 2010, Arthur Corliss wrote:
> But what the hell do I know. I don't run a *CPAN* mirror, so I must be
> freaking clueless...
It's not about what you know, but about what you are willing to
do yourself.
At some point you have to accept that the people who *do* the work
decide *how*
On 27 Mar 2010, at 00:59, Elaine Ashton wrote:
> The only snag I can forsee in trimming back on the abundance of modules is
> the case where some modules have version requirements for other modules where
> it will barf with a mismatch/newer version of the required module (I bumped
> into this re
On 26 Mar 2010, at 23:32, Arthur Corliss wrote:
> But it's the weakest and simplest link to replace.
Quite a bit of the discussion here on this topic has revolved around an
explanation of why that isn't the case. Setting up rsync is trivial for mirror
operators. Any alternative would likely be
On Mar 26, 2010, at 8:23 PM, Arthur Corliss wrote:
>
> Sure, I don't run a CPAN mirror, but I do manage many, many terrabytes of
> storage as part of my day job. I think it's a tad presumptuous to disregard
> input just because we're not in your inner sanctum. As I mentioned in a
> follow up e-
On Fri, 26 Mar 2010, Ask Bj?rn Hansen wrote:
I find it curious that everyone who's actually involved in syncing the files or running
mirror servers seem to think it generally sounds like a good idea and everyone who
doesn't say it's "not worth the effort".
Sure, I don't run a CPAN mirror, bu
On Fri, 26 Mar 2010, Jarkko Hietaniemi wrote:
We wait your implementation breathlessly. By the time all the CPAN mirrors
have started using that, we probably will be rather blue in
the face.
Now, let's not be that way. :-) You need to pick your problem domain. You
guys can try to go throu
On Fri, 26 Mar 2010, Jarkko Hietaniemi wrote:
The total size is not the problem. The number of files is. Vanilla
rsync is horribly inefficient (not the protocol, which is genius, mind)
because a client coming by and asking for updates basically ends up
requiring the moral equivalent of
"find .
On Fri, 26 Mar 2010, Andy Lester wrote:
Absolutely. This factual info would ideally look like this:
"Of the 17,000 distros on CPAN, there are 8,000 that have versions more than a year
older than the most recent one. If those distros with versions more than a year out of
date were purged, th
> -Original Message-
> From: Ask Bjørn Hansen [mailto:a...@perl.org]
> Sent: Thursday, March 25, 2010 5:11 PM
> To: Tim Bunce
> Cc: cpan-work...@perl.org; module-authors@perl.org; Andreas J. Koenig
> Subject: Re: Trimming the CPAN - "Automatic Purging"
>
On Mar 26, 2010, at 4:55 AM, Lars Thegler wrote:
> I appreciate that the number of files on CPAN has implications for the
> infrastructure, but I feel a need to have some more factual info
> before conceding to such measures.
Absolutely. This factual info would ideally look like this:
"Of the
On Thu, Mar 25, 2010 at 4:55 PM, Ask Bjørn Hansen wrote:
>
> On Mar 25, 2010, at 8:38, Andy Armstrong wrote:
>
>>> I like that solution better
>>
>> [snip]
>>
>> But solution to what? Are we convinced there's actually a problem here?
>
> CPAN has almost 200k files. www.cpan.org says there are "17
What Jarkko said.
On Mar 25, 2010, at 08:00, Jarkko Hietaniemi wrote:
> I have one case where the v1 and v2 of a module are simply
> incompatible, but v1 still works, and unless the users have a
> compelling reason, they won't migrate. Pulling the rug from under
> them would be quite unsportsman
I have one case where the v1 and v2 of a module are simply
incompatible, but v1 still works, and unless the users have a
compelling reason, they won't migrate. Pulling the rug from under
them would be quite unsportsmanlike.
Deletion should be opt-in, and there should be a way to "pin" some
releas
On Mar 25, 2010, at 08:10, Ask Bjørn Hansen wrote:
> I agree with Jarkko that there should be a way to "pin" some versions and the
> configuration should be "more than N newer releases" or some such.
>
> I think it should be on by default though. Older than 3 (or 6?) months and
> at least 2 or
On Mar 25, 2010, at 8:38, Andy Armstrong wrote:
>> I like that solution better
>
>
> [snip]
>
> But solution to what? Are we convinced there's actually a problem here?
CPAN has almost 200k files. www.cpan.org says there are "17627 modules".
rsyncing a gazillion files doesn't work that well
On Mar 25, 2010, at 4:12, Tim Bunce wrote:
> Currently on PAUSE you have to explicitly delete old uploads.
>
> How about changing it so you have to explicitly KEEP old uploads
> that appear to have been superseded?
I like it.
I agree with Jarkko that there should be a way to "pin" some version
Currently on PAUSE you have to explicitly delete old uploads.
How about changing it so you have to explicitly KEEP old uploads
that appear to have been superseded?
PAUSE already has a mechanism to delete files at some future point in
time. That's currently only used as part of a safety/sanity che
On Mar 25, 2010, at 8:42 AM, Barbie wrote:
>
> Lastly I would also personnally be annoyed if only the latest versions
> were available, as I often make great use of the diff tool on
> search.cpan.org. Having only the latest version renders that great tool
> redundant :(
I use that too :-) and it
On Mar 25, 2010, at 10:38 AM, Andy Armstrong wrote:
> But solution to what? Are we convinced there's actually a problem here?
The first two rules of optimization club:
1) You do not optimize.
2) You do not optimize without measuring.
As soon as someone can explain specifics of the problem, inc
On 25 Mar 2010, at 15:36, Chris Nandor wrote:
> I like that solution better
[snip]
But solution to what? Are we convinced there's actually a problem here?
--
Andy Armstrong, Hexten
1 - 100 of 101 matches
Mail list logo
