On Sun, 28 Mar 2010, Elaine Ashton wrote:
I'm not sending any barbs, only my reasonable opinion borne from years on the
reality-based operations side of this equation. As for who you are, it doesn't
matter as I work daily with those who wrote, and continue to write, large
chunks of operating systems, X, etc., and though their legend may precede them
when it comes to my having to implement what works fabulously in their
imagination, I do my best to bring them back to the grim reality that is
operations. It's a frequent problem of engineers and those of us stuck having
to live with and fix their grand ideas. Lofty goals usually die somewhere
between dreams and production.
Ah, let the chest thumping begin. My point is that regardless of where the
idea comes from if it comes from a solid rationale it should be given
consideration. And to date I have yet to see any one of you refute my
technical understanding of the problem, only my political understanding of
the problem. I/O is the issue, and it is driven predominantly by rsync.
Well, you'll have to forgive those who mock your n?ivete as if it were so basic
and trivial to replace rsync, it would have been done several times over by now
as it's limitations are well known to all who use it on any large scale.
However, it is a well-known, well-used, multi-platform and time-tested tool
that will not be unseated very easily without good reason and a reason that
reads something along the lines of improving performance on an archive that
should have been trimmed back a bit is not a compelling reason for adoption.
Naivete? Again: show me where my assertions about the primary root of your
problem is incorrect? Show me how pruning CPAN isn't a temporary band-aid
that fails to address a fundamental weakness in the syncing process? you
haven't. You can try to dress it up any way you like in effort to discredit
me, but until you do based on the facts, you have nothing.
Rsync is a good tool, but for different use case scenarios.
And this is a good point to make, yes, it will continue to grow and I know that
the current manager(s) of nic.funet.fi have commented on the burden it presents
to the system which is also home to a number of other mirrors. You cannot
assume that the generosity and the resources of the mirror ops are limitless
and finding out where that limit lies will come too late to make amends.
<G> And you make my point for me. I'm sure he would love to find a more
efficient use of his I/O. I assume nothing, I only allow that you'll find
more interest than you assume in managing I/O. Nor does what I'm proposing
preclude the intractable from continuing to use rsync. Given that rsync is
your driver of the I/O problem taking away any significant percentage of the
problem with have the largest dividends.
Pruning back the archive is a good compromise until and unless another solution
can be done that will not bother the mirror ops terribly much in terms of real
work.
At least you admit you're only treating the symptoms now, not the disease
itself. Sure, it will buy you some time, but there'll also be some
political problems to work through which will likely burn as much if not
more manhours than just treating the disease. And in the end time runs
out and the problem remains.
Look, I don't care if you guys decide against it, but let's be honest about
the compromises you're making. Hell, pruning isn't even a compromise, it's
not a solution, it's only a delaying tactic.
--Arthur Corliss
Live Free or Die
