On Sat, Sep 20, 2014 at 9:27 PM, Peter Stuge <pe...@stuge.se> wrote:
>
> I've so far gotten zero feedback on my hosting offer, intended to
> help find some starting processes.
>

hassufel's repository on github should be more than adequate:
https://github.com/gentoo/gentoo-gitmig

If we need history we can always substitute this with one with full history.

On a side note, my migration scripts take a few hours to run on a
fairly old Phenom II quad-core.  From the looks of things,
single-threaded performance seems to be what matters the most.  I did
try running a migration on EC2 for kicks, and this only reinforced the
need for good single-threaded performance (which is awful on EC2) -
with 32 cores it did make it through a majority of the tree in very
short order, but the longest categories (especially profiles) still
took something like 1.5 hours to convert and then the second stage of
the migration doesn't really use more than a core or two.  It was
faster than doing it on the old Phenom, but not by all that much.

If somebody has a beefy server they'd like to test a migration on let
me know.  I have a tarball of a chroot that is all set to run a
migration, and a squashfs of the cvs tree which just needs to be
mounted on top of it.  It is fairly trivial to transport, and in the
event of an actual migration it could be tested and then at the time
of the migration all you need is a current cvsroot to mount on top.

I suspect that more than 4 cores would be optimal, but I'd put the
highest priority on single-thread performance.  It didn't really seem
to be IO-bound on the writes (and this is on btrfs on spinning disks
which is less than stellar for performance), but if you have a large
number of cores IO might become a bottleneck.  I don't think IO reads
are going to be much of an issue at all - the starting tree is only a
few hundred MB in squashfs, and the disks didn't look busy for reads
in the later steps at all.

Even a few hours for such a huge migration isn't really that big a
deal I would think, but I've heard that on a good system this can be
cut down to under an hour (plus all the time moving trees around,
setting up back-end, etc).

--
Rich

Reply via email to