From: Brian Elliott Finley on behalf of Brian Elliott Finley
Sent: Sat 31/12/2005 11:07
To: Bernard Li
Cc: [EMAIL PROTECTED]; [EMAIL PROTECTED]; [EMAIL PROTECTED]; SIS Devel
Subject: Re: Bit-Torrent transport
Thus spake Bernard Li ([EMAIL PROTECTED]):
>Hey
guys:
>
>Just did some more test with the latest BitTorrent (4.2.2)
downloaded from Dag's site:
>
>http://dag.wieers.com/packages/bittorrent/
>
>If
you want to bt an image, you'll have to exclude the /dev dir for sure.
Also, as Erich pointed out, the file ownership are all wrong as they will become
that of the user running the bittorrent program (on the compute
node).
>
>I think we should just stick with the way flamethrower
does it and basically tar up the image directory. On a dual 2.4GHz
Opteron, it took 11 seconds to tar up a ~800MB directory and it would probably
take less than 1 minute to transfer to each client.
Ok, lemme give some
of my thoughts on tarballs now. If we must use
tarballs, we
can:
* Always maintain a tarball copy of the image
directory.
* Update the tarball when certain trigger
events occur:
* After an
si_getimage, si_cpimage, etc.
* Potentially use FAM to tell us if a file in the image
has
changed, then wait
X minutes, then re-create the
image's
tarball.
* I like the idea of a single tarball, no
matter how large the
image gets. And
simply require the use of a filesystem that
can
take it.
*
Because some people may not want: a) to use the BT transport
and
b) slow down certain image server
operations and c) take up the
extra space on
their imageserver, there should be an option
in
/etc/systemimager/systemimager.conf:
MAINTAIN_IMAGE_TARBALLS = [yes|no]
With an
appropriate description. If set, then we auto-create
and
update image tarballs to match the
images. Default would be "no",
unless we
get to the point where we think BT should be the
default
transport for image deployment,
instead of rsync. The BT
transport would
need to prove itself quite solidly before we
would
consider making it the
default.
* I think that tarballs should be stored
right next to image
directories.
Ie:
$ ls
/var/lib/systemimager/images/
my_image-v1
my_image-v1.tar{.gz if
we determine that's appropriate}
* If we do this, then
we should consider if it makes sense to have
the multicast transport use the same tarballs.
>We should probably
daemonize the BitTorrent tracker and start it by doing running
/etc/init.d/systemimager-server-btd (or something like that) and when we image a
node, we would seed the image we're trying to transfer. We'll need to
figure out a way to stop all the bt processes once all imaging is done (probably
controlled via the GUI or commandline).
I agree ->
/etc/init.d/systemimager-server-btd
>I noticed that the seeder is
uploading much faster than the peers, I wonder if there's a way to tweak it
(although I ran each bittorrent instance with --max_upload_rate 0, meaning no
limit).
>
>Cheers,
>
>Bernard
>
>________________________________
>
>From:
Erich Focht [mailto:[EMAIL PROTECTED]]
>Sent:
Thu 01/12/2005 00:56
>To: Bernard Li
>Cc: [EMAIL PROTECTED]; Brian
Elliott Finley; [EMAIL PROTECTED]
>Subject: Re: Bit-Torrent
transport
>
>
>
>Bernard,
>
>did you try to
transport the directory to another node?
>
>What happened to the
file ownerships and the permission flags? How about the
>symbolic links?
Did you change anything in
BT?
>
>Thanks,
>Erich
>
>On Thursday 01 December
2005 09:07, Bernard Li wrote:
>> With the latest version of BitTorrent
(4.2), I was able to create a torrent of the entire image directory (except for
"dev") so I think the only way for us to deal with this is to extend BT to
support a --exlude argument so that we can exclude specific directories. I
was able to create a torrent file with everything (even /proc). We will
probably have to rsync the /dev files separately.
>>
>> I am
still figuring out the new commandline so I haven't been able to actually
transfer the files, but I'll keep you guys posted.
>>
>>
Cheers,
>>
>>
Bernard
>
>
>
--
Brian Elliott
Finley
Mobile: 630.631.6621
