Thus spake Bernard Li ([EMAIL PROTECTED]):
Hey guys:
Just did some more test with the latest BitTorrent (4.2.2) downloaded from
Dag's site:
http://dag.wieers.com/packages/bittorrent/
If you want to bt an image, you'll have to exclude the /dev dir for sure.
Also, as Erich pointed out, the file ownership are all wrong as they will
become that of the user running the bittorrent program (on the compute node).
I think we should just stick with the way flamethrower does it and basically
tar up the image directory. On a dual 2.4GHz Opteron, it took 11 seconds to
tar up a ~800MB directory and it would probably take less than 1 minute to
transfer to each client.
Ok, lemme give some of my thoughts on tarballs now. If we must use
tarballs, we can:
* Always maintain a tarball copy of the image directory.
* Update the tarball when certain trigger events occur:
* After an si_getimage, si_cpimage, etc.
* Potentially use FAM to tell us if a file in the image has
changed, then wait X minutes, then re-create the image's
tarball.
* I like the idea of a single tarball, no matter how large the
image gets. And simply require the use of a filesystem that can
take it.
* Because some people may not want: a) to use the BT transport and
b) slow down certain image server operations and c) take up the
extra space on their imageserver, there should be an option in
/etc/systemimager/systemimager.conf:
MAINTAIN_IMAGE_TARBALLS = [yes|no]
With an appropriate description. If set, then we auto-create and
update image tarballs to match the images. Default would be "no",
unless we get to the point where we think BT should be the default
transport for image deployment, instead of rsync. The BT
transport would need to prove itself quite solidly before we would
consider making it the default.
* I think that tarballs should be stored right next to image
directories. Ie:
$ ls /var/lib/systemimager/images/
my_image-v1
my_image-v1.tar{.gz if we determine that's appropriate}
* If we do this, then we should consider if it makes sense to have
the multicast transport use the same tarballs.
We should probably daemonize the BitTorrent tracker and start it by doing
running /etc/init.d/systemimager-server-btd (or something like that) and when
we image a node, we would seed the image we're trying to transfer. We'll need
to figure out a way to stop all the bt processes once all imaging is done
(probably controlled via the GUI or commandline).
I agree -> /etc/init.d/systemimager-server-btd
I noticed that the seeder is uploading much faster than the peers, I wonder if
there's a way to tweak it (although I ran each bittorrent instance with
--max_upload_rate 0, meaning no limit).
Cheers,
Bernard
________________________________
From: Erich Focht [mailto:[EMAIL PROTECTED]
Sent: Thu 01/12/2005 00:56
To: Bernard Li
Cc: [EMAIL PROTECTED]; Brian Elliott Finley; [EMAIL PROTECTED]
Subject: Re: Bit-Torrent transport
Bernard,
did you try to transport the directory to another node?
What happened to the file ownerships and the permission flags? How about the
symbolic links? Did you change anything in BT?
Thanks,
Erich
On Thursday 01 December 2005 09:07, Bernard Li wrote:
With the latest version of BitTorrent (4.2), I was able to create a torrent of the entire
image directory (except for "dev") so I think the only way for us to deal with
this is to extend BT to support a --exlude argument so that we can exclude specific
directories. I was able to create a torrent file with everything (even /proc). We will
probably have to rsync the /dev files separately.
I am still figuring out the new commandline so I haven't been able to actually
transfer the files, but I'll keep you guys posted.
Cheers,
Bernard
--
Brian Elliott Finley
Mobile: 630.631.6621
-------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc. Do you grep through log files
for problems? Stop! Download the new AJAX search engine that makes
searching your log files as easy as surfing the web. DOWNLOAD SPLUNK!
http://ads.osdn.com/?ad_id=7637&alloc_id=16865&op=click
_______________________________________________
Sisuite-devel mailing list
Sisuite-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/sisuite-devel
