Stephen J. Turnbull wrote:
Scott> Currently we have offices in several countries that have
Scott> large NFS servers - I think most are NetApps (Network
Scott> Appliance) NAS. These are currently being syncronized by
Scott> using rsync cron jobs at night. The problem is this
Scott> doesn't scale well and the network is having trouble
Scott> keeping up.
Have you considered the BitTorrent protocol instead of rsyncing to a
central server? Of course you'd still have to find a way to identify
new and changed files (BitTorrent as such assumes that's already been
done), and if the changes are small relative to the files you might
want to use rsync for transport anyway. Those might be bigger
problems than daisy-chaining by hand if it's a fairly small number of
servers.
Besides the comments Ivan made, you should also be aware that Coda
doesn't do a lot to optimize file transport IIRC: if one bit changes,
you have to transport the whole file before you can access any of it
again.
<a little off topic but maybe it will help the next guy that asks>
We have talked about BitTorrent a bit. The trick would be to syncronize
the small changes and not the whole structure, like you said so there
would have to write our own client. (I had not thought about writing
our own software ontop of bittorrent -- thanks) We probably have 180GB
in the tree and say 2GB change nightly.
Something even more exotic would be Directory Tree Mirroring by
multicasting. There are a couple of projects, one on Sourceforge but it
requires kernel modules and I believe it's own partition type. A
different project's draft paper here looks cool:
http://member.*wide*.ad.jp/draft/*wide*-draft-soiasia-mtm-00.pdf
(Interesting Stuff)
Thanks again!
-Scott
