Crossfire wrote:
I've just spent some time quickly researching this to no real satisfaction.
What I'm looking for is a way to do real-time hot-replication of a whole
filesystem or filesystem tree over 2 nodes (and strictly 2 nodes)
without STOMITH[1].
The scenario is I have two identical systems with local (software)
RAID1. They will be tethered onto their internet feed via ethernet, and
can optionally be tethered to each other via Gig.
I want to be able to set it up so /home (and maybe other filesystems)
are replicated from one to the other, in both directions, in real time
so they can run in an all-hot redundant cluster.
The environment should be mostly read-oriented, so I can live with
write-latent solutions as long as they handle the race/collision
gracefully (preferably by actually detecting and reporting it if they
can't avoid it).
I've had some success with Software iSCSI targets on Linux to date. I'm
currently using software iSCSI over Gigabit Ethernet to back a VMware
ESX cluster[0].
Software iSCSI targets (I have experience only with tgtd - the only one
that seemed current) present a Linux block device as an iSCSI target
over the network. I present an LVM logical volume.
One could conceive of an eventuality where you made both machines iSCSI
targets and initiators and ran RAID1 over the local and remote iSCSI
targets[1]. I have no idea what sort of (terrible) performance you might
get out of this sort of setup, but it would meet your requirements, and
with enough RAM for read-caching in each node, it might not be too bad.
You would need that Gigabit cross-connect.
There are large warnings in the scsi_tgt code about using it in
production, however.
I suspect this problem space isn't addressed terribly often because,
well, (1) it's Hard, (2) most people who care about this stuff buy
shared storage (check ebay), (3) It's even Harder once you start talking
file systems that do this[2].
Cheers,
Matt
0. I can post my recipe for the target bits, if anyone cares.
1. With a global filesystem, of course.
2. Ceph, which Robert Collins suggested above, is a really good example
of a brilliantly designed distributed file system (much better than
MogileFS, which is more an on-disk hash table with quirks), but I have
my doubts about it's suitability for production systems (though I hope
it gets there).
--
SLUG - Sydney Linux User's Group Mailing List - http://slug.org.au/
Subscription info and FAQs: http://slug.org.au/faq/mailinglists.html
