On Wed, Jan 04, 2012 at 12:09:39AM -0600, [email protected] wrote:
> Could you remark on GPFS services hosting mail storage over a WAN between
> two geographically separated data centers?
I haven't tried that, but know the theory quite well. There are 2 or 3 options:
1 - shared SAN between the data centers. Should work the same as
a single data center, but you'd want to use disk quorum or
a quorum node on a 3. site to avoid split brain.
2 - different SANs on the two sites. Disks on SAN1 would belong
to failure group 1 and disks on SAN2 would belong to failure
group 2. GPFS will write every block to disks in different
failure groups. Nodes on location 1 will use SAN1 directly,
and write to SAN2 via tcp/ip to nodes on location 2 (and vica
versa). It's configurable if you want to return success when
first block is written (asyncronous replication), or if you
need both replicas to be written. Ref: mmcrfs -K:
http://publib.boulder.ibm.com/infocenter/clresctr/vxrx/index.jsp?topic=%2Fcom.ibm.cluster.gpfs.v3r4.gpfs300.doc%2Fbl1adm_mmcrfs.html
With asyncronous replication it will try to allocate both
replicas, but if it fails you can re-establish the
replication level later using "mmrestripefs".
Reading will happen from a direct attached disk if possible,
and over tcp/ip if there are no local replica of the needed
block.
Again you'll need a quorum node on a 3. site to avoid split brain.
3 - GPFS multi-cluster. Separate GPFS clusters on the two
locations. Let them mount each others filesystems over IP,
and access disks over either SAN or IP network. Each cluster is
managed locally, if one site goes down the other site also
loses access to the fs.
I don't have any experience with this kind of config, but believe
it's quite popular to use to share fs between HPC-sites.
http://www.ibm.com/developerworks/systems/library/es-multiclustergpfs/index.html
http://www.cisl.ucar.edu/hss/ssg/presentations/storage/NCAR-GPFS_Elahi.pdf
-jf
