Hello list,
I'm thinking about a setup that looks like this:
- 2 headnodes with FC connectivity (OpenSolaris)
- 2 backend FC srtorages (Disk Shelves with RAID Controllers presenting
a huge 15 TB RAID5)
- 2 datacenters (distance 1 km with dark fibre)
- one headnode and one storage in each data center
(Sorry for this ascii art :)
( Data Center 1) <--1km--> (Data Center 2)
(primary) (backup)
[Disk Array 1 with Raid CRTL] [Disk Array 2 with Raid CRTL]
[ -- LUN1 16 TB -- ] [ -- LUN2 16 TB -- ]
| \ / |
| / \ |
| / \ |
[ FABRIC 1 ] [ FABRIC 2 ]
| \ / |
| / \ |
| / \ |
[ Osol HeadNode 1 ] [ Osol HeadNode 2 ]
[ -- active -- ]
Zpool "mytest" on HeadNode2 :
mytest
| - mirror
|- LUN1
|- LUN2
Both headnodes can see both storages. The storages are connected to the
hosts via SAN switches and two fabrics (redundant multipath
configuration).
This setup should be a active / passive setup with manual failover (pool
import in case of a site failure)
When thinking about this setup some questions popped into my mind. Most
of them are concerened with resilvering.
SAS-analogy:
If using OpenSolaris in a simple SAS backplane server with SAS disks,
if I pull a disk, the disk failure is detected and the volume continues
in degrated mode. Now if I plug the SAS disk back, automatic resilvering
happens to the disk. Only deltas are resilvered.
How there are different corner cases of outage in the FC example that
are intersting and I'm, not sure how ZFS would react (unfortunately I do
not have the boxes here to test).
Failure scenarios:
a) temporary storage failure
(e.g. Disk Array 1 rebooting)
In this case I expect that the pool continues in degraded mode. When the
storage comes back up I'm not sure if the disks are automatically hot
added to the OS and thus I dont know if an automatic resilvering takes
place.
b) permanent storage failure
(e.g. Disk Array 1 burning down or having 2 disk failure in RAID5 )
In this case I expect that the pool continues in degraded mode. When a
new storage is put back, no automatic resilvering takes place (no vdev
label found) . The LUN has to be replaced manually.
c) split brain - no volume import
(e.g. connection between the sites failing, administrator not issueing
"volume import" on HeadNode2)
This case is similar to a).
d) Short Failure of Data Center 1
(e.g. short power failure in data center 1. No manual failover to data
center 2 by administrator.)
.. actually I have no idea what happens :)
e) Power Outage in Data Center 1
(e.g. long power failure in data center 1. Administrator performs volume
import on HeadNode2)
.. actually I have no idea what happens ... again :)
f) split brain - volume is imported
(e.g. connection between the sites failing, administrator issueing
"volume import" on HeadNode2)
This is a critical case. The pool is active on two nodes, while
HeadNode1 uses LUN1 and HeadNode2 uses LUN2 of the pool. If automatic
resilvering takes place, in which direction will resilving take place ?
Will the nodes overwrite each others data in the backend ? - no idea.
My question is:
Has anyone setup something like this and can give some insights on how
ZFS behaves in the cases above ? Is this a safe setup (guaranteed data
integrity of ZFS) ? How does resilvering identify the direction in which
resilvered should happen ?
I would appreshiate any input on this.
Regards,
Robert
_______________________________________________
storage-discuss mailing list
storage-discuss@opensolaris.org
http://mail.opensolaris.org/mailman/listinfo/storage-discuss
