Hello folks,
I'm sorry if it goes a bit off topic but here is my experience of using OCFS2
in a four node HA cluster.
I have set up Corosync/OpenAIS, Pacemaker, DRBD cluster of four nodes.
On the node1 and node2 I have DRBD replicated partition as /dev/drbd0 of 3.1 TB
for MySQL, SVN and other data storage formatted with ocfs2 and mounted as /data
on the node 1 and node2.
On the node3 and node4 I have the same DRBD replicated partition formatted as
ocfs2 and mounted as /files for storing user files and emails in maildir format.
Now I wanted to share /data and /files between all four nodes, so node1 and
node2 will have access to /files served by node3 and node4, and node1 and node2
will have access to /data served by node1 and node2. I don't think that it's an
unreasonable setup from the practical point of view.
First, I tried to export over NFS v3/v4 /data from node1/node2 to node3/node4
and /files from node3/node4 to node1/node2. Due to NFS locking mechanism
conflicting with OCSF2 DLM, after a short period of time osfs2 partitions
became inaccessible and then clients and servers just froze up.
Second, I tried to share /data and /files between these four nodes over iSCSI
protocol.
The only configuration which worked some what was when I used just one node1 or
node2 as an iscsi target for /data and node3 or node4 as an iscsi target for
/files.
It was ridiculous that I had to initiate and mount iscsi device as /dev/sdb on
iscsi target itself and do the same on the initiator, instead of mounting and
using their local /dev/drbd0 block devices with the exactly same data.
Unfortunately any other configuration didn't allow OCSF2 to create proper DLM
locking. Even with this awkward but working data sharing configuration, after
some time ocfs2 again was freezing the whole cluster.
Then I started to think what are the practical benefits of using OCFS2 in my
four node cluster setup. For example, MySQL servers cannot use shared data
storage either through ocfs2 or gfs2. SVN servers can work with shared or
distributed data storage even without ocfs2. Mail storage may potentially
benefit from using ocfs2 but only in those rare cases when two users are trying
simultaneously access the same file in the same maildir. Plus ocfs2 has a
32,000 subdirectories number limit which is a big inconvenience for mail
storage for hundred thousands of users.
So, finally I decided to switch to regular XFS file system which though doesn't
have any DLM mechanisms but allows more flexibility in data sharing between the
nodes.
Could you please post your comments on benefits of ocfs2 and its practical
application in cluster setups.
Many thanks in advance,
Alex
_______________________________________________
Linux-HA mailing list
[email protected]
http://lists.linux-ha.org/mailman/listinfo/linux-ha
See also: http://linux-ha.org/ReportingProblems
