Fencing is not a fs operation but a cluster operation. The fs is only a
client
of the cluster stack.
Alexei_Roudnev wrote:
It all depends of the usage scenario.
Tipical usage is, for example:
(1) Shared application home. Writes happens once / week during maintanance,
otehr time files are opened for reading only. Few logfiles
can be redirected if required.
So, when server see a problems, it HAD NOT any pending IO for a 3 days - so
what the purpose of reboot? It 100% knows that NO ANY IO
is pending, and other nodes have not any IO pending as well.
(2) Backup storage for the RAC. FS is not opened 90% of the time. At night,
one node opens it and creates a few files. Other node have not any pending
IO on this FS. Fencing passive node (which dont run any backup) is not
useful because it HAD NOT ANY PENDING IO for a few hours.
(3) WEB server. 10 nodes, 1 only makes updates. The same - most nodes have
not any pending IO.
Of course there is always a risk of FS corruption in the clusters. Any layer
can keep pending IO forever (I saw Linux kernel keeping it for 10 minutes).
Problem is that in such cases software fencing can't help as well because
node is half-dead and can't detect it's own status.
So, the key point here is not in _fence for each ap-chi_ but _keep system
without pending writes as long as possible and make clean transition between
active write/active read / passive states. Then you can avoid self-fencing
in 90% cases (because of server wil be in passive or active reads state). I
mounT FS but don't cd into it, or just CD but dont read - passive status. I
read file - active read for 1 minute, tbhnen flush buffers so that it is in
passive mode again. I began top write - switch system to write mode. I did
not write blocks for 1 minute - flush everything, wait 1 more minute and
switch to passive mode.
----- Original Message -----
From: "Sunil Mushran" <[EMAIL PROTECTED]>
To: "David Miller" <[EMAIL PROTECTED]>
Cc: <[email protected]>
Sent: Monday, April 09, 2007 3:18 PM
Subject: Re: [Ocfs2-users] Catatonic nodes under SLES10
For io fencing to be graceful, one requires better hardware. Read
expensive.
As in, switches where one can choke off all the ios to the storage from
a specific
node.
Read the following for a discussion on force umounts. In short, not
possible as yet.
http://lwn.net/Articles/192632/
Readonly does not work wrt to io fencing. As in, ro only stops any new
userspace
writes but cannot stop pending writes. And writes could be lodged in any
io layer.
A reboot is the cheapest way to avoid corruption. (While a reboot is
painful, it is
much less painful than a corrupted fs.)
With 1.2.5 you should be able to increase the network timeouts and
hopefully avoid
the problem.
David Miller wrote:
Alexei_Roudnev wrote:
Did you checked
/proc/sys/kernel/panic /proc/sys/kernel/panic_on_oops
system variables?
No. Maybe I'm missing something here.
Are you saying that a panic/freeze/reboot is the expected/desirable
behavior? That nothing more graceful could be done, like to just
dismount the ocfs2 file systems, or force them to a read-only mount or
something like that? We have to reload the kernel?
Thanks,
--- David
----- Original Message ----- From: "David Miller" <[EMAIL PROTECTED]>
To: <[email protected]>
Sent: Monday, April 02, 2007 9:01 AM
Subject: [Ocfs2-users] Catatonic nodes under SLES10
[snip]
Both servers will be connected to a dual-host external RAID system.
I've setup ocfs2 on a couple of test systems and everything appears
to work fine.
Until, that is, one of the systems loses network connectivity.
When the systems can't talk to each other anymore, but the disk
heartbeat is still alive, the high numbered node goes catatonic.
Under SLES 9 it fenced itself off with a kernel panic; under 10 it
simply stops responding to network or console. A power cycling is
required to bring it back up.
The desired behavior would be for the higher numbered node to lose
access to the ocfs2 file system(s). I don't really care whether it
would simply timeout ala stale NFS mounts, or immediately error like
access to non-existent files.
_______________________________________________
Ocfs2-users mailing list
[email protected]
http://oss.oracle.com/mailman/listinfo/ocfs2-users
_______________________________________________
Ocfs2-users mailing list
[email protected]
http://oss.oracle.com/mailman/listinfo/ocfs2-users
_______________________________________________
Ocfs2-users mailing list
[email protected]
http://oss.oracle.com/mailman/listinfo/ocfs2-users
