Hello all!
The case is over. Just a few words about the way I could solve the problem
1. The original node names & nodeid-s were restored as they were from the
very beginning (mkdb01 nodeid 0, mkdb02 nodeid 1).
But vxfen module could no be configured. Instead it produced
VXFEN vxfenconfig NOTICE Driver will use SCSI-3 compliant disks.
Thu May 6 11:56:53 MSD 2010 cluster fencing. will retry. sleeping for 21
Thu May 6 11:57:14 MSD 2010 calling regular vxfenconfig
Thu May 6 11:57:19 MSD 2010 return value from above operation is 10
Thu May 6 11:57:19 MSD 2010 output was VXFEN vxfenconfig ERROR V-11-2-1016
There exists the potential for a preexisting split-brain
The coordinator disks list no nodes which,
are in the current membership. However, they,
also list nodes which are not in the,
current membership.
I/O Fencing Disabled!
The first thing to be done is to clean the keys. I tried to do it with
vxfenclearpre
and
"vxfenadm -r all -f /etc/vxfendg" showed no keys on disks
but vxfenconfig -c could not configure vxfen driver with same error :
The coordinator disks list no nodes which,
are in the current membership. However, they,
also list nodes which are not in the,
current membership.
At last it came to me to perform vxfentsthdw on disks (EMC Clariion) . And
only after that all the keys were cleaned out of disks
and fence drivers were configured on both nodes .... And CVM started!
So I can make two conclusions about SF 5.1 P1 for Oracle RAC on RedHat
Linux :
1. Changig nodeids can lead to a unability to configure CVM cluster.
2. vxfenclearpre cannot sometimes remove the old reservation keys.
vxfenttsthdw can do it for sure.
Kind regards! Pavel
> Hello all!
>
> I see a strange problem with my VCS for RAC configuration SF5.1
> patch1 & patch 3 for RedHat Linux.
>
> I had two-node configuration: mkdb01 and mkdb02 with nodeid 0 and 1
> correspondingly.
> But the master (mkdb01) crashed and it was reinstalled. The second
> node (mkdb02) was re-configured manually to get nodeid 0 (this is a
> had-been nodeid of mkdb01).
> It continued to operate as a single-RAC under VCS SF after reboot.
> Now I try to restore two-node RAC configuration with mkdb01 with
> nodeid 1 . So nodeids are changed.
>
> The main problem appeared when I tried to add mkdb01 into CVM group
> again. CVMCluster resource on mkdb01 node cannot start.
>
> Then I changed the node name mkdb01 for mkdb03 and nodeid 1 for
> nodeid 2. Just the same.
>
>
> r...@mkdb03~]#vxclustadm nodestate
> state: joining
> reconfig: master selection
>
> [r...@mkdb03 ~]# gabconfig -a
> GAB Port Memberships
> ===============================================================
> Port a gen c13d1b membership 0 2
> Port b gen c13d19 membership 0 2
> Port d gen c13d1c membership 0 2
> Port h gen c13d3f membership 0 2
> Port o gen c13d1b membership 0 2
> Port v gen c13d31 membership 0 2
>
>
> No port w registration! But vxconfigd is running...
>
>
>
>
> [r...@mkdb03 ~]# lltstat -vvn
> LLT node information:
> Node State Link Status Address
> 0 mkdb02 OPEN
> eth2 UP 00:21:5E:99:8D:A4
> eth3 UP 00:21:5E:99:8D:A6
> 1 CONNWAIT
> eth2 DOWN
> eth3 DOWN
> * 2 mkdb03 OPEN
> eth2 UP 00:21:5E:98:E1:38
> eth3 UP 00:21:5E:98:E1:3A
>
>
>
>
>
> And /var/log/messages on master node (mkdb02) tells
>
> mkdb02 kernel: VxVM vxio V-5-0-415 convert: returning -1 for cvm id 1
> mkdb02 kernel: VxVM vxio V-5-3-708 reconfd: can't find nodeid for mkdb03
>
> This error seems to be very like as in http://seer.entsupport.
> symantec.com/docs/263660.htm
>
> 12) CVMCluster NodeId attribute does not match that defined in
> /etc/llthosts or /etc/llttab
>
> But the nodeids do match in the files!
>
> On the master cvm node:
>
> [r...@mkdb02 ~]# vxclustadm nodestate
> state: cluster member
>
> [r...@mkdb02 ~]# gabconfig -a
> GAB Port Memberships
> ===============================================================
> Port a gen c13d1b membership 0 2
> Port b gen c13d19 membership 0 2
> Port d gen c13d1c membership 0 2
> Port f gen c13d10 membership 0
> Port f gen c13d10 visible ; 2
> Port h gen c13d3f membership 0 2
> Port o gen c13d1b membership 0 2
> Port v gen c13d31 membership 0 2
> Port w gen c13d0e membership 0
> Port w gen c13d0e visible ; 2
>
> Any suggestions ?
>
> Thank you! Regards, Pavel Tsvetkov
>
>
>
>
> _______________________________________________
> Veritas-ha maillist - Veritas-ha@mailman.eng.auburn.edu
> http://mailman.eng.auburn.edu/mailman/listinfo/veritas-ha_______________________________________________
Veritas-ha maillist - Veritas-ha@mailman.eng.auburn.edu
http://mailman.eng.auburn.edu/mailman/listinfo/veritas-ha
