Also, why not use an FC volume for quorum? That would be better. Ceri
On Fri, Sep 04, 2009 at 09:38:36PM +0100, Ceri Davies wrote: > This looks just like a quorum problem. > > Can you describe what your network to the quorum server is? > Is it only reachable via node xCloud? > > Ceri > > On Fri, Sep 04, 2009 at 01:00:31PM -0700, Janey Le wrote: > > After setting up SunCluster on OpenSolaris, and when I reboot the second > > node of the cluster, my first node panic. Can you please let me know if > > there is anyone that I can contact to know if this is setup issue or it is > > cluster bug? > > > > Below is the setup that I had: > > > > - 2x1 ( 2 OpenSolaris 2009.06 x86 hosts named xCid and xCloud connected > > to one FC array) > > - Created 32 volumes and mapped to the host group; under the host groups > > are the 2 nodes cluster > > - Format the volumes > > - Setup cluster with quorum server named Auron (all 2 nodes joined > > cluster, all of the resource groups and resources are online on 1st node > > xCid) > > > > Below is the status of the cluster before rebooting the nodes. > > root at xCid:~# scstat -p > > ------------------------------------------------------------------ > > > > -- Cluster Nodes -- > > > > Node name Status > > --------- ------ > > Cluster node: xCid Online > > Cluster node: xCloud Online > > > > ------------------------------------------------------------------ > > > > -- Cluster Transport Paths -- > > > > Endpoint Endpoint Status > > -------- -------- ------ > > Transport path: xCid:e1000g3 xCloud:e1000g3 Path > > online > > Transport path: xCid:e1000g2 xCloud:e1000g2 Path > > online > > > > ------------------------------------------------------------------ > > > > -- Quorum Summary from latest node reconfiguration -- > > > > Quorum votes possible: 3 > > Quorum votes needed: 2 > > Quorum votes present: 3 > > > > > > -- Quorum Votes by Node (current status) -- > > > > Node Name Present Possible Status > > --------- ------- -------- ------ > > Node votes: xCid 1 1 Online > > Node votes: xCloud 1 1 Online > > > > > > -- Quorum Votes by Device (current status) -- > > > > Device Name Present Possible Status > > ----------- ------- -------- ------ > > Device votes: Auron 1 1 Online > > > > ------------------------------------------------------------------ > > > > -- Device Group Servers -- > > > > Device Group Primary Secondary > > ------------ ------- --------- > > > > > > -- Device Group Status -- > > > > Device Group Status > > ------------ ------ > > > > > > -- Multi-owner Device Groups -- > > > > Device Group Online Status > > ------------ ------------- > > > > ------------------------------------------------------------------ > > > > -- Resource Groups and Resources -- > > > > Group Name Resources > > ---------- --------- > > Resources: xCloud-rg xCloud-nfsres r-nfs > > Resources: nfs-rg nfs-lh-rs nfs-hastp-rs nfs-rs > > > > > > -- Resource Groups -- > > > > Group Name Node Name State Suspended > > ---------- --------- ----- --------- > > Group: xCloud-rg xCid Online No > > Group: xCloud-rg xCloud Offline No > > > > Group: nfs-rg xCid Online No > > Group: nfs-rg xCloud Offline No > > > > > > -- Resources -- > > > > Resource Name Node Name State Status > > Message > > ------------- --------- ----- > > -------------- > > Resource: xCloud-nfsres xCid Online Online - > > LogicalHostname online. > > Resource: xCloud-nfsres xCloud Offline Offline > > > > Resource: r-nfs xCid Online Online - > > Service is online. > > Resource: r-nfs xCloud Offline Offline > > > > Resource: nfs-lh-rs xCid Online Online - > > LogicalHostname online. > > Resource: nfs-lh-rs xCloud Offline Offline > > > > Resource: nfs-hastp-rs xCid Online Online > > Resource: nfs-hastp-rs xCloud Offline Offline > > > > Resource: nfs-rs xCid Online Online - > > Service is online. > > Resource: nfs-rs xCloud Offline Offline > > > > ------------------------------------------------------------------ > > > > -- IPMP Groups -- > > > > Node Name Group Status Adapter Status > > --------- ----- ------ ------- ------ > > IPMP Group: xCid sc_ipmp0 Online e1000g1 Online > > > > IPMP Group: xCloud sc_ipmp0 Online e1000g0 Online > > > > > > -- IPMP Groups in Zones -- > > > > Zone Name Group Status Adapter Status > > --------- ----- ------ ------- ------ > > ------------------------------------------------------------------ > > root at xCid:~# > > > > > > root at xCid:~# clnode show > > > > === Cluster Nodes === > > > > Node Name: xCid > > Node ID: 1 > > Enabled: yes > > privatehostname: clusternode1-priv > > reboot_on_path_failure: disabled > > globalzoneshares: 1 > > defaultpsetmin: 1 > > quorum_vote: 1 > > quorum_defaultvote: 1 > > quorum_resv_key: 0x4A9B35C600000001 > > Transport Adapter List: e1000g2, e1000g3 > > > > Node Name: xCloud > > Node ID: 2 > > Enabled: yes > > privatehostname: clusternode2-priv > > reboot_on_path_failure: disabled > > globalzoneshares: 1 > > defaultpsetmin: 1 > > quorum_vote: 1 > > quorum_defaultvote: 1 > > quorum_resv_key: 0x4A9B35C600000002 > > Transport Adapter List: e1000g2, e1000g3 > > > > root at xCid:~# > > > > > > ****** Reboot 1st node xCid, all of the resources transfer to 2nd node > > xCloud and online on node xCloud ************ > > > > root at xCloud:~# scstat -p > > ------------------------------------------------------------------ > > > > -- Cluster Nodes -- > > > > Node name Status > > --------- ------ > > Cluster node: xCid Online > > Cluster node: xCloud Online > > > > ------------------------------------------------------------------ > > > > -- Cluster Transport Paths -- > > > > Endpoint Endpoint Status > > -------- -------- ------ > > Transport path: xCid:e1000g3 xCloud:e1000g3 Path > > online > > Transport path: xCid:e1000g2 xCloud:e1000g2 Path > > online > > > > ------------------------------------------------------------------ > > > > -- Quorum Summary from latest node reconfiguration -- > > > > Quorum votes possible: 3 > > Quorum votes needed: 2 > > Quorum votes present: 3 > > > > > > -- Quorum Votes by Node (current status) -- > > > > Node Name Present Possible Status > > --------- ------- -------- ------ > > Node votes: xCid 1 1 Online > > Node votes: xCloud 1 1 Online > > > > > > -- Quorum Votes by Device (current status) -- > > > > Device Name Present Possible Status > > ----------- ------- -------- ------ > > Device votes: Auron 1 1 Online > > > > ------------------------------------------------------------------ > > > > -- Device Group Servers -- > > > > Device Group Primary Secondary > > ------------ ------- --------- > > > > > > -- Device Group Status -- > > > > Device Group Status > > ------------ ------ > > > > > > -- Multi-owner Device Groups -- > > > > Device Group Online Status > > ------------ ------------- > > > > ------------------------------------------------------------------ > > > > -- Resource Groups and Resources -- > > > > Group Name Resources > > ---------- --------- > > Resources: xCloud-rg xCloud-nfsres r-nfs > > Resources: nfs-rg nfs-lh-rs nfs-hastp-rs nfs-rs > > > > > > -- Resource Groups -- > > > > Group Name Node Name State Suspended > > ---------- --------- ----- --------- > > Group: xCloud-rg xCid Offline No > > Group: xCloud-rg xCloud Online No > > > > Group: nfs-rg xCid Offline No > > Group: nfs-rg xCloud Online No > > > > > > -- Resources -- > > > > Resource Name Node Name State Status > > Message > > ------------- --------- ----- > > -------------- > > Resource: xCloud-nfsres xCid Offline Offline > > Resource: xCloud-nfsres xCloud Online Online - > > LogicalHostname online. > > > > Resource: r-nfs xCid Offline Offline > > Resource: r-nfs xCloud Online Online - > > Service is online. > > > > Resource: nfs-lh-rs xCid Offline Offline > > Resource: nfs-lh-rs xCloud Online Online - > > LogicalHostname online. > > > > Resource: nfs-hastp-rs xCid Offline Offline > > Resource: nfs-hastp-rs xCloud Online Online > > > > Resource: nfs-rs xCid Offline Offline > > Resource: nfs-rs xCloud Online Online - > > Service is online. > > > > ------------------------------------------------------------------ > > > > -- IPMP Groups -- > > > > Node Name Group Status Adapter Status > > --------- ----- ------ ------- ------ > > IPMP Group: xCid sc_ipmp0 Online e1000g1 Online > > > > IPMP Group: xCloud sc_ipmp0 Online e1000g0 Online > > > > > > -- IPMP Groups in Zones -- > > > > Zone Name Group Status Adapter Status > > --------- ----- ------ ------- ------ > > ------------------------------------------------------------------ > > root at xCloud:~# > > > > > > ***********Wait for about 5 minutes, then reboot 2nd node xCloud and node > > xCid panic with the error below ********************* > > > > root at xCid:~# Notifying cluster that this node is panicking > > WARNING: CMM: Reading reservation keys from quorum device Auron failed with > > error 2. > > > > panic[cpu0]/thread=ffffff02d0a623c0: CMM: Cluster lost operational quorum; > > aborting. > > > > ffffff0011976b50 genunix:vcmn_err+2c () > > ffffff0011976b60 > > cl_runtime:__1cZsc_syslog_msg_log_no_args6FpviipkcpnR__va_list_element__nZsc_syslog_msg_status_enum__+1f > > () > > ffffff0011976c40 > > cl_runtime:__1cCosNsc_syslog_msgDlog6MiipkcE_nZsc_syslog_msg_status_enum__+8c > > () > > ffffff0011976e30 > > cl_haci:__1cOautomaton_implbAstate_machine_qcheck_state6M_nVcmm_automaton_event_t__+57f > > () > > ffffff0011976e70 cl_haci:__1cIcmm_implStransitions_thread6M_v_+b7 () > > ffffff0011976e80 cl_haci:__1cIcmm_implYtransitions_thread_start6Fpv_v_+9 () > > ffffff0011976ed0 cl_orb:cllwpwrapper+d7 () > > ffffff0011976ee0 unix:thread_start+8 () > > > > syncing file systems... done > > dumping to /dev/zvol/dsk/rpool/dump, offset 65536, content: kernel > > 51% done[2mMIdoOe > > > > the host log is attched. > > > > I have gone thru the SunCluster doc on how to setup SunCluster for > > OpenSolaris multiple times, but I don???t see any steps that I miss. Can > > you please help to see if this is setup issue or it is a bug? > > > > Thanks, > > > > Janey > > -- > > This message posted from opensolaris.org > > > > _______________________________________________ > > ha-clusters-discuss mailing list > > ha-clusters-discuss at opensolaris.org > > http://mail.opensolaris.org/mailman/listinfo/ha-clusters-discuss > > > -- > That must be wonderful! I don't understand it at all. > -- Moliere > _______________________________________________ > ha-clusters-discuss mailing list > ha-clusters-discuss at opensolaris.org > http://mail.opensolaris.org/mailman/listinfo/ha-clusters-discuss -- That must be wonderful! I don't understand it at all. -- Moliere
