Hi there,
I am currently working on a 2 node drbd+pacemaker mysql active / passive
cluster under SLES11 SP1 + HAE on VMware vSphere 4. The Clusters working quite
well when I`m testing failover situations etc., but irregularly reboots both
nodes without logging the reason for the reboots. As nothing like stonith
shutdown commands or error notices appear in the system logs, it looks like the
virtual machine got powered off suddenly. Other VMs (yet SLES10 with Heartbeat
+ Stonith) running without any problems on the virtual infrastructure..
Yesterday I set up 2 clones of the virtual machines and configured them running
without stonith device. These 2 clones survived the night. The ones I`m talking
about right now shut down at different time (one at 7:57pm, one at 05:19 am
this morning). Could be a coincidence but to be honest I don`t think so,
because I got the same problems for weeks now, even after completely
configuring the VMs from OS Installation until cluster configuration.. I will
monitor the clones without stonith device over the weekend to see if theyre
running without rebooting. Anyone got similar problems or can tell me where to
look for restricting what could be the problem?
Thanks in advance!
In order to help you understand my configuration, some facts:
Summary of setup:
- 2 x SLES11 SP1 + HAE virtual machines on VMware vSphere4
- both VMs running on same ESX
- VMware Tools installed
- 2 x vNICs: 1 for Public Interface, 1 for Private Interconnect
- 3 vDisks: 1 for OS, 1 for DRBD, 1 shared disc for Stonith SBD
- pacemaker-1.1.2-0.2.1
- openais-1.1.2-0.5.19
- drbd-8.3.7-0.4.15
- Active / Passive Configuration
lsmod | grep softdog
softdog 4055 2
/etc/sysconfig/sbd
SBD_DEVICE="/dev/sdc"
SBD_OPTS="-W"
/var/log/warn
...
...
Jan 21 09:50:09 mysql-node1 stonith: external/sbd device OK.
Jan 21 09:51:10 mysql-node1 stonith: external/sbd device OK.
Jan 21 09:52:11 mysql-node1 stonith: external/sbd device OK.
...
...
CRM Configure
node mysql-node1 \
attributes standby="off"
node mysql-node2 \
attributes standby="off"
primitive drbd_mysql ocf:linbit:drbd \
params drbd_resource="mysql" \
op monitor interval="15s"
primitive fs_mysql ocf:heartbeat:Filesystem \
params device="/dev/drbd0" directory="/var/lib/mysql" fstype="ext3"
primitive ip_mysql ocf:heartbeat:IPaddr2 \
params ip="192.168.96.10" nic="eth0"
primitive mysqld lsb:mysql
primitive notification ocf:heartbeat:MailTo \
meta target-role="Started" \
operations $id="notification-operations" \
op monitor interval="10" timeout="10" \
params email="[email protected]" subject="Cluster Takeover on mysql-cluster"
primitive ping ocf:pacemaker:ping \
op monitor interval="15" timeout="5" \
params host_list="192.168.96.2"
primitive stonith-device stonith:external/sbd \
params sbd_device="/dev/sdc" \
op monitor interval="60s"
group mysql fs_mysql ip_mysql mysqld
ms ms_drbd_mysql drbd_mysql \
meta master-max="1" master-node-max="1" clone-max="2"
clone-node-max="1" notify="true"
clone notificationclone notification
clone pingclone ping \
meta globally-unique="false"
clone stonith stonith-device
location location_mysql mysql \
rule $id="location_mysql-rule" -inf: not_defined pingd or pingd lte 0
colocation mysql_on_drbd inf: mysql ms_drbd_mysql:Master
order mysql_after_drbd inf: ms_drbd_mysql:promote mysql:start
property $id="cib-bootstrap-options" \
dc-version="1.1.2-2e096a41a5f9e184a1c1537c82c6da1093698eb5" \
cluster-infrastructure="openais" \
expected-quorum-votes="2" \
last-lrm-refresh="1294994548" \
no-quorum-policy="ignore" \
stonith-enabled="true"
Corosync.conf
compatibility: whitetank
aisexec {
user: root
group: root
}
totem {
version: 2
token: 3000
join: 1000
token_retransmits_before_loss_const: 20
consensus: 3600
max_messages: 20
clear_node_high_bit: yes
secauth: off
threads: 0
interface {
ringnumber: 0
bindnetaddr: 10.0.0.0
mcastaddr: 226.94.1.1
mcastport: 5405
}
}
logging {
fileline: off
to_stderr: no
to_logfile: no
to_syslog: yes
syslog_facility: daemon
debug: on
timestamp: on
}
amf {
mode: disabled
}
service {
name: pacemaker
ver: 0
use_mgmtd: yes
use_logd: yes
}
/etc/drbd.d/mysql.res
resource mysql {
protocol C;
startup {
wfc-timeout 0;
degr-wfc-timeout 120;
}
disk {
on-io-error detach;
}
net {
max-buffers 2048;
ko-count 4;
after-sb-0pri disconnect;
after-sb-1pri disconnect;
after-sb-2pri disconnect;
rr-conflict disconnect;
}
syncer {
rate 100M;
al-extents 257;
}
on mysql-node1 {
device /dev/drbd0;
disk /dev/sdb;
address 10.0.0.101:7788;
meta-disk internal;
}
on mysql-node2 {
device /dev/drbd0;
disk /dev/sdb;
address 10.0.0.102:7788;
meta-disk internal;
}
}
/etc/drbd.d/global_common.conf
global {
# usage-count yes;
usage-count no;
# minor-count dialog-refresh disable-ip-verification
}
common {
protocol C;
handlers {
pri-on-incon-degr "/usr/lib/drbd/notify-pri-on-incon-degr.sh;
/usr/lib/drbd/notify-emergency-reboot.sh; echo b > /proc/sysrq-trigger ; reboot
-f";
pri-lost-after-sb "/usr/lib/drbd/notify-pri-lost-after-sb.sh;
/usr/lib/drbd/notify-emergency-reboot.sh; echo b > /proc/sysrq-trigger ; reboot
-f";
local-io-error "/usr/lib/drbd/notify-io-error.sh;
/usr/lib/drbd/notify-emergency-shutdown.sh; echo o > /proc/sysrq-trigger ; halt
-f";
}
startup {
}
disk {
}
net {
}
syncer {
rate 100M;
}
}
Node1 debug log:
Jan 22 10:24:17 mysql-node1 attrd_updater: [14240]: info: attrd_lazy_update:
Connecting to cluster... 5 retries remaining
Jan 22 10:24:17 mysql-node1 attrd_updater: [14240]: debug:
init_client_ipc_comms_nodispatch: Attempting to talk on: /var/run/crm/attrd
Jan 22 10:24:17 mysql-node1 attrd: [9245]: debug: attrd_local_callback: update
message from attrd_updater: pingd=1
Jan 22 10:24:17 mysql-node1 attrd: [9245]: debug: attrd_local_callback:
Supplied: 1, Current: 1, Stored: 1
Jan 22 10:24:17 mysql-node1 attrd_updater: [14240]: debug: attrd_update: Sent
update: pingd=1 for localhost
Jan 22 10:24:18 mysql-node1 lrmd: [9244]: debug: rsc:notification:0:41: monitor
Jan 22 10:24:18 mysql-node1 lrmd: [14241]: debug: perform_ra_op: resetting
scheduler class to SCHED_OTHER
Jan 22 10:24:18 mysql-node1 lrmd: [9244]: debug: RA output:
(notification:0:monitor:stdout) running
Jan 22 10:24:21 mysql-node1 crmd: [9247]: info: crm_timer_popped: PEngine
Recheck Timer (I_PE_CALC) just popped!
Jan 22 10:24:21 mysql-node1 crmd: [9247]: debug: s_crmd_fsa: Processing
I_PE_CALC: [ state=S_IDLE cause=C_TIMER_POPPED origin=crm_timer_popped ]
Jan 22 10:24:21 mysql-node1 crmd: [9247]: info: do_state_transition: State
transition S_IDLE -> S_POLICY_ENGINE [ input=I_PE_CALC cause=C_TIMER_POPPED
origin=crm_timer_popped ]
Jan 22 10:24:21 mysql-node1 crmd: [9247]: info: do_state_transition: Progressed
to state S_POLICY_ENGINE after C_TIMER_POPPED
Jan 22 10:24:21 mysql-node1 crmd: [9247]: info: do_state_transition: All 2
cluster nodes are eligible to run resources.
Jan 22 10:24:21 mysql-node1 crmd: [9247]: debug: do_fsa_action: actions:trace:
// A_DC_TIMER_STOP
Jan 22 10:24:21 mysql-node1 crmd: [9247]: debug: do_fsa_action: actions:trace:
// A_INTEGRATE_TIMER_STOP
Jan 22 10:24:21 mysql-node1 crmd: [9247]: debug: do_fsa_action: actions:trace:
// A_FINALIZE_TIMER_STOP
Jan 22 10:24:21 mysql-node1 crmd: [9247]: debug: do_fsa_action: actions:trace:
// A_PE_INVOKE
Jan 22 10:24:21 mysql-node1 crmd: [9247]: info: do_pe_invoke: Query 207:
Requesting the current CIB: S_POLICY_ENGINE
Jan 22 10:24:21 mysql-node1 crmd: [9247]: info: do_pe_invoke_callback: Invoking
the PE: query=207, ref=pe_calc-dc-1295688261-214, seq=220, quorate=1
Jan 22 10:24:21 mysql-node1 pengine: [9246]: info: unpack_config: Startup
probes: enabled
Jan 22 10:24:21 mysql-node1 crmd: [9247]: debug: s_crmd_fsa: Processing
I_PE_SUCCESS: [ state=S_POLICY_ENGINE cause=C_IPC_MESSAGE
origin=handle_response ]
Jan 22 10:24:21 mysql-node1 pengine: [9246]: debug: unpack_config: STONITH
timeout: 60000
Jan 22 10:24:21 mysql-node1 crmd: [9247]: debug: do_fsa_action: actions:trace:
// A_LOG
Jan 22 10:24:21 mysql-node1 pengine: [9246]: debug: unpack_config: STONITH of
failed nodes is enabled
Jan 22 10:24:21 mysql-node1 crmd: [9247]: info: do_state_transition: State
transition S_POLICY_ENGINE -> S_TRANSITION_ENGINE [ input=I_PE_SUCCESS
cause=C_IPC_MESSAGE origin=handle_response ]
Jan 22 10:24:21 mysql-node1 pengine: [9246]: debug: unpack_config: Stop all
active resources: false
Jan 22 10:24:21 mysql-node1 crmd: [9247]: debug: do_fsa_action: actions:trace:
// A_DC_TIMER_STOP
Jan 22 10:24:21 mysql-node1 pengine: [9246]: debug: unpack_config: Cluster is
symmetric - resources can run anywhere by default
Jan 22 10:24:21 mysql-node1 crmd: [9247]: debug: do_fsa_action: actions:trace:
// A_INTEGRATE_TIMER_STOP
Jan 22 10:24:21 mysql-node1 pengine: [9246]: debug: unpack_config: Default
stickiness: 0
Jan 22 10:24:21 mysql-node1 crmd: [9247]: debug: do_fsa_action: actions:trace:
// A_FINALIZE_TIMER_STOP
Jan 22 10:24:21 mysql-node1 pengine: [9246]: notice: unpack_config: On loss of
CCM Quorum: Ignore
Jan 22 10:24:21 mysql-node1 crmd: [9247]: debug: do_fsa_action: actions:trace:
// A_TE_INVOKE
Jan 22 10:24:21 mysql-node1 pengine: [9246]: info: unpack_config: Node scores:
'red' = -INFINITY, 'yellow' = 0, 'green' = 0
Jan 22 10:24:21 mysql-node1 crmd: [9247]: info: unpack_graph: Unpacked
transition 114: 0 actions in 0 synapses
Jan 22 10:24:21 mysql-node1 pengine: [9246]: info: unpack_domains: Unpacking
domains
Jan 22 10:24:21 mysql-node1 crmd: [9247]: info: do_te_invoke: Processing graph
114 (ref=pe_calc-dc-1295688261-214) derived from
/var/lib/pengine/pe-input-659.bz2
Jan 22 10:24:21 mysql-node1 pengine: [9246]: info: determine_online_status:
Node mysql-node1 is online
Jan 22 10:24:21 mysql-node1 crmd: [9247]: debug: print_graph: ## Empty
transition graph ##
Jan 22 10:24:21 mysql-node1 pengine: [9246]: info: determine_online_status:
Node mysql-node2 is online
Jan 22 10:24:21 mysql-node1 crmd: [9247]: info: run_graph:
====================================================
Jan 22 10:24:21 mysql-node1 pengine: [9246]: info: find_clone: Internally
renamed drbd_mysql:0 on mysql-node2 to drbd_mysql:1
Jan 22 10:24:21 mysql-node1 crmd: [9247]: notice: run_graph: Transition 114
(Complete=0, Pending=0, Fired=0, Skipped=0, Incomplete=0,
Source=/var/lib/pengine/pe-input-659.bz2): Complete
Jan 22 10:24:21 mysql-node1 pengine: [9246]: info: find_clone: Internally
renamed stonith-device:0 on mysql-node2 to stonith-device:1
Jan 22 10:24:21 mysql-node1 crmd: [9247]: debug: print_graph: ## Empty
transition graph ##
Jan 22 10:24:21 mysql-node1 pengine: [9246]: info: find_clone: Internally
renamed notification:0 on mysql-node2 to notification:1
Jan 22 10:24:21 mysql-node1 crmd: [9247]: info: te_graph_trigger: Transition
114 is now complete
Jan 22 10:24:21 mysql-node1 pengine: [9246]: info: find_clone: Internally
renamed ping:0 on mysql-node2 to ping:1
Jan 22 10:24:21 mysql-node1 crmd: [9247]: debug: notify_crmd: Processing
transition completion in state S_TRANSITION_ENGINE
Jan 22 10:24:21 mysql-node1 pengine: [9246]: notice: clone_print: Master/Slave
Set: ms_drbd_mysql
Jan 22 10:24:21 mysql-node1 crmd: [9247]: info: notify_crmd: Transition 114
status: done - <null>
Jan 22 10:24:21 mysql-node1 pengine: [9246]: debug: native_active: Resource
drbd_mysql:0 active on mysql-node1
Jan 22 10:24:21 mysql-node1 crmd: [9247]: debug: s_crmd_fsa: Processing
I_TE_SUCCESS: [ state=S_TRANSITION_ENGINE cause=C_FSA_INTERNAL
origin=notify_crmd ]
Jan 22 10:24:21 mysql-node1 pengine: [9246]: debug: native_active: Resource
drbd_mysql:0 active on mysql-node1
Jan 22 10:24:21 mysql-node1 crmd: [9247]: debug: do_fsa_action: actions:trace:
// A_LOG
Jan 22 10:24:21 mysql-node1 pengine: [9246]: debug: native_active: Resource
drbd_mysql:1 active on mysql-node2
Jan 22 10:24:21 mysql-node1 crmd: [9247]: info: do_state_transition: State
transition S_TRANSITION_ENGINE -> S_IDLE [ input=I_TE_SUCCESS
cause=C_FSA_INTERNAL origin=notify_crmd ]
Jan 22 10:24:21 mysql-node1 pengine: [9246]: debug: native_active: Resource
drbd_mysql:1 active on mysql-node2
Jan 22 10:24:21 mysql-node1 crmd: [9247]: info: do_state_transition: Starting
PEngine Recheck Timer
Jan 22 10:24:21 mysql-node1 pengine: [9246]: notice: short_print: Masters:
[ mysql-node2 ]
Jan 22 10:24:21 mysql-node1 crmd: [9247]: debug: crm_timer_start: Started
PEngine Recheck Timer (I_PE_CALC:900000ms), src=400
Jan 22 10:24:21 mysql-node1 pengine: [9246]: notice: short_print: Slaves:
[ mysql-node1 ]
Jan 22 10:24:21 mysql-node1 crmd: [9247]: debug: do_fsa_action: actions:trace:
// A_DC_TIMER_STOP
Jan 22 10:24:21 mysql-node1 pengine: [9246]: notice: group_print: Resource
Group: mysql
Jan 22 10:24:21 mysql-node1 crmd: [9247]: debug: do_fsa_action: actions:trace:
// A_INTEGRATE_TIMER_STOP
Jan 22 10:24:21 mysql-node1 pengine: [9246]: notice: native_print:
fs_mysql (ocf::heartbeat:Filesystem): Started mysql-node2
Jan 22 10:24:21 mysql-node1 crmd: [9247]: debug: do_fsa_action: actions:trace:
// A_FINALIZE_TIMER_STOP
Jan 22 10:24:21 mysql-node1 pengine: [9246]: notice: native_print:
ip_mysql (ocf::heartbeat:IPaddr2): Started mysql-node2
Jan 22 10:24:21 mysql-node1 pengine: [9246]: notice: native_print: mysqld
(lsb:mysql): Started mysql-node2
....
...
Jan 22 10:25:46 mysql-node1 rchal: CPU frequency scaling is not supported by
your processor.
Jan 22 10:25:46 mysql-node1 rchal: boot with 'CPUFREQ=no' in to avoid this
warning.
Jan 22 10:25:47 mysql-node1 rchal: Cannot load cpufreq governors - No cpufreq
driver available
Jan 22 10:25:48 mysql-node1 ifup: lo
Jan 22 10:25:48 mysql-node1 ifup: lo
Jan 22 10:25:48 mysql-node1 ifup: IP address: 127.0.0.1/8
Jan 22 10:25:48 mysql-node1 ifup:
Jan 22 10:25:48 mysql-node1 ifup:
Jan 22 10:25:48 mysql-node1 ifup: IP address: 127.0.0.2/8
Jan 22 10:25:48 mysql-node1 ifup:
Jan 22 10:25:48 mysql-node1 ifup: eth0 device: Intel Corporation
82545EM Gigabit Ethernet Controller (Copper) (rev 01)
Jan 22 10:25:48 mysql-node1 ifup: eth0
Jan 22 10:25:48 mysql-node1 ifup: IP address: 192.168.96.11/24
Node2 debug log:
Jan 22 10:25:13 mysql-node2 attrd_updater: [21677]: info: attrd_lazy_update:
Connecting to cluster... 5 retries remaining
Jan 22 10:25:13 mysql-node2 attrd_updater: [21677]: debug:
init_client_ipc_comms_nodispatch: Attempting to talk on: /var/run/crm/attrd
Jan 22 10:25:13 mysql-node2 attrd_updater: [21677]: debug: attrd_update: Sent
update: pingd=1 for localhost
Jan 22 10:25:13 mysql-node2 attrd: [27805]: debug: attrd_local_callback: update
message from attrd_updater: pingd=1
Jan 22 10:25:13 mysql-node2 attrd: [27805]: debug: attrd_local_callback:
Supplied: 1, Current: 1, Stored: 1
Jan 22 10:25:17 mysql-node2 kernel: [104450.241909] block drbd0: PingAck did
not arrive in time.
Jan 22 10:25:17 mysql-node2 kernel: [104450.241919] block drbd0: peer(
Secondary -> Unknown ) conn( Connected -> NetworkFailure ) pdsk( UpToDate ->
DUnknown )
Jan 22 10:25:17 mysql-node2 kernel: [104450.241940] block drbd0: asender
terminated
Jan 22 10:25:17 mysql-node2 kernel: [104450.241943] block drbd0: Terminating
asender thread
Jan 22 10:25:17 mysql-node2 kernel: [104450.241989] block drbd0: short read
expecting header on sock: r=-512
Jan 22 10:25:17 mysql-node2 kernel: [104450.242055] block drbd0: Creating new
current UUID
Jan 22 10:25:17 mysql-node2 kernel: [104450.242316] block drbd0: Connection
closed
Jan 22 10:25:17 mysql-node2 kernel: [104450.242322] block drbd0: conn(
NetworkFailure -> Unconnected )
Jan 22 10:25:17 mysql-node2 kernel: [104450.242326] block drbd0: receiver
terminated
Jan 22 10:25:17 mysql-node2 kernel: [104450.242328] block drbd0: Restarting
receiver thread
Jan 22 10:25:17 mysql-node2 kernel: [104450.242330] block drbd0: receiver
(re)started
Jan 22 10:25:17 mysql-node2 kernel: [104450.242334] block drbd0: conn(
Unconnected -> WFConnection )
Jan 22 10:25:18 mysql-node2 corosync[27796]: [TOTEM ] The token was lost in
the OPERATIONAL state.
Jan 22 10:25:18 mysql-node2 corosync[27796]: [TOTEM ] A processor failed,
forming new configuration.
Jan 22 10:25:18 mysql-node2 corosync[27796]: [TOTEM ] Receive multicast
socket recv buffer size (262142 bytes).
Jan 22 10:25:18 mysql-node2 corosync[27796]: [TOTEM ] Transmit multicast
socket send buffer size (262142 bytes).
Jan 22 10:25:18 mysql-node2 corosync[27796]: [TOTEM ] entering GATHER state
from 2.
Jan 22 10:25:23 mysql-node2 lrmd: [27804]: debug: rsc:notification:1:14: monitor
Jan 22 10:25:23 mysql-node2 lrmd: [21678]: debug: perform_ra_op: resetting
scheduler class to SCHED_OTHER
Jan 22 10:25:23 mysql-node2 lrmd: [27804]: debug: RA output:
(notification:1:monitor:stdout) running
Jan 22 10:25:26 mysql-node2 corosync[27796]: [TOTEM ] entering GATHER state
from 0.
Jan 22 10:25:26 mysql-node2 corosync[27796]: [TOTEM ] Creating commit token
because I am the rep.
Jan 22 10:25:26 mysql-node2 corosync[27796]: [TOTEM ] Saving state aru 1d2
high seq received 1d2
Jan 22 10:25:26 mysql-node2 corosync[27796]: [TOTEM ] Storing new sequence id
for ring e0
Jan 22 10:25:26 mysql-node2 corosync[27796]: [TOTEM ] entering COMMIT state.
Jan 22 10:25:26 mysql-node2 corosync[27796]: [TOTEM ] got commit token
Jan 22 10:25:26 mysql-node2 corosync[27796]: [TOTEM ] entering RECOVERY state.
Jan 22 10:25:26 mysql-node2 corosync[27796]: [TOTEM ] TRANS [0] member
10.0.0.102:
Jan 22 10:25:26 mysql-node2 corosync[27796]: [TOTEM ] position [0] member
10.0.0.102:
Jan 22 10:25:26 mysql-node2 corosync[27796]: [TOTEM ] previous ring seq 220
rep 10.0.0.101
Jan 22 10:25:26 mysql-node2 corosync[27796]: [TOTEM ] aru 1d2 high delivered
1d2 received flag 1
Jan 22 10:25:26 mysql-node2 corosync[27796]: [TOTEM ] Did not need to
originate any messages in recovery.
Jan 22 10:25:26 mysql-node2 corosync[27796]: [TOTEM ] got commit token
Jan 22 10:25:26 mysql-node2 corosync[27796]: [TOTEM ] Sending initial ORF
token
Jan 22 10:25:26 mysql-node2 corosync[27796]: [TOTEM ] token retrans flag is 0
my set retrans flag0 retrans queue empty 1 count 0, aru 0
Jan 22 10:25:26 mysql-node2 corosync[27796]: [TOTEM ] install seq 0 aru 0
high seq received 0
Jan 22 10:25:26 mysql-node2 corosync[27796]: [TOTEM ] token retrans flag is 0
my set retrans flag0 retrans queue empty 1 count 1, aru 0
Jan 22 10:25:26 mysql-node2 corosync[27796]: [TOTEM ] install seq 0 aru 0
high seq received 0
Jan 22 10:25:26 mysql-node2 corosync[27796]: [TOTEM ] token retrans flag is 0
my set retrans flag0 retrans queue empty 1 count 2, aru 0
Jan 22 10:25:26 mysql-node2 corosync[27796]: [TOTEM ] install seq 0 aru 0
high seq received 0
Jan 22 10:25:26 mysql-node2 corosync[27796]: [TOTEM ] token retrans flag is 0
my set retrans flag0 retrans queue empty 1 count 3, aru 0
Jan 22 10:25:26 mysql-node2 corosync[27796]: [TOTEM ] install seq 0 aru 0
high seq received 0
Jan 22 10:25:26 mysql-node2 corosync[27796]: [TOTEM ] retrans flag count 4
token aru 0 install seq 0 aru 0 0
Jan 22 10:25:26 mysql-node2 corosync[27796]: [TOTEM ] Resetting old ring state
Jan 22 10:25:26 mysql-node2 corosync[27796]: [TOTEM ] recovery to regular 1-0
Jan 22 10:25:26 mysql-node2 corosync[27796]: [TOTEM ] Delivering to app 1d3
to 1d2
Jan 22 10:25:26 mysql-node2 corosync[27796]: [CLM ] CLM CONFIGURATION CHANGE
Jan 22 10:25:26 mysql-node2 corosync[27796]: [CLM ] New Configuration:
Jan 22 10:25:26 mysql-node2 corosync[27796]: [CLM ] r(0)
ip(10.0.0.102)
Jan 22 10:25:26 mysql-node2 corosync[27796]: [CLM ] Members Left:
Jan 22 10:25:26 mysql-node2 corosync[27796]: [CLM ] r(0)
ip(10.0.0.101)
Jan 22 10:25:26 mysql-node2 corosync[27796]: [CLM ] Members Joined:
Jan 22 10:25:26 mysql-node2 corosync[27796]: [EVT ] Evt conf change 1
Jan 22 10:25:26 mysql-node2 corosync[27796]: [EVT ] m 1, j 0 l 1
Jan 22 10:25:26 mysql-node2 corosync[27796]: [LCK ] [DEBUG]: lck_confchg_fn
Jan 22 10:25:26 mysql-node2 corosync[27796]: [MSG ] [DEBUG]: msg_confchg_fn
Jan 22 10:25:26 mysql-node2 corosync[27796]: [pcmk ] notice:
pcmk_peer_update: Transitional membership event on ring 224: memb=1, new=0,
lost=1
Jan 22 10:25:26 mysql-node2 corosync[27796]: [pcmk ] info: pcmk_peer_update:
memb: mysql-node2 1711276042
Jan 22 10:25:26 mysql-node2 corosync[27796]: [pcmk ] info: pcmk_peer_update:
lost: mysql-node1 1694498826
Jan 22 10:25:26 mysql-node2 corosync[27796]: [CLM ] CLM CONFIGURATION CHANGE
Jan 22 10:25:26 mysql-node2 corosync[27796]: [CLM ] New Configuration:
Jan 22 10:25:26 mysql-node2 corosync[27796]: [CLM ] r(0)
ip(10.0.0.102)
Jan 22 10:25:26 mysql-node2 corosync[27796]: [CLM ] Members Left:
Jan 22 10:25:26 mysql-node2 corosync[27796]: [CLM ] Members Joined:
Jan 22 10:25:26 mysql-node2 corosync[27796]: [EVT ] Evt conf change 0
Jan 22 10:25:26 mysql-node2 corosync[27796]: [EVT ] m 1, j 0 l 0
Jan 22 10:25:26 mysql-node2 corosync[27796]: [LCK ] [DEBUG]: lck_confchg_fn
Jan 22 10:25:26 mysql-node2 corosync[27796]: [MSG ] [DEBUG]: msg_confchg_fn
Jan 22 10:25:26 mysql-node2 corosync[27796]: [pcmk ] notice:
pcmk_peer_update: Stable membership event on ring 224: memb=1, new=0, lost=0
Jan 22 10:25:26 mysql-node2 corosync[27796]: [pcmk ] info: pcmk_peer_update:
MEMB: mysql-node2 1711276042
Jan 22 10:25:26 mysql-node2 corosync[27796]: [pcmk ] info:
ais_mark_unseen_peer_dead: Node mysql-node1 was not seen in the previous
transition
Jan 22 10:25:26 mysql-node2 corosync[27796]: [pcmk ] info: update_member:
Node 1694498826/mysql-node1 is now: lost
Jan 22 10:25:26 mysql-node2 corosync[27796]: [pcmk ] debug:
pcmk_peer_update: 1 nodes changed
Jan 22 10:25:26 mysql-node2 corosync[27796]: [pcmk ] info:
send_member_notification: Sending membership update 224 to 2 children
Jan 22 10:25:26 mysql-node2 corosync[27796]: [pcmk ] debug: send_cluster_id:
Local update: id=1711276042, born=220, seq=224
Jan 22 10:25:26 mysql-node2 corosync[27796]: [SYNC ] This node is within the
primary component and will provide service.
Jan 22 10:25:26 mysql-node2 corosync[27796]: [TOTEM ] entering OPERATIONAL
state.
Jan 22 10:25:26 mysql-node2 corosync[27796]: [TOTEM ] A processor joined or
left the membership and a new membership was formed.
Jan 22 10:25:26 mysql-node2 corosync[27796]: [TOTEM ] mcasted message added
to pending queue
Jan 22 10:25:26 mysql-node2 corosync[27796]: [TOTEM ] mcasted message added
to pending queue
Jan 22 10:25:26 mysql-node2 corosync[27796]: [TOTEM ] Delivering 0 to 2
Jan 22 10:25:26 mysql-node2 corosync[27796]: [TOTEM ] Delivering MCAST
message with seq 1 to pending delivery queue
Jan 22 10:25:26 mysql-node2 corosync[27796]: [pcmk ] debug:
pcmk_cluster_id_callback: Node update: mysql-node2 (1.1.2)
Jan 22 10:25:26 mysql-node2 corosync[27796]: [SYNC ] confchg entries 1
Jan 22 10:25:26 mysql-node2 corosync[27796]: [SYNC ] Barrier Start Received
From 1711276042
Jan 22 10:25:26 mysql-node2 corosync[27796]: [SYNC ] Barrier completion
status for nodeid 1711276042 = 1.
Jan 22 10:25:26 mysql-node2 corosync[27796]: [SYNC ] Synchronization barrier
completed
Jan 22 10:25:26 mysql-node2 corosync[27796]: [SYNC ] Synchronization actions
starting for (openais cluster membership service B.01.01)
...
...
Viele Grüße aus Neustadt,
Thomas Fachtan
Database Administrator
Fon: +49 9602 9442-301
Fax: +49 9602 9442-309
E-Mail: [email protected]
.................................................
motiondrive AG
Im Wiesengrund 1
92660 Neustadt
http://www.motiondrive.ag
Vorstandsvorsitzender: Werner A. Krachtus
Aufsichtsratsvorsitzender: Dipl.-Kfm. Ralf Barth
Handelsregister Weiden: HRB 2817
_______________________________________________
Linux-HA mailing list
[email protected]
http://lists.linux-ha.org/mailman/listinfo/linux-ha
See also: http://linux-ha.org/ReportingProblems
