2 Node HB Cluster
Heartbeat 2.0.7
Debian Linux 4.0 (Etch).
Linux Kernel 2.6.18
Node Names: lvs-test-1, lvs-test-2
I'm trying to test a very basic HA config in which an aliased IP address
is migrated between 2 boxes. Since I'm just testing HA there is no load
on the boxes, and I know the network is reliable as this is simply an
isolated test network.
Everything appears to work correctly when the heartbeat processes can
reliably communicate from the get-go. To be more precise if both systems
are booted at the same time only one get assigned the aliases IP and the
other operates in a stand-by state. If I bring down the active server or
use hb_takeover to swing the resource all works well.
The problem arises after severing the network connection (booting both
systems when the ethernet switch used for the heartbeat network is
powered off) Both systems activate the same IP address as expected
(split brain condition). Once the heartbeat is re-established I would
expect one system, to relinquish the aliased IP and go to a stand-by
state. This does not occur. The logfiles don't seem to exhibit any
errors. If I do a 'cl_status rscstatus' on each system one outputs
'all', the other outputs 'transition'. The system in 'transition' never
relinquishes the aliased IP and the heartbeat process appears to be hung
waiting for some event as it will not repond to a 'heartbeat -k'. This
also prevents the system from shutting down since Debian's shutdown
script ends up running 'heartbeat -k'.
You'll note that in the logfiles below that one system always outputs
more "debug: hist..." entries. This happens the same way every time.
After the "all clients are now resumed" line.. that's it, one system is
in transition forever:(
Any and all help is greatly appreciated,
Wayne
------------------------------------------------------------------------
I have included the ha.cf, haresource and the logfiles from both
systems.
ha.cf files
---
# lvs-test-1
use_logd off
logfacility daemon
keepalive 2
initdead 20
deadtime 10
warntime 5
auto_failback off
msgfmt netstring
ucast eth2 192.168.199.10
node lvs-test-1
node lvs-test-2
# EOF
# lvs-test-2
use_logd off
logfacility daemon
keepalive 2
initdead 20
deadtime 10
warntime 5
auto_failback off
msgfmt netstring
ucast eth2 192.168.199.11
node lvs-test-1
node lvs-test-2
# EOF
haresource file, identical on both systems
---
lvs-test-1 IPaddr2::10.204.6.25/24/eth0
# EOF
lvs-test-1 logfile
---
Oct 30 19:13:01 lvs-test-1 heartbeat: [3169]: debug: add_option(keepalive,2)
Oct 30 19:13:01 lvs-test-1 heartbeat: [3169]: debug: add_option(initdead,20)
Oct 30 19:13:01 lvs-test-1 heartbeat: [3169]: debug: add_option(deadtime,10)
Oct 30 19:13:01 lvs-test-1 heartbeat: [3169]: debug: add_option(warntime,5)
Oct 30 19:13:01 lvs-test-1 heartbeat: [3169]: debug:
add_option(auto_failback,off)
Oct 30 19:13:01 lvs-test-1 heartbeat: [3169]: debug:
add_option(msgfmt,netstring)
Oct 30 19:13:01 lvs-test-1 heartbeat: [3169]: debug: add_option(node,lvs-test-1)
Oct 30 19:13:01 lvs-test-1 heartbeat: [3169]: debug: add_option(node,lvs-test-2)
Oct 30 19:13:01 lvs-test-1 heartbeat: [3169]: debug: uid=hacluster, gid=<null>
Oct 30 19:13:01 lvs-test-1 heartbeat: [3169]: debug: uid=hacluster, gid=<null>
Oct 30 19:13:01 lvs-test-1 heartbeat: [3169]: debug: uid=<null>, gid=haclient
Oct 30 19:13:01 lvs-test-1 heartbeat: [3169]: debug: uid=root, gid=<null>
Oct 30 19:13:01 lvs-test-1 heartbeat: [3169]: debug: uid=<null>, gid=haclient
Oct 30 19:13:01 lvs-test-1 heartbeat: [3169]: debug: Beginning authentication
parsing
Oct 30 19:13:01 lvs-test-1 heartbeat: [3169]: debug: 16 max authentication
methods
Oct 30 19:13:01 lvs-test-1 heartbeat: [3169]: debug: Keyfile opened
Oct 30 19:13:01 lvs-test-1 heartbeat: [3169]: debug: Keyfile perms OK
Oct 30 19:13:01 lvs-test-1 heartbeat: [3169]: debug: 16 max authentication
methods
Oct 30 19:13:01 lvs-test-1 heartbeat: [3169]: debug: Found authentication
method [crc]
Oct 30 19:13:01 lvs-test-1 heartbeat: [3169]: info: AUTH: i=1: key = 0x5d5b18,
auth=0x2aae5b426550, authname=crc
Oct 30 19:13:01 lvs-test-1 heartbeat: [3169]: debug: Outbound signing method is
1
Oct 30 19:13:01 lvs-test-1 heartbeat: [3169]: debug: Authentication parsing
complete [1]
Oct 30 19:13:01 lvs-test-1 heartbeat: [3169]: debug:
add_option(cluster,linux-ha)
Oct 30 19:13:01 lvs-test-1 heartbeat: [3169]: debug: add_option(hopfudge,1)
Oct 30 19:13:01 lvs-test-1 heartbeat: [3169]: debug: add_option(baud,19200)
Oct 30 19:13:01 lvs-test-1 heartbeat: [3169]: debug:
add_option(hbgenmethod,file)
Oct 30 19:13:01 lvs-test-1 heartbeat: [3169]: debug: add_option(realtime,true)
Oct 30 19:13:01 lvs-test-1 heartbeat: [3169]: debug:
add_option(conn_logd_time,60)
Oct 30 19:13:01 lvs-test-1 heartbeat: [3169]: debug:
add_option(log_badpack,true)
Oct 30 19:13:01 lvs-test-1 heartbeat: [3169]: debug: add_option(coredumps,true)
Oct 30 19:13:01 lvs-test-1 heartbeat: [3169]: WARN: Core dumps could be lost if
multiple dumps occur
Oct 30 19:13:01 lvs-test-1 heartbeat: [3169]: WARN: Consider setting
/proc/sys/kernel/core_uses_pid (or equivalent) to 1 for maximum supportability
Oct 30 19:13:01 lvs-test-1 heartbeat: [3169]: debug: add_option(crm,false)
Oct 30 19:13:01 lvs-test-1 heartbeat: [3169]: debug: add_option(autojoin,none)
Oct 30 19:13:01 lvs-test-1 heartbeat: [3169]: debug: add_option(uuidfrom,file)
Oct 30 19:13:01 lvs-test-1 heartbeat: [3169]: debug:
add_option(compression,zlib)
Oct 30 19:13:01 lvs-test-1 heartbeat: [3169]: debug:
add_option(compression_threshold,2)
Oct 30 19:13:01 lvs-test-1 heartbeat: [3169]: debug:
add_option(traditional_compression,no)
Oct 30 19:13:01 lvs-test-1 heartbeat: [3169]: debug:
add_option(max_rexmit_delay,250)
Oct 30 19:13:01 lvs-test-1 heartbeat: [3169]: debug: Setting max_rexmit_delay
to 250 ms
Oct 30 19:13:01 lvs-test-1 heartbeat: [3169]: debug:
add_option(record_config_changes,on)
Oct 30 19:13:01 lvs-test-1 heartbeat: [3169]: debug:
add_option(record_pengine_inputs,on)
Oct 30 19:13:01 lvs-test-1 heartbeat: [3169]: debug:
add_option(enable_config_writes,on)
Oct 30 19:13:01 lvs-test-1 heartbeat: [3169]: debug: add_option(memreserve,6500)
Oct 30 19:13:01 lvs-test-1 heartbeat: [3169]: debug: add_option(debug,1)
Oct 30 19:13:01 lvs-test-1 heartbeat: [3169]: WARN: Logging daemon is disabled
--enabling logging daemon is recommended
Oct 30 19:13:01 lvs-test-1 heartbeat: [3169]: info: **************************
Oct 30 19:13:01 lvs-test-1 heartbeat: [3169]: info: Configuration validated.
Starting heartbeat 2.0.7
Oct 30 19:13:01 lvs-test-1 heartbeat: [3169]: debug: HA configuration OK.
Heartbeat starting.
Oct 30 19:13:01 lvs-test-1 heartbeat: [3170]: info: heartbeat: version 2.0.7
Oct 30 19:13:03 lvs-test-1 heartbeat: [3170]: info: Heartbeat generation: 13
Oct 30 19:13:04 lvs-test-1 heartbeat: [3170]: debug: uuid
is:31c440a3-cc9c-487b-b07e-53ad7038d550
Oct 30 19:13:04 lvs-test-1 heartbeat: [3170]: info: G_main_add_TriggerHandler:
Added signal manual handler
Oct 30 19:13:04 lvs-test-1 heartbeat: [3170]: info: G_main_add_TriggerHandler:
Added signal manual handler
Oct 30 19:13:04 lvs-test-1 heartbeat: [3170]: info: Removing
/var/run/heartbeat/rsctmp failed, recreating.
Oct 30 19:13:04 lvs-test-1 heartbeat: [3170]: debug: opening ucast eth2 (UDP/IP
unicast)
Oct 30 19:13:04 lvs-test-1 heartbeat: [3170]: debug: FIFO process pid: 3240
Oct 30 19:13:04 lvs-test-1 heartbeat: [3170]: info: glib: ucast: write socket
priority set to IPTOS_LOWDELAY on eth2
Oct 30 19:13:04 lvs-test-1 heartbeat: [3170]: info: glib: ucast: bound send
socket to device: eth2
Oct 30 19:13:04 lvs-test-1 heartbeat: [3170]: info: glib: ucast: bound receive
socket to device: eth2
Oct 30 19:13:04 lvs-test-1 heartbeat: [3170]: info: glib: ucast: started on
port 694 interface eth2 to 192.168.199.10
Oct 30 19:13:04 lvs-test-1 heartbeat: [3170]: debug: write process pid: 3241
Oct 30 19:13:04 lvs-test-1 heartbeat: [3170]: debug: read child process pid:
3242
Oct 30 19:13:04 lvs-test-1 heartbeat: [3170]: info: G_main_add_SignalHandler:
Added signal handler for signal 17
Oct 30 19:13:04 lvs-test-1 heartbeat: [3170]: debug: Limiting CPU: 42 CPU
seconds every 60000 milliseconds
Oct 30 19:13:04 lvs-test-1 heartbeat: [3170]: debug: pid 3170 locked in memory.
Oct 30 19:13:04 lvs-test-1 heartbeat: [3170]: debug: Waiting for child
processes to start
Oct 30 19:13:04 lvs-test-1 heartbeat: [3170]: info: Local status now set to:
'up'
Oct 30 19:13:04 lvs-test-1 heartbeat: [3170]: debug: All your child process are
belong to us
Oct 30 19:13:04 lvs-test-1 heartbeat: [3170]: debug: Starting local status
message @ 2000 ms intervals
Oct 30 19:13:05 lvs-test-1 heartbeat: [3240]: debug: pid 3240 locked in memory.
Oct 30 19:13:05 lvs-test-1 heartbeat: [3240]: debug: Limiting CPU: 6 CPU
seconds every 60000 milliseconds
Oct 30 19:13:05 lvs-test-1 heartbeat: [3242]: debug: pid 3242 locked in memory.
Oct 30 19:13:05 lvs-test-1 heartbeat: [3241]: debug: pid 3241 locked in memory.
Oct 30 19:13:05 lvs-test-1 heartbeat: [3242]: debug: Limiting CPU: 6 CPU
seconds every 60000 milliseconds
Oct 30 19:13:05 lvs-test-1 heartbeat: [3241]: debug: Limiting CPU: 24 CPU
seconds every 60000 milliseconds
Oct 30 19:13:24 lvs-test-1 heartbeat: [3170]: WARN: node lvs-test-2: is dead
Oct 30 19:13:24 lvs-test-1 heartbeat: [3170]: info: Comm_now_up(): updating
status to active
Oct 30 19:13:24 lvs-test-1 heartbeat: [3170]: info: Local status now set to:
'active'
Oct 30 19:13:24 lvs-test-1 heartbeat: [3170]: debug: Sending local starting
msg: resourcestate = 0
Oct 30 19:13:24 lvs-test-1 heartbeat: [3170]: debug: hb_rsc_isstable:
ResourceMgmt_child_count: 0, other_is_stable: 0, takeover_in_progress: 0,
going_standby: 0, standby running(ms): 0, resourcestate: 0
Oct 30 19:13:25 lvs-test-1 heartbeat: [3170]: debug: CreateInitialFilter:
hb_takeover
Oct 30 19:13:25 lvs-test-1 heartbeat: [3170]: debug: CreateInitialFilter: status
Oct 30 19:13:25 lvs-test-1 heartbeat: [3170]: debug: CreateInitialFilter:
ask_resources
Oct 30 19:13:25 lvs-test-1 heartbeat: [3170]: debug: CreateInitialFilter:
ip-request-resp
Oct 30 19:13:25 lvs-test-1 heartbeat: [3170]: debug: CreateInitialFilter:
ip-request
Oct 30 19:13:25 lvs-test-1 heartbeat: [3170]: WARN: No STONITH device
configured.
Oct 30 19:13:25 lvs-test-1 heartbeat: [3170]: WARN: Shared disks are not
protected.
Oct 30 19:13:25 lvs-test-1 heartbeat: [3170]: info: Resources being acquired
from lvs-test-2.
Oct 30 19:13:25 lvs-test-1 heartbeat: [3170]: debug: StartNextRemoteRscReq() -
calling hook
Oct 30 19:13:25 lvs-test-1 heartbeat: [3170]: debug: notify_world: invoking
harc: OLD status: active
Oct 30 19:13:25 lvs-test-1 heartbeat: [3170]: debug: Process [status] started
pid 3244
Oct 30 19:13:25 lvs-test-1 heartbeat: [3170]: debug: Starting notify process
[status]
Oct 30 19:13:25 lvs-test-1 heartbeat: [3170]: debug: takeover_from_node: other
now stable
Oct 30 19:13:25 lvs-test-1 heartbeat: [3170]: debug: Process
[req_our_resources] started pid 3245
Oct 30 19:13:25 lvs-test-1 heartbeat: [3244]: debug: notify_world: setting
SIGCHLD Handler to SIG_DFL
Oct 30 19:13:25 lvs-test-1 heartbeat: [3244]: debug: notify_world: Running harc
status
Oct 30 19:13:25 lvs-test-1 heartbeat: [3245]: debug:
req_our_resources(/usr/lib/heartbeat/ResourceManager listkeys lvs-test-1)
Oct 30 19:13:25 lvs-test-1 heartbeat: [3245]: debug: req_our_resources():
running [/usr/lib/heartbeat/req_resource IPaddr2::10.204.6.25/24/eth0]
Oct 30 19:13:25 lvs-test-1 harc[3244]: info: Running /etc/ha.d/rc.d/status
status
Oct 30 19:13:25 lvs-test-1 mach_down[3275]: info: /usr/lib/heartbeat/mach_down:
nice_failback: foreign resources acquired
Oct 30 19:13:25 lvs-test-1 mach_down[3275]: info: mach_down takeover complete
for node lvs-test-2.
Oct 30 19:13:25 lvs-test-1 heartbeat: [3170]: info: AnnounceTakeover(local 0,
foreign 1, reason 'T_RESOURCES' (0))
Oct 30 19:13:25 lvs-test-1 heartbeat: [3170]: info: AnnounceTakeover(local 1,
foreign 1, reason 'T_RESOURCES(us)' (0))
Oct 30 19:13:25 lvs-test-1 heartbeat: [3170]: info: Initial resource
acquisition complete (T_RESOURCES(us))
Oct 30 19:13:25 lvs-test-1 heartbeat: [3170]: info: mach_down takeover complete.
Oct 30 19:13:25 lvs-test-1 heartbeat: [3170]: debug: process_resources(3):
other now stable
Oct 30 19:13:25 lvs-test-1 heartbeat: [3170]: info: AnnounceTakeover(local 1,
foreign 1, reason 'mach_down' (1))
Oct 30 19:13:25 lvs-test-1 heartbeat: [3170]: info: STATE 1 => 3
Oct 30 19:13:25 lvs-test-1 heartbeat: [3170]: debug: hb_rsc_isstable:
ResourceMgmt_child_count: 2, other_is_stable: 1, takeover_in_progress: 0,
going_standby: 0, standby running(ms): 0, resourcestate: 3
Oct 30 19:13:25 lvs-test-1 heartbeat: [3170]: info: Exiting status process 3244
returned rc 0.
Oct 30 19:13:25 lvs-test-1 heartbeat: [3170]: debug: RscMgmtProc 'status'
exited code 0
Oct 30 19:13:25 lvs-test-1 heartbeat: [3170]: debug: StartNextRemoteRscReq():
child count 1
Oct 30 19:13:25 lvs-test-1 IPaddr2[3282]: INFO: IPaddr2 Resource is stopped
Oct 30 19:13:25 lvs-test-1 req_resource[3258]: debug: in
/usr/lib/heartbeat/req_resource IPaddr2::10.204.6.25/24/eth0
Oct 30 19:13:25 lvs-test-1 req_resource[3258]: debug: dont_ask: yes
nice_failback: yes
Oct 30 19:13:25 lvs-test-1 heartbeat: [3245]: info: 1 local resources from
[/usr/lib/heartbeat/ResourceManager listkeys lvs-test-1]
Oct 30 19:13:25 lvs-test-1 heartbeat: [3245]: info: Local Resource acquisition
completed.
Oct 30 19:13:25 lvs-test-1 heartbeat: [3245]: debug: Sending hold resources
msg: all, stable=1 # req_our_resources()
Oct 30 19:13:25 lvs-test-1 heartbeat: [3245]: info: FIFO message [type
resource] written rc=108
Oct 30 19:13:25 lvs-test-1 heartbeat: [3170]: debug: StartNextRemoteRscReq():
child count 1
Oct 30 19:13:25 lvs-test-1 heartbeat: [3170]: info: AnnounceTakeover(local 1,
foreign 1, reason 'T_RESOURCES(us)' (1))
Oct 30 19:13:25 lvs-test-1 heartbeat: [3170]: debug: hb_rsc_isstable:
ResourceMgmt_child_count: 1, other_is_stable: 1, takeover_in_progress: 0,
going_standby: 0, standby running(ms): 0, resourcestate: 3
Oct 30 19:13:25 lvs-test-1 heartbeat: [3170]: info: Exiting req_our_resources
process 3245 returned rc 0.
Oct 30 19:13:25 lvs-test-1 heartbeat: [3170]: debug: RscMgmtProc
'req_our_resources' exited code 0
Oct 30 19:13:25 lvs-test-1 heartbeat: [3170]: info: AnnounceTakeover(local 1,
foreign 1, reason 'req_our_resources' (1))
Oct 30 19:13:25 lvs-test-1 heartbeat: [3170]: debug: StartNextRemoteRscReq() -
calling hook
Oct 30 19:13:25 lvs-test-1 heartbeat: [3170]: debug: notify_world: invoking
harc: OLD status: active
Oct 30 19:13:25 lvs-test-1 heartbeat: [3170]: debug: Process [ip-request-resp]
started pid 3431
Oct 30 19:13:25 lvs-test-1 heartbeat: [3170]: debug: Starting notify process
[ip-request-resp]
Oct 30 19:13:25 lvs-test-1 heartbeat: [3431]: debug: notify_world: setting
SIGCHLD Handler to SIG_DFL
Oct 30 19:13:25 lvs-test-1 heartbeat: [3431]: debug: notify_world: Running harc
ip-request-resp
Oct 30 19:13:25 lvs-test-1 harc[3431]: info: Running
/etc/ha.d/rc.d/ip-request-resp ip-request-resp
Oct 30 19:13:25 lvs-test-1 ip-request-resp[3431]: received ip-request-resp
IPaddr2::10.204.6.25/24/eth0 OK yes
Oct 30 19:13:25 lvs-test-1 ResourceManager[3446]: info: Acquiring resource
group: lvs-test-1 IPaddr2::10.204.6.25/24/eth0
Oct 30 19:13:25 lvs-test-1 IPaddr2[3470]: INFO: IPaddr2 Resource is stopped
Oct 30 19:13:25 lvs-test-1 ResourceManager[3446]: info: Running
/etc/ha.d/resource.d/IPaddr2 10.204.6.25/24/eth0 start
Oct 30 19:13:25 lvs-test-1 ResourceManager[3446]: debug: Starting
/etc/ha.d/resource.d/IPaddr2 10.204.6.25/24/eth0 start
Oct 30 19:13:25 lvs-test-1 IPaddr2[3684]: INFO: /sbin/ip -f inet addr add
10.204.6.25/24 brd 10.204.6.255 dev eth0
Oct 30 19:13:25 lvs-test-1 IPaddr2[3684]: INFO: /sbin/ip link set eth0 up
Oct 30 19:13:25 lvs-test-1 IPaddr2[3684]: INFO: /usr/lib/heartbeat/send_arp -i
200 -r 5 -p /var/run/heartbeat/rsctmp/send_arp/send_arp-10.204.6.25 eth0
10.204.6.25 auto 10.204.6.25 ffffffffffff
Oct 30 19:13:25 lvs-test-1 IPaddr2[3602]: INFO: IPaddr2 Success
Oct 30 19:13:25 lvs-test-1 ResourceManager[3446]: debug:
/etc/ha.d/resource.d/IPaddr2 10.204.6.25/24/eth0 start done. RC=0
Oct 30 19:13:25 lvs-test-1 heartbeat: [3170]: info: Exiting ip-request-resp
process 3431 returned rc 0.
Oct 30 19:13:25 lvs-test-1 heartbeat: [3170]: debug: RscMgmtProc
'ip-request-resp' exited code 0
Oct 30 19:13:25 lvs-test-1 heartbeat: [3170]: info: AnnounceTakeover(local 1,
foreign 1, reason 'ip-request-resp' (1))
Oct 30 19:13:35 lvs-test-1 heartbeat: [3170]: info: Local Resource acquisition
completed. (none)
Oct 30 19:13:35 lvs-test-1 heartbeat: [3170]: info: local resource transition
completed.
Oct 30 19:13:35 lvs-test-1 heartbeat: [3170]: debug: Sending hold resources
msg: all, stable=1 # <none>
Oct 30 19:13:35 lvs-test-1 heartbeat: [3170]: info: AnnounceTakeover(local 1,
foreign 1, reason 'T_RESOURCES(us)' (1))
Oct 30 19:13:35 lvs-test-1 heartbeat: [3170]: debug: hb_rsc_isstable:
ResourceMgmt_child_count: 0, other_is_stable: 1, takeover_in_progress: 0,
going_standby: 0, standby running(ms): 0, resourcestate: 4
Oct 30 19:13:35 lvs-test-1 heartbeat: [3170]: debug: hb_rsc_isstable:
ResourceMgmt_child_count: 0, other_is_stable: 1, takeover_in_progress: 0,
going_standby: 0, standby running(ms): 0, resourcestate: 4
Oct 30 19:16:27 lvs-test-1 heartbeat: [3170]: info: Link lvs-test-2:eth2 up.
Oct 30 19:16:27 lvs-test-1 heartbeat: [3170]: info: Status update for node
lvs-test-2: status active
Oct 30 19:16:27 lvs-test-1 heartbeat: [3170]: debug: Status seqno: 104 msgtime:
1225408580
Oct 30 19:16:27 lvs-test-1 heartbeat: [3170]: debug: StartNextRemoteRscReq() -
calling hook
Oct 30 19:16:27 lvs-test-1 heartbeat: [3170]: debug: notify_world: invoking
harc: OLD status: active
Oct 30 19:16:27 lvs-test-1 heartbeat: [3170]: debug: Process [status] started
pid 3746
Oct 30 19:16:27 lvs-test-1 heartbeat: [3170]: debug: Starting notify process
[status]
Oct 30 19:16:27 lvs-test-1 heartbeat: [3746]: debug: notify_world: setting
SIGCHLD Handler to SIG_DFL
Oct 30 19:16:27 lvs-test-1 heartbeat: [3746]: debug: notify_world: Running harc
status
Oct 30 19:16:27 lvs-test-1 harc[3746]: info: Running /etc/ha.d/rc.d/status
status
Oct 30 19:16:27 lvs-test-1 heartbeat: [3170]: info: Exiting status process 3746
returned rc 0.
Oct 30 19:16:27 lvs-test-1 heartbeat: [3170]: debug: RscMgmtProc 'status'
exited code 0
Oct 30 19:16:30 lvs-test-1 heartbeat: [3170]: info: all clients are now paused
Oct 30 19:16:30 lvs-test-1 heartbeat: [3170]: debug: hist->ackseq =13
Oct 30 19:16:30 lvs-test-1 heartbeat: [3170]: debug: hist->lowseq =12,
hist->hiseq=114
Oct 30 19:16:30 lvs-test-1 heartbeat: [3170]: debug:
Oct 30 19:16:32 lvs-test-1 heartbeat: [3170]: debug: hist->ackseq =13
Oct 30 19:16:32 lvs-test-1 heartbeat: [3170]: debug: hist->lowseq =12,
hist->hiseq=115
Oct 30 19:16:32 lvs-test-1 heartbeat: [3170]: debug:
Oct 30 19:16:34 lvs-test-1 heartbeat: [3170]: debug: hist->ackseq =13
Oct 30 19:16:34 lvs-test-1 heartbeat: [3170]: debug: hist->lowseq =12,
hist->hiseq=116
Oct 30 19:16:34 lvs-test-1 heartbeat: [3170]: debug:
Oct 30 19:16:35 lvs-test-1 heartbeat: [3170]: info: all clients are now resumed
# EOF
lvs-test-2 logfile
---
Oct 30 23:13:08 lvs-test-2 heartbeat: [3248]: debug: add_option(keepalive,2)
Oct 30 23:13:08 lvs-test-2 heartbeat: [3248]: debug: add_option(initdead,20)
Oct 30 23:13:08 lvs-test-2 heartbeat: [3248]: debug: add_option(deadtime,10)
Oct 30 23:13:08 lvs-test-2 heartbeat: [3248]: debug: add_option(warntime,5)
Oct 30 23:13:08 lvs-test-2 heartbeat: [3248]: debug:
add_option(auto_failback,off)
Oct 30 23:13:08 lvs-test-2 heartbeat: [3248]: debug:
add_option(msgfmt,netstring)
Oct 30 23:13:08 lvs-test-2 heartbeat: [3248]: debug: add_option(node,lvs-test-1)
Oct 30 23:13:08 lvs-test-2 heartbeat: [3248]: debug: add_option(node,lvs-test-2)
Oct 30 23:13:08 lvs-test-2 heartbeat: [3248]: debug: uid=hacluster, gid=<null>
Oct 30 23:13:08 lvs-test-2 heartbeat: [3248]: debug: uid=hacluster, gid=<null>
Oct 30 23:13:08 lvs-test-2 heartbeat: [3248]: debug: uid=<null>, gid=haclient
Oct 30 23:13:08 lvs-test-2 heartbeat: [3248]: debug: uid=root, gid=<null>
Oct 30 23:13:08 lvs-test-2 heartbeat: [3248]: debug: uid=<null>, gid=haclient
Oct 30 23:13:08 lvs-test-2 heartbeat: [3248]: debug: Beginning authentication
parsing
Oct 30 23:13:08 lvs-test-2 heartbeat: [3248]: debug: 16 max authentication
methods
Oct 30 23:13:08 lvs-test-2 heartbeat: [3248]: debug: Keyfile opened
Oct 30 23:13:08 lvs-test-2 heartbeat: [3248]: debug: Keyfile perms OK
Oct 30 23:13:08 lvs-test-2 heartbeat: [3248]: debug: 16 max authentication
methods
Oct 30 23:13:08 lvs-test-2 heartbeat: [3248]: debug: Found authentication
method [crc]
Oct 30 23:13:08 lvs-test-2 heartbeat: [3248]: info: AUTH: i=1: key = 0x5d5218,
auth=0x2adad07a7550, authname=crc
Oct 30 23:13:08 lvs-test-2 heartbeat: [3248]: debug: Outbound signing method is
1
Oct 30 23:13:08 lvs-test-2 heartbeat: [3248]: debug: Authentication parsing
complete [1]
Oct 30 23:13:08 lvs-test-2 heartbeat: [3248]: debug:
add_option(cluster,linux-ha)
Oct 30 23:13:08 lvs-test-2 heartbeat: [3248]: debug: add_option(hopfudge,1)
Oct 30 23:13:08 lvs-test-2 heartbeat: [3248]: debug: add_option(baud,19200)
Oct 30 23:13:08 lvs-test-2 heartbeat: [3248]: debug:
add_option(hbgenmethod,file)
Oct 30 23:13:08 lvs-test-2 heartbeat: [3248]: debug: add_option(realtime,true)
Oct 30 23:13:08 lvs-test-2 heartbeat: [3248]: debug:
add_option(conn_logd_time,60)
Oct 30 23:13:08 lvs-test-2 heartbeat: [3248]: debug:
add_option(log_badpack,true)
Oct 30 23:13:08 lvs-test-2 heartbeat: [3248]: debug: add_option(coredumps,true)
Oct 30 23:13:08 lvs-test-2 heartbeat: [3248]: WARN: Core dumps could be lost if
multiple dumps occur
Oct 30 23:13:08 lvs-test-2 heartbeat: [3248]: WARN: Consider setting
/proc/sys/kernel/core_uses_pid (or equivalent) to 1 for maximum supportability
Oct 30 23:13:08 lvs-test-2 heartbeat: [3248]: debug: add_option(crm,false)
Oct 30 23:13:08 lvs-test-2 heartbeat: [3248]: debug: add_option(autojoin,none)
Oct 30 23:13:08 lvs-test-2 heartbeat: [3248]: debug: add_option(uuidfrom,file)
Oct 30 23:13:08 lvs-test-2 heartbeat: [3248]: debug:
add_option(compression,zlib)
Oct 30 23:13:08 lvs-test-2 heartbeat: [3248]: debug:
add_option(compression_threshold,2)
Oct 30 23:13:08 lvs-test-2 heartbeat: [3248]: debug:
add_option(traditional_compression,no)
Oct 30 23:13:08 lvs-test-2 heartbeat: [3248]: debug:
add_option(max_rexmit_delay,250)
Oct 30 23:13:08 lvs-test-2 heartbeat: [3248]: debug: Setting max_rexmit_delay
to 250 ms
Oct 30 23:13:08 lvs-test-2 heartbeat: [3248]: debug:
add_option(record_config_changes,on)
Oct 30 23:13:08 lvs-test-2 heartbeat: [3248]: debug:
add_option(record_pengine_inputs,on)
Oct 30 23:13:08 lvs-test-2 heartbeat: [3248]: debug:
add_option(enable_config_writes,on)
Oct 30 23:13:08 lvs-test-2 heartbeat: [3248]: debug: add_option(memreserve,6500)
Oct 30 23:13:08 lvs-test-2 heartbeat: [3248]: debug: add_option(debug,1)
Oct 30 23:13:08 lvs-test-2 heartbeat: [3248]: WARN: Logging daemon is disabled
--enabling logging daemon is recommended
Oct 30 23:13:08 lvs-test-2 heartbeat: [3248]: info: **************************
Oct 30 23:13:08 lvs-test-2 heartbeat: [3248]: info: Configuration validated.
Starting heartbeat 2.0.7
Oct 30 23:13:08 lvs-test-2 heartbeat: [3248]: debug: HA configuration OK.
Heartbeat starting.
Oct 30 23:13:08 lvs-test-2 heartbeat: [3249]: info: heartbeat: version 2.0.7
Oct 30 23:13:09 lvs-test-2 heartbeat: [3249]: info: Heartbeat generation: 11
Oct 30 23:13:10 lvs-test-2 heartbeat: [3249]: debug: uuid
is:6097c427-ad10-4ead-95eb-179b0a3ac540
Oct 30 23:13:10 lvs-test-2 heartbeat: [3249]: info: G_main_add_TriggerHandler:
Added signal manual handler
Oct 30 23:13:10 lvs-test-2 heartbeat: [3249]: info: G_main_add_TriggerHandler:
Added signal manual handler
Oct 30 23:13:10 lvs-test-2 heartbeat: [3249]: info: Removing
/var/run/heartbeat/rsctmp failed, recreating.
Oct 30 23:13:10 lvs-test-2 heartbeat: [3249]: debug: opening ucast eth2 (UDP/IP
unicast)
Oct 30 23:13:10 lvs-test-2 heartbeat: [3249]: debug: FIFO process pid: 3314
Oct 30 23:13:10 lvs-test-2 heartbeat: [3249]: info: glib: ucast: write socket
priority set to IPTOS_LOWDELAY on eth2
Oct 30 23:13:10 lvs-test-2 heartbeat: [3249]: info: glib: ucast: bound send
socket to device: eth2
Oct 30 23:13:10 lvs-test-2 heartbeat: [3249]: info: glib: ucast: bound receive
socket to device: eth2
Oct 30 23:13:10 lvs-test-2 heartbeat: [3249]: info: glib: ucast: started on
port 694 interface eth2 to 192.168.199.11
Oct 30 23:13:10 lvs-test-2 heartbeat: [3249]: debug: write process pid: 3315
Oct 30 23:13:10 lvs-test-2 heartbeat: [3249]: debug: read child process pid:
3316
Oct 30 23:13:10 lvs-test-2 heartbeat: [3249]: info: G_main_add_SignalHandler:
Added signal handler for signal 17
Oct 30 23:13:10 lvs-test-2 heartbeat: [3249]: debug: Limiting CPU: 42 CPU
seconds every 60000 milliseconds
Oct 30 23:13:10 lvs-test-2 heartbeat: [3249]: debug: pid 3249 locked in memory.
Oct 30 23:13:10 lvs-test-2 heartbeat: [3249]: debug: Waiting for child
processes to start
Oct 30 23:13:10 lvs-test-2 heartbeat: [3249]: info: Local status now set to:
'up'
Oct 30 23:13:10 lvs-test-2 heartbeat: [3249]: debug: All your child process are
belong to us
Oct 30 23:13:10 lvs-test-2 heartbeat: [3249]: debug: Starting local status
message @ 2000 ms intervals
Oct 30 23:13:11 lvs-test-2 heartbeat: [3314]: debug: pid 3314 locked in memory.
Oct 30 23:13:11 lvs-test-2 heartbeat: [3314]: debug: Limiting CPU: 6 CPU
seconds every 60000 milliseconds
Oct 30 23:13:11 lvs-test-2 heartbeat: [3315]: debug: pid 3315 locked in memory.
Oct 30 23:13:11 lvs-test-2 heartbeat: [3315]: debug: Limiting CPU: 24 CPU
seconds every 60000 milliseconds
Oct 30 23:13:11 lvs-test-2 heartbeat: [3316]: debug: pid 3316 locked in memory.
Oct 30 23:13:11 lvs-test-2 heartbeat: [3316]: debug: Limiting CPU: 6 CPU
seconds every 60000 milliseconds
Oct 30 23:13:30 lvs-test-2 heartbeat: [3249]: WARN: node lvs-test-1: is dead
Oct 30 23:13:30 lvs-test-2 heartbeat: [3249]: info: Comm_now_up(): updating
status to active
Oct 30 23:13:30 lvs-test-2 heartbeat: [3249]: info: Local status now set to:
'active'
Oct 30 23:13:30 lvs-test-2 heartbeat: [3249]: debug: Sending local starting
msg: resourcestate = 0
Oct 30 23:13:30 lvs-test-2 heartbeat: [3249]: debug: hb_rsc_isstable:
ResourceMgmt_child_count: 0, other_is_stable: 0, takeover_in_progress: 0,
going_standby: 0, standby running(ms): 0, resourcestate: 0
Oct 30 23:13:30 lvs-test-2 heartbeat: [3249]: debug: CreateInitialFilter:
hb_takeover
Oct 30 23:13:30 lvs-test-2 heartbeat: [3249]: debug: CreateInitialFilter:
ip-request-resp
Oct 30 23:13:30 lvs-test-2 heartbeat: [3249]: debug: CreateInitialFilter: status
Oct 30 23:13:30 lvs-test-2 heartbeat: [3249]: debug: CreateInitialFilter:
ip-request
Oct 30 23:13:30 lvs-test-2 heartbeat: [3249]: debug: CreateInitialFilter:
ask_resources
Oct 30 23:13:30 lvs-test-2 heartbeat: [3249]: WARN: No STONITH device
configured.
Oct 30 23:13:30 lvs-test-2 heartbeat: [3249]: WARN: Shared disks are not
protected.
Oct 30 23:13:30 lvs-test-2 heartbeat: [3249]: info: Resources being acquired
from lvs-test-1.
Oct 30 23:13:30 lvs-test-2 heartbeat: [3249]: debug: StartNextRemoteRscReq() -
calling hook
Oct 30 23:13:30 lvs-test-2 heartbeat: [3249]: debug: notify_world: invoking
harc: OLD status: active
Oct 30 23:13:30 lvs-test-2 heartbeat: [3249]: debug: Process [status] started
pid 3318
Oct 30 23:13:30 lvs-test-2 heartbeat: [3249]: debug: Starting notify process
[status]
Oct 30 23:13:30 lvs-test-2 heartbeat: [3249]: debug: takeover_from_node: other
now stable
Oct 30 23:13:30 lvs-test-2 heartbeat: [3249]: debug: Process
[req_our_resources] started pid 3319
Oct 30 23:13:30 lvs-test-2 heartbeat: [3318]: debug: notify_world: setting
SIGCHLD Handler to SIG_DFL
Oct 30 23:13:30 lvs-test-2 heartbeat: [3318]: debug: notify_world: Running harc
status
Oct 30 23:13:30 lvs-test-2 heartbeat: [3319]: debug:
req_our_resources(/usr/lib/heartbeat/ResourceManager listkeys lvs-test-2)
Oct 30 23:13:30 lvs-test-2 harc[3318]: info: Running /etc/ha.d/rc.d/status
status
Oct 30 23:13:30 lvs-test-2 heartbeat: [3319]: info: No local resources
[/usr/lib/heartbeat/ResourceManager listkeys lvs-test-2] to acquire.
Oct 30 23:13:30 lvs-test-2 heartbeat: [3319]: debug: Sending hold resources
msg: all, stable=1 # req_our_resources()
Oct 30 23:13:30 lvs-test-2 heartbeat: [3319]: info: FIFO message [type
resource] written rc=108
Oct 30 23:13:30 lvs-test-2 heartbeat: [3249]: info: AnnounceTakeover(local 0,
foreign 1, reason 'T_RESOURCES' (0))
Oct 30 23:13:30 lvs-test-2 heartbeat: [3249]: info: AnnounceTakeover(local 1,
foreign 1, reason 'T_RESOURCES(us)' (0))
Oct 30 23:13:30 lvs-test-2 heartbeat: [3249]: info: Initial resource
acquisition complete (T_RESOURCES(us))
Oct 30 23:13:30 lvs-test-2 heartbeat: [3249]: info: STATE 1 => 3
Oct 30 23:13:30 lvs-test-2 heartbeat: [3249]: debug: hb_rsc_isstable:
ResourceMgmt_child_count: 2, other_is_stable: 1, takeover_in_progress: 1,
going_standby: 0, standby running(ms): 0, resourcestate: 3
Oct 30 23:13:30 lvs-test-2 heartbeat: [3249]: info: Exiting req_our_resources
process 3319 returned rc 0.
Oct 30 23:13:30 lvs-test-2 heartbeat: [3249]: debug: RscMgmtProc
'req_our_resources' exited code 0
Oct 30 23:13:30 lvs-test-2 heartbeat: [3249]: info: AnnounceTakeover(local 1,
foreign 1, reason 'req_our_resources' (1))
Oct 30 23:13:30 lvs-test-2 heartbeat: [3249]: debug: StartNextRemoteRscReq():
child count 1
Oct 30 23:13:31 lvs-test-2 mach_down[3338]: info: Taking over resource group
IPaddr2::10.204.6.25/24/eth0
Oct 30 23:13:31 lvs-test-2 ResourceManager[3358]: info: Acquiring resource
group: lvs-test-1 IPaddr2::10.204.6.25/24/eth0
Oct 30 23:13:31 lvs-test-2 IPaddr2[3382]: INFO: IPaddr2 Resource is stopped
Oct 30 23:13:31 lvs-test-2 ResourceManager[3358]: info: Running
/etc/ha.d/resource.d/IPaddr2 10.204.6.25/24/eth0 start
Oct 30 23:13:31 lvs-test-2 ResourceManager[3358]: debug: Starting
/etc/ha.d/resource.d/IPaddr2 10.204.6.25/24/eth0 start
Oct 30 23:13:31 lvs-test-2 IPaddr2[3596]: INFO: /sbin/ip -f inet addr add
10.204.6.25/24 brd 10.204.6.255 dev eth0
Oct 30 23:13:31 lvs-test-2 IPaddr2[3596]: INFO: /sbin/ip link set eth0 up
Oct 30 23:13:31 lvs-test-2 IPaddr2[3596]: INFO: /usr/lib/heartbeat/send_arp -i
200 -r 5 -p /var/run/heartbeat/rsctmp/send_arp/send_arp-10.204.6.25 eth0
10.204.6.25 auto 10.204.6.25 ffffffffffff
Oct 30 23:13:31 lvs-test-2 IPaddr2[3514]: INFO: IPaddr2 Success
Oct 30 23:13:31 lvs-test-2 ResourceManager[3358]: debug:
/etc/ha.d/resource.d/IPaddr2 10.204.6.25/24/eth0 start done. RC=0
Oct 30 23:13:31 lvs-test-2 mach_down[3338]: info: /usr/lib/heartbeat/mach_down:
nice_failback: foreign resources acquired
Oct 30 23:13:31 lvs-test-2 mach_down[3338]: info: mach_down takeover complete
for node lvs-test-1.
Oct 30 23:13:31 lvs-test-2 heartbeat: [3249]: info: Exiting status process 3318
returned rc 0.
Oct 30 23:13:31 lvs-test-2 heartbeat: [3249]: debug: RscMgmtProc 'status'
exited code 0
Oct 30 23:13:41 lvs-test-2 heartbeat: [3249]: info: Local Resource acquisition
completed. (none)
Oct 30 23:13:41 lvs-test-2 heartbeat: [3249]: info: local resource transition
completed.
Oct 30 23:13:41 lvs-test-2 heartbeat: [3249]: debug: Sending hold resources
msg: all, stable=1 # <none>
Oct 30 23:13:41 lvs-test-2 heartbeat: [3249]: info: AnnounceTakeover(local 1,
foreign 1, reason 'T_RESOURCES(us)' (1))
Oct 30 23:13:41 lvs-test-2 heartbeat: [3249]: debug: hb_rsc_isstable:
ResourceMgmt_child_count: 0, other_is_stable: 1, takeover_in_progress: 1,
going_standby: 0, standby running(ms): 0, resourcestate: 4
Oct 30 23:13:41 lvs-test-2 heartbeat: [3249]: debug: hb_rsc_isstable:
ResourceMgmt_child_count: 0, other_is_stable: 1, takeover_in_progress: 1,
going_standby: 0, standby running(ms): 0, resourcestate: 4
Oct 30 23:16:22 lvs-test-2 heartbeat: [3249]: info: Link lvs-test-1:eth2 up.
Oct 30 23:16:22 lvs-test-2 heartbeat: [3249]: info: Status update for node
lvs-test-1: status active
Oct 30 23:16:22 lvs-test-2 heartbeat: [3249]: debug: Status seqno: 113 msgtime:
1225408588
Oct 30 23:16:22 lvs-test-2 heartbeat: [3249]: debug: StartNextRemoteRscReq() -
calling hook
Oct 30 23:16:22 lvs-test-2 heartbeat: [3249]: debug: notify_world: invoking
harc: OLD status: active
Oct 30 23:16:22 lvs-test-2 heartbeat: [3249]: debug: Process [status] started
pid 3674
Oct 30 23:16:22 lvs-test-2 heartbeat: [3249]: debug: Starting notify process
[status]
Oct 30 23:16:22 lvs-test-2 heartbeat: [3674]: debug: notify_world: setting
SIGCHLD Handler to SIG_DFL
Oct 30 23:16:22 lvs-test-2 heartbeat: [3674]: debug: notify_world: Running harc
status
Oct 30 23:16:22 lvs-test-2 harc[3674]: info: Running /etc/ha.d/rc.d/status
status
Oct 30 23:16:22 lvs-test-2 heartbeat: [3249]: info: Exiting status process 3674
returned rc 0.
Oct 30 23:16:22 lvs-test-2 heartbeat: [3249]: debug: RscMgmtProc 'status'
exited code 0
Oct 30 23:16:26 lvs-test-2 heartbeat: [3249]: info: all clients are now paused
Oct 30 23:16:26 lvs-test-2 heartbeat: [3249]: debug: hist->ackseq =6
Oct 30 23:16:26 lvs-test-2 heartbeat: [3249]: debug: hist->lowseq =5,
hist->hiseq=107
Oct 30 23:16:26 lvs-test-2 heartbeat: [3249]: debug: expecting from lvs-test-1
Oct 30 23:16:26 lvs-test-2 heartbeat: [3249]: debug: it's ackseq=0
Oct 30 23:16:26 lvs-test-2 heartbeat: [3249]: debug:
Oct 30 23:16:28 lvs-test-2 heartbeat: [3249]: debug: hist->ackseq =6
Oct 30 23:16:28 lvs-test-2 heartbeat: [3249]: debug: hist->lowseq =5,
hist->hiseq=108
Oct 30 23:16:28 lvs-test-2 heartbeat: [3249]: debug: expecting from lvs-test-1
Oct 30 23:16:28 lvs-test-2 heartbeat: [3249]: debug: it's ackseq=0
Oct 30 23:16:28 lvs-test-2 heartbeat: [3249]: debug:
Oct 30 23:16:30 lvs-test-2 heartbeat: [3249]: debug: hist->ackseq =6
Oct 30 23:16:30 lvs-test-2 heartbeat: [3249]: debug: hist->lowseq =5,
hist->hiseq=109
Oct 30 23:16:30 lvs-test-2 heartbeat: [3249]: debug: expecting from lvs-test-1
Oct 30 23:16:30 lvs-test-2 heartbeat: [3249]: debug: it's ackseq=0
Oct 30 23:16:30 lvs-test-2 heartbeat: [3249]: debug:
Oct 30 23:16:32 lvs-test-2 heartbeat: [3249]: debug: hist->ackseq =6
Oct 30 23:16:32 lvs-test-2 heartbeat: [3249]: debug: hist->lowseq =5,
hist->hiseq=110
Oct 30 23:16:32 lvs-test-2 heartbeat: [3249]: debug: expecting from lvs-test-1
Oct 30 23:16:32 lvs-test-2 heartbeat: [3249]: debug: it's ackseq=0
Oct 30 23:16:32 lvs-test-2 heartbeat: [3249]: debug:
Oct 30 23:16:34 lvs-test-2 heartbeat: [3249]: debug: hist->ackseq =6
Oct 30 23:16:34 lvs-test-2 heartbeat: [3249]: debug: hist->lowseq =5,
hist->hiseq=111
Oct 30 23:16:34 lvs-test-2 heartbeat: [3249]: debug: expecting from lvs-test-1
Oct 30 23:16:34 lvs-test-2 heartbeat: [3249]: debug: it's ackseq=0
Oct 30 23:16:34 lvs-test-2 heartbeat: [3249]: debug:
Oct 30 23:16:36 lvs-test-2 heartbeat: [3249]: debug: hist->ackseq =6
Oct 30 23:16:36 lvs-test-2 heartbeat: [3249]: debug: hist->lowseq =5,
hist->hiseq=112
Oct 30 23:16:36 lvs-test-2 heartbeat: [3249]: debug: expecting from lvs-test-1
Oct 30 23:16:36 lvs-test-2 heartbeat: [3249]: debug: it's ackseq=0
Oct 30 23:16:36 lvs-test-2 heartbeat: [3249]: debug:
Oct 30 23:16:36 lvs-test-2 heartbeat: [3249]: info: all clients are now resumed
Oct 30 23:17:16 lvs-test-2 heartbeat: [3249]: debug: APIregistration_dispatch()
{
Oct 30 23:17:16 lvs-test-2 heartbeat: [3249]: debug: process_registerevent() {
Oct 30 23:17:16 lvs-test-2 heartbeat: [3249]: debug: client->gsource = 0x637408
Oct 30 23:17:16 lvs-test-2 heartbeat: [3249]: debug: }/*process_registerevent*/;
Oct 30 23:17:16 lvs-test-2 heartbeat: [3249]: debug:
}/*APIregistration_dispatch*/;
Oct 30 23:17:16 lvs-test-2 heartbeat: [3249]: debug: Checking client
authorization for client 3689 (0:105)
Oct 30 23:17:16 lvs-test-2 heartbeat: [3249]: debug:
create_seq_snapshot_table:no missing packets found for node lvs-test-1
Oct 30 23:17:16 lvs-test-2 heartbeat: [3249]: debug:
create_seq_snapshot_table:no missing packets found for node lvs-test-2
Oct 30 23:17:16 lvs-test-2 heartbeat: [3249]: debug: Signing on API client 3689
('casual')
Oct 30 23:17:16 lvs-test-2 heartbeat: [3249]: debug: hb_rsc_isstable:
ResourceMgmt_child_count: 0, other_is_stable: 1, takeover_in_progress: 1,
going_standby: 0, standby running(ms): 0, resourcestate: 4
Oct 30 23:17:16 lvs-test-2 heartbeat: [3249]: debug: Signing client 3689 off
Oct 30 23:17:16 lvs-test-2 heartbeat: [3249]: debug: G_remove_client(pid=3689,
reason='signoff' gsource=0x637408) {
Oct 30 23:17:16 lvs-test-2 heartbeat: [3249]: debug: api_remove_client_int:
removing pid [3689] reason: signoff
Oct 30 23:17:16 lvs-test-2 heartbeat: [3249]: debug: }/*G_remove_client;*/
Oct 30 23:23:10 lvs-test-2 heartbeat: [3249]: info: Daily informational memory
statistics
Oct 30 23:23:10 lvs-test-2 heartbeat: [3249]: info: MSG stats: 9/619 ms age 0
[pid3249/MST_CONTROL]
Oct 30 23:23:10 lvs-test-2 heartbeat: [3249]: info: ha_malloc stats: 634/21506
87224/41510 [pid3249/MST_CONTROL]
Oct 30 23:23:10 lvs-test-2 heartbeat: [3249]: info: RealMalloc stats: 364356
total malloc bytes. pid [3249/MST_CONTROL]
Oct 30 23:23:10 lvs-test-2 heartbeat: [3249]: info: Current arena value: 0
Oct 30 23:23:10 lvs-test-2 heartbeat: [3249]: info: MSG stats: 0/1 ms age
579430 [pid3314/HBFIFO]
Oct 30 23:23:10 lvs-test-2 heartbeat: [3249]: info: ha_malloc stats: 341/404
40556/18454 [pid3314/HBFIFO]
Oct 30 23:23:10 lvs-test-2 heartbeat: [3249]: info: RealMalloc stats: 43132
total malloc bytes. pid [3314/HBFIFO]
Oct 30 23:23:10 lvs-test-2 heartbeat: [3249]: info: Current arena value: 0
Oct 30 23:23:10 lvs-test-2 heartbeat: [3249]: info: MSG stats: 0/0 ms age
17181271090 [pid3315/HBWRITE]
Oct 30 23:23:10 lvs-test-2 heartbeat: [3249]: info: ha_malloc stats: 338/766
40632/18627 [pid3315/HBWRITE]
Oct 30 23:23:10 lvs-test-2 heartbeat: [3249]: info: RealMalloc stats: 49128
total malloc bytes. pid [3315/HBWRITE]
Oct 30 23:23:10 lvs-test-2 heartbeat: [3249]: info: Current arena value: 0
Oct 30 23:23:10 lvs-test-2 heartbeat: [3249]: info: MSG stats: 0/0 ms age
17181271090 [pid3316/HBREAD]
Oct 30 23:23:10 lvs-test-2 heartbeat: [3249]: info: ha_malloc stats: 339/878
40724/18691 [pid3316/HBREAD]
Oct 30 23:23:10 lvs-test-2 heartbeat: [3249]: info: RealMalloc stats: 41100
total malloc bytes. pid [3316/HBREAD]
Oct 30 23:23:10 lvs-test-2 heartbeat: [3249]: info: Current arena value: 0
Oct 30 23:23:10 lvs-test-2 heartbeat: [3249]: info: These are nothing to worry
about.
_______________________________________________
Linux-HA mailing list
[email protected]
http://lists.linux-ha.org/mailman/listinfo/linux-ha
See also: http://linux-ha.org/ReportingProblems
