No takers? I must be doing something wrong in the below configuration as
I would assume that someone else would have tested and experienced the
following problem as this seems like one of the scenarios HB was
designed to handle.
Perhaps there are know problems with 2.0.7? I've located newer Debian
packages but it seems that I would need to update to testing in order to
use them.
Perhaps using quorumd on a 3rd server would prevent this split-brain
condition? Is it possible for me to use quorumd with the following 1.x
style resource setup?
On Fri, Oct 31, 2008 at 11:49:22AM +0000, [EMAIL PROTECTED] wrote:
> 2 Node HB Cluster
> Heartbeat 2.0.7
> Debian Linux 4.0 (Etch).
> Linux Kernel 2.6.18
> Node Names: lvs-test-1, lvs-test-2
>
> I'm trying to test a very basic HA config in which an aliased IP address
> is migrated between 2 boxes. Since I'm just testing HA there is no load
> on the boxes, and I know the network is reliable as this is simply an
> isolated test network.
>
> Everything appears to work correctly when the heartbeat processes can
> reliably communicate from the get-go. To be more precise if both systems
> are booted at the same time only one get assigned the aliases IP and the
> other operates in a stand-by state. If I bring down the active server or
> use hb_takeover to swing the resource all works well.
>
> The problem arises after severing the network connection (booting both
> systems when the ethernet switch used for the heartbeat network is
> powered off) Both systems activate the same IP address as expected
> (split brain condition). Once the heartbeat is re-established I would
> expect one system, to relinquish the aliased IP and go to a stand-by
> state. This does not occur. The logfiles don't seem to exhibit any
> errors. If I do a 'cl_status rscstatus' on each system one outputs
> 'all', the other outputs 'transition'. The system in 'transition' never
> relinquishes the aliased IP and the heartbeat process appears to be hung
> waiting for some event as it will not repond to a 'heartbeat -k'. This
> also prevents the system from shutting down since Debian's shutdown
> script ends up running 'heartbeat -k'.
>
> You'll note that in the logfiles below that one system always outputs
> more "debug: hist..." entries. This happens the same way every time.
> After the "all clients are now resumed" line.. that's it, one system is
> in transition forever:(
>
> Any and all help is greatly appreciated,
> Wayne
>
> ------------------------------------------------------------------------
>
> I have included the ha.cf, haresource and the logfiles from both
> systems.
>
> ha.cf files
> ---
> # lvs-test-1
> use_logd off
> logfacility daemon
>
> keepalive 2
> initdead 20
> deadtime 10
> warntime 5
> auto_failback off
>
> msgfmt netstring
>
> ucast eth2 192.168.199.10
>
> node lvs-test-1
> node lvs-test-2
> # EOF
>
> # lvs-test-2
> use_logd off
> logfacility daemon
>
> keepalive 2
> initdead 20
> deadtime 10
> warntime 5
> auto_failback off
>
> msgfmt netstring
>
> ucast eth2 192.168.199.11
>
> node lvs-test-1
> node lvs-test-2
> # EOF
>
> haresource file, identical on both systems
> ---
> lvs-test-1 IPaddr2::10.204.6.25/24/eth0
> # EOF
>
> lvs-test-1 logfile
> ---
> Oct 30 19:13:01 lvs-test-1 heartbeat: [3169]: debug: add_option(keepalive,2)
> Oct 30 19:13:01 lvs-test-1 heartbeat: [3169]: debug: add_option(initdead,20)
> Oct 30 19:13:01 lvs-test-1 heartbeat: [3169]: debug: add_option(deadtime,10)
> Oct 30 19:13:01 lvs-test-1 heartbeat: [3169]: debug: add_option(warntime,5)
> Oct 30 19:13:01 lvs-test-1 heartbeat: [3169]: debug:
> add_option(auto_failback,off)
> Oct 30 19:13:01 lvs-test-1 heartbeat: [3169]: debug:
> add_option(msgfmt,netstring)
> Oct 30 19:13:01 lvs-test-1 heartbeat: [3169]: debug:
> add_option(node,lvs-test-1)
> Oct 30 19:13:01 lvs-test-1 heartbeat: [3169]: debug:
> add_option(node,lvs-test-2)
> Oct 30 19:13:01 lvs-test-1 heartbeat: [3169]: debug: uid=hacluster, gid=<null>
> Oct 30 19:13:01 lvs-test-1 heartbeat: [3169]: debug: uid=hacluster, gid=<null>
> Oct 30 19:13:01 lvs-test-1 heartbeat: [3169]: debug: uid=<null>, gid=haclient
> Oct 30 19:13:01 lvs-test-1 heartbeat: [3169]: debug: uid=root, gid=<null>
> Oct 30 19:13:01 lvs-test-1 heartbeat: [3169]: debug: uid=<null>, gid=haclient
> Oct 30 19:13:01 lvs-test-1 heartbeat: [3169]: debug: Beginning authentication
> parsing
> Oct 30 19:13:01 lvs-test-1 heartbeat: [3169]: debug: 16 max authentication
> methods
> Oct 30 19:13:01 lvs-test-1 heartbeat: [3169]: debug: Keyfile opened
> Oct 30 19:13:01 lvs-test-1 heartbeat: [3169]: debug: Keyfile perms OK
> Oct 30 19:13:01 lvs-test-1 heartbeat: [3169]: debug: 16 max authentication
> methods
> Oct 30 19:13:01 lvs-test-1 heartbeat: [3169]: debug: Found authentication
> method [crc]
> Oct 30 19:13:01 lvs-test-1 heartbeat: [3169]: info: AUTH: i=1: key =
> 0x5d5b18, auth=0x2aae5b426550, authname=crc
> Oct 30 19:13:01 lvs-test-1 heartbeat: [3169]: debug: Outbound signing method
> is 1
> Oct 30 19:13:01 lvs-test-1 heartbeat: [3169]: debug: Authentication parsing
> complete [1]
> Oct 30 19:13:01 lvs-test-1 heartbeat: [3169]: debug:
> add_option(cluster,linux-ha)
> Oct 30 19:13:01 lvs-test-1 heartbeat: [3169]: debug: add_option(hopfudge,1)
> Oct 30 19:13:01 lvs-test-1 heartbeat: [3169]: debug: add_option(baud,19200)
> Oct 30 19:13:01 lvs-test-1 heartbeat: [3169]: debug:
> add_option(hbgenmethod,file)
> Oct 30 19:13:01 lvs-test-1 heartbeat: [3169]: debug: add_option(realtime,true)
> Oct 30 19:13:01 lvs-test-1 heartbeat: [3169]: debug:
> add_option(conn_logd_time,60)
> Oct 30 19:13:01 lvs-test-1 heartbeat: [3169]: debug:
> add_option(log_badpack,true)
> Oct 30 19:13:01 lvs-test-1 heartbeat: [3169]: debug:
> add_option(coredumps,true)
> Oct 30 19:13:01 lvs-test-1 heartbeat: [3169]: WARN: Core dumps could be lost
> if multiple dumps occur
> Oct 30 19:13:01 lvs-test-1 heartbeat: [3169]: WARN: Consider setting
> /proc/sys/kernel/core_uses_pid (or equivalent) to 1 for maximum supportability
> Oct 30 19:13:01 lvs-test-1 heartbeat: [3169]: debug: add_option(crm,false)
> Oct 30 19:13:01 lvs-test-1 heartbeat: [3169]: debug: add_option(autojoin,none)
> Oct 30 19:13:01 lvs-test-1 heartbeat: [3169]: debug: add_option(uuidfrom,file)
> Oct 30 19:13:01 lvs-test-1 heartbeat: [3169]: debug:
> add_option(compression,zlib)
> Oct 30 19:13:01 lvs-test-1 heartbeat: [3169]: debug:
> add_option(compression_threshold,2)
> Oct 30 19:13:01 lvs-test-1 heartbeat: [3169]: debug:
> add_option(traditional_compression,no)
> Oct 30 19:13:01 lvs-test-1 heartbeat: [3169]: debug:
> add_option(max_rexmit_delay,250)
> Oct 30 19:13:01 lvs-test-1 heartbeat: [3169]: debug: Setting max_rexmit_delay
> to 250 ms
> Oct 30 19:13:01 lvs-test-1 heartbeat: [3169]: debug:
> add_option(record_config_changes,on)
> Oct 30 19:13:01 lvs-test-1 heartbeat: [3169]: debug:
> add_option(record_pengine_inputs,on)
> Oct 30 19:13:01 lvs-test-1 heartbeat: [3169]: debug:
> add_option(enable_config_writes,on)
> Oct 30 19:13:01 lvs-test-1 heartbeat: [3169]: debug:
> add_option(memreserve,6500)
> Oct 30 19:13:01 lvs-test-1 heartbeat: [3169]: debug: add_option(debug,1)
> Oct 30 19:13:01 lvs-test-1 heartbeat: [3169]: WARN: Logging daemon is
> disabled --enabling logging daemon is recommended
> Oct 30 19:13:01 lvs-test-1 heartbeat: [3169]: info: **************************
> Oct 30 19:13:01 lvs-test-1 heartbeat: [3169]: info: Configuration validated.
> Starting heartbeat 2.0.7
> Oct 30 19:13:01 lvs-test-1 heartbeat: [3169]: debug: HA configuration OK.
> Heartbeat starting.
> Oct 30 19:13:01 lvs-test-1 heartbeat: [3170]: info: heartbeat: version 2.0.7
> Oct 30 19:13:03 lvs-test-1 heartbeat: [3170]: info: Heartbeat generation: 13
> Oct 30 19:13:04 lvs-test-1 heartbeat: [3170]: debug: uuid
> is:31c440a3-cc9c-487b-b07e-53ad7038d550
> Oct 30 19:13:04 lvs-test-1 heartbeat: [3170]: info:
> G_main_add_TriggerHandler: Added signal manual handler
> Oct 30 19:13:04 lvs-test-1 heartbeat: [3170]: info:
> G_main_add_TriggerHandler: Added signal manual handler
> Oct 30 19:13:04 lvs-test-1 heartbeat: [3170]: info: Removing
> /var/run/heartbeat/rsctmp failed, recreating.
> Oct 30 19:13:04 lvs-test-1 heartbeat: [3170]: debug: opening ucast eth2
> (UDP/IP unicast)
> Oct 30 19:13:04 lvs-test-1 heartbeat: [3170]: debug: FIFO process pid: 3240
> Oct 30 19:13:04 lvs-test-1 heartbeat: [3170]: info: glib: ucast: write socket
> priority set to IPTOS_LOWDELAY on eth2
> Oct 30 19:13:04 lvs-test-1 heartbeat: [3170]: info: glib: ucast: bound send
> socket to device: eth2
> Oct 30 19:13:04 lvs-test-1 heartbeat: [3170]: info: glib: ucast: bound
> receive socket to device: eth2
> Oct 30 19:13:04 lvs-test-1 heartbeat: [3170]: info: glib: ucast: started on
> port 694 interface eth2 to 192.168.199.10
> Oct 30 19:13:04 lvs-test-1 heartbeat: [3170]: debug: write process pid: 3241
> Oct 30 19:13:04 lvs-test-1 heartbeat: [3170]: debug: read child process pid:
> 3242
> Oct 30 19:13:04 lvs-test-1 heartbeat: [3170]: info: G_main_add_SignalHandler:
> Added signal handler for signal 17
> Oct 30 19:13:04 lvs-test-1 heartbeat: [3170]: debug: Limiting CPU: 42 CPU
> seconds every 60000 milliseconds
> Oct 30 19:13:04 lvs-test-1 heartbeat: [3170]: debug: pid 3170 locked in
> memory.
> Oct 30 19:13:04 lvs-test-1 heartbeat: [3170]: debug: Waiting for child
> processes to start
> Oct 30 19:13:04 lvs-test-1 heartbeat: [3170]: info: Local status now set to:
> 'up'
> Oct 30 19:13:04 lvs-test-1 heartbeat: [3170]: debug: All your child process
> are belong to us
> Oct 30 19:13:04 lvs-test-1 heartbeat: [3170]: debug: Starting local status
> message @ 2000 ms intervals
> Oct 30 19:13:05 lvs-test-1 heartbeat: [3240]: debug: pid 3240 locked in
> memory.
> Oct 30 19:13:05 lvs-test-1 heartbeat: [3240]: debug: Limiting CPU: 6 CPU
> seconds every 60000 milliseconds
> Oct 30 19:13:05 lvs-test-1 heartbeat: [3242]: debug: pid 3242 locked in
> memory.
> Oct 30 19:13:05 lvs-test-1 heartbeat: [3241]: debug: pid 3241 locked in
> memory.
> Oct 30 19:13:05 lvs-test-1 heartbeat: [3242]: debug: Limiting CPU: 6 CPU
> seconds every 60000 milliseconds
> Oct 30 19:13:05 lvs-test-1 heartbeat: [3241]: debug: Limiting CPU: 24 CPU
> seconds every 60000 milliseconds
> Oct 30 19:13:24 lvs-test-1 heartbeat: [3170]: WARN: node lvs-test-2: is dead
> Oct 30 19:13:24 lvs-test-1 heartbeat: [3170]: info: Comm_now_up(): updating
> status to active
> Oct 30 19:13:24 lvs-test-1 heartbeat: [3170]: info: Local status now set to:
> 'active'
> Oct 30 19:13:24 lvs-test-1 heartbeat: [3170]: debug: Sending local starting
> msg: resourcestate = 0
> Oct 30 19:13:24 lvs-test-1 heartbeat: [3170]: debug: hb_rsc_isstable:
> ResourceMgmt_child_count: 0, other_is_stable: 0, takeover_in_progress: 0,
> going_standby: 0, standby running(ms): 0, resourcestate: 0
> Oct 30 19:13:25 lvs-test-1 heartbeat: [3170]: debug: CreateInitialFilter:
> hb_takeover
> Oct 30 19:13:25 lvs-test-1 heartbeat: [3170]: debug: CreateInitialFilter:
> status
> Oct 30 19:13:25 lvs-test-1 heartbeat: [3170]: debug: CreateInitialFilter:
> ask_resources
> Oct 30 19:13:25 lvs-test-1 heartbeat: [3170]: debug: CreateInitialFilter:
> ip-request-resp
> Oct 30 19:13:25 lvs-test-1 heartbeat: [3170]: debug: CreateInitialFilter:
> ip-request
> Oct 30 19:13:25 lvs-test-1 heartbeat: [3170]: WARN: No STONITH device
> configured.
> Oct 30 19:13:25 lvs-test-1 heartbeat: [3170]: WARN: Shared disks are not
> protected.
> Oct 30 19:13:25 lvs-test-1 heartbeat: [3170]: info: Resources being acquired
> from lvs-test-2.
> Oct 30 19:13:25 lvs-test-1 heartbeat: [3170]: debug: StartNextRemoteRscReq()
> - calling hook
> Oct 30 19:13:25 lvs-test-1 heartbeat: [3170]: debug: notify_world: invoking
> harc: OLD status: active
> Oct 30 19:13:25 lvs-test-1 heartbeat: [3170]: debug: Process [status] started
> pid 3244
> Oct 30 19:13:25 lvs-test-1 heartbeat: [3170]: debug: Starting notify process
> [status]
> Oct 30 19:13:25 lvs-test-1 heartbeat: [3170]: debug: takeover_from_node:
> other now stable
> Oct 30 19:13:25 lvs-test-1 heartbeat: [3170]: debug: Process
> [req_our_resources] started pid 3245
> Oct 30 19:13:25 lvs-test-1 heartbeat: [3244]: debug: notify_world: setting
> SIGCHLD Handler to SIG_DFL
> Oct 30 19:13:25 lvs-test-1 heartbeat: [3244]: debug: notify_world: Running
> harc status
> Oct 30 19:13:25 lvs-test-1 heartbeat: [3245]: debug:
> req_our_resources(/usr/lib/heartbeat/ResourceManager listkeys lvs-test-1)
> Oct 30 19:13:25 lvs-test-1 heartbeat: [3245]: debug: req_our_resources():
> running [/usr/lib/heartbeat/req_resource IPaddr2::10.204.6.25/24/eth0]
> Oct 30 19:13:25 lvs-test-1 harc[3244]: info: Running /etc/ha.d/rc.d/status
> status
> Oct 30 19:13:25 lvs-test-1 mach_down[3275]: info:
> /usr/lib/heartbeat/mach_down: nice_failback: foreign resources acquired
> Oct 30 19:13:25 lvs-test-1 mach_down[3275]: info: mach_down takeover complete
> for node lvs-test-2.
> Oct 30 19:13:25 lvs-test-1 heartbeat: [3170]: info: AnnounceTakeover(local 0,
> foreign 1, reason 'T_RESOURCES' (0))
> Oct 30 19:13:25 lvs-test-1 heartbeat: [3170]: info: AnnounceTakeover(local 1,
> foreign 1, reason 'T_RESOURCES(us)' (0))
> Oct 30 19:13:25 lvs-test-1 heartbeat: [3170]: info: Initial resource
> acquisition complete (T_RESOURCES(us))
> Oct 30 19:13:25 lvs-test-1 heartbeat: [3170]: info: mach_down takeover
> complete.
> Oct 30 19:13:25 lvs-test-1 heartbeat: [3170]: debug: process_resources(3):
> other now stable
> Oct 30 19:13:25 lvs-test-1 heartbeat: [3170]: info: AnnounceTakeover(local 1,
> foreign 1, reason 'mach_down' (1))
> Oct 30 19:13:25 lvs-test-1 heartbeat: [3170]: info: STATE 1 => 3
> Oct 30 19:13:25 lvs-test-1 heartbeat: [3170]: debug: hb_rsc_isstable:
> ResourceMgmt_child_count: 2, other_is_stable: 1, takeover_in_progress: 0,
> going_standby: 0, standby running(ms): 0, resourcestate: 3
> Oct 30 19:13:25 lvs-test-1 heartbeat: [3170]: info: Exiting status process
> 3244 returned rc 0.
> Oct 30 19:13:25 lvs-test-1 heartbeat: [3170]: debug: RscMgmtProc 'status'
> exited code 0
> Oct 30 19:13:25 lvs-test-1 heartbeat: [3170]: debug: StartNextRemoteRscReq():
> child count 1
> Oct 30 19:13:25 lvs-test-1 IPaddr2[3282]: INFO: IPaddr2 Resource is stopped
> Oct 30 19:13:25 lvs-test-1 req_resource[3258]: debug: in
> /usr/lib/heartbeat/req_resource IPaddr2::10.204.6.25/24/eth0
> Oct 30 19:13:25 lvs-test-1 req_resource[3258]: debug: dont_ask: yes
> nice_failback: yes
> Oct 30 19:13:25 lvs-test-1 heartbeat: [3245]: info: 1 local resources from
> [/usr/lib/heartbeat/ResourceManager listkeys lvs-test-1]
> Oct 30 19:13:25 lvs-test-1 heartbeat: [3245]: info: Local Resource
> acquisition completed.
> Oct 30 19:13:25 lvs-test-1 heartbeat: [3245]: debug: Sending hold resources
> msg: all, stable=1 # req_our_resources()
> Oct 30 19:13:25 lvs-test-1 heartbeat: [3245]: info: FIFO message [type
> resource] written rc=108
> Oct 30 19:13:25 lvs-test-1 heartbeat: [3170]: debug: StartNextRemoteRscReq():
> child count 1
> Oct 30 19:13:25 lvs-test-1 heartbeat: [3170]: info: AnnounceTakeover(local 1,
> foreign 1, reason 'T_RESOURCES(us)' (1))
> Oct 30 19:13:25 lvs-test-1 heartbeat: [3170]: debug: hb_rsc_isstable:
> ResourceMgmt_child_count: 1, other_is_stable: 1, takeover_in_progress: 0,
> going_standby: 0, standby running(ms): 0, resourcestate: 3
> Oct 30 19:13:25 lvs-test-1 heartbeat: [3170]: info: Exiting req_our_resources
> process 3245 returned rc 0.
> Oct 30 19:13:25 lvs-test-1 heartbeat: [3170]: debug: RscMgmtProc
> 'req_our_resources' exited code 0
> Oct 30 19:13:25 lvs-test-1 heartbeat: [3170]: info: AnnounceTakeover(local 1,
> foreign 1, reason 'req_our_resources' (1))
> Oct 30 19:13:25 lvs-test-1 heartbeat: [3170]: debug: StartNextRemoteRscReq()
> - calling hook
> Oct 30 19:13:25 lvs-test-1 heartbeat: [3170]: debug: notify_world: invoking
> harc: OLD status: active
> Oct 30 19:13:25 lvs-test-1 heartbeat: [3170]: debug: Process
> [ip-request-resp] started pid 3431
> Oct 30 19:13:25 lvs-test-1 heartbeat: [3170]: debug: Starting notify process
> [ip-request-resp]
> Oct 30 19:13:25 lvs-test-1 heartbeat: [3431]: debug: notify_world: setting
> SIGCHLD Handler to SIG_DFL
> Oct 30 19:13:25 lvs-test-1 heartbeat: [3431]: debug: notify_world: Running
> harc ip-request-resp
> Oct 30 19:13:25 lvs-test-1 harc[3431]: info: Running
> /etc/ha.d/rc.d/ip-request-resp ip-request-resp
> Oct 30 19:13:25 lvs-test-1 ip-request-resp[3431]: received ip-request-resp
> IPaddr2::10.204.6.25/24/eth0 OK yes
> Oct 30 19:13:25 lvs-test-1 ResourceManager[3446]: info: Acquiring resource
> group: lvs-test-1 IPaddr2::10.204.6.25/24/eth0
> Oct 30 19:13:25 lvs-test-1 IPaddr2[3470]: INFO: IPaddr2 Resource is stopped
> Oct 30 19:13:25 lvs-test-1 ResourceManager[3446]: info: Running
> /etc/ha.d/resource.d/IPaddr2 10.204.6.25/24/eth0 start
> Oct 30 19:13:25 lvs-test-1 ResourceManager[3446]: debug: Starting
> /etc/ha.d/resource.d/IPaddr2 10.204.6.25/24/eth0 start
> Oct 30 19:13:25 lvs-test-1 IPaddr2[3684]: INFO: /sbin/ip -f inet addr add
> 10.204.6.25/24 brd 10.204.6.255 dev eth0
> Oct 30 19:13:25 lvs-test-1 IPaddr2[3684]: INFO: /sbin/ip link set eth0 up
> Oct 30 19:13:25 lvs-test-1 IPaddr2[3684]: INFO: /usr/lib/heartbeat/send_arp
> -i 200 -r 5 -p /var/run/heartbeat/rsctmp/send_arp/send_arp-10.204.6.25 eth0
> 10.204.6.25 auto 10.204.6.25 ffffffffffff
> Oct 30 19:13:25 lvs-test-1 IPaddr2[3602]: INFO: IPaddr2 Success
> Oct 30 19:13:25 lvs-test-1 ResourceManager[3446]: debug:
> /etc/ha.d/resource.d/IPaddr2 10.204.6.25/24/eth0 start done. RC=0
> Oct 30 19:13:25 lvs-test-1 heartbeat: [3170]: info: Exiting ip-request-resp
> process 3431 returned rc 0.
> Oct 30 19:13:25 lvs-test-1 heartbeat: [3170]: debug: RscMgmtProc
> 'ip-request-resp' exited code 0
> Oct 30 19:13:25 lvs-test-1 heartbeat: [3170]: info: AnnounceTakeover(local 1,
> foreign 1, reason 'ip-request-resp' (1))
> Oct 30 19:13:35 lvs-test-1 heartbeat: [3170]: info: Local Resource
> acquisition completed. (none)
> Oct 30 19:13:35 lvs-test-1 heartbeat: [3170]: info: local resource transition
> completed.
> Oct 30 19:13:35 lvs-test-1 heartbeat: [3170]: debug: Sending hold resources
> msg: all, stable=1 # <none>
> Oct 30 19:13:35 lvs-test-1 heartbeat: [3170]: info: AnnounceTakeover(local 1,
> foreign 1, reason 'T_RESOURCES(us)' (1))
> Oct 30 19:13:35 lvs-test-1 heartbeat: [3170]: debug: hb_rsc_isstable:
> ResourceMgmt_child_count: 0, other_is_stable: 1, takeover_in_progress: 0,
> going_standby: 0, standby running(ms): 0, resourcestate: 4
> Oct 30 19:13:35 lvs-test-1 heartbeat: [3170]: debug: hb_rsc_isstable:
> ResourceMgmt_child_count: 0, other_is_stable: 1, takeover_in_progress: 0,
> going_standby: 0, standby running(ms): 0, resourcestate: 4
> Oct 30 19:16:27 lvs-test-1 heartbeat: [3170]: info: Link lvs-test-2:eth2 up.
> Oct 30 19:16:27 lvs-test-1 heartbeat: [3170]: info: Status update for node
> lvs-test-2: status active
> Oct 30 19:16:27 lvs-test-1 heartbeat: [3170]: debug: Status seqno: 104
> msgtime: 1225408580
> Oct 30 19:16:27 lvs-test-1 heartbeat: [3170]: debug: StartNextRemoteRscReq()
> - calling hook
> Oct 30 19:16:27 lvs-test-1 heartbeat: [3170]: debug: notify_world: invoking
> harc: OLD status: active
> Oct 30 19:16:27 lvs-test-1 heartbeat: [3170]: debug: Process [status] started
> pid 3746
> Oct 30 19:16:27 lvs-test-1 heartbeat: [3170]: debug: Starting notify process
> [status]
> Oct 30 19:16:27 lvs-test-1 heartbeat: [3746]: debug: notify_world: setting
> SIGCHLD Handler to SIG_DFL
> Oct 30 19:16:27 lvs-test-1 heartbeat: [3746]: debug: notify_world: Running
> harc status
> Oct 30 19:16:27 lvs-test-1 harc[3746]: info: Running /etc/ha.d/rc.d/status
> status
> Oct 30 19:16:27 lvs-test-1 heartbeat: [3170]: info: Exiting status process
> 3746 returned rc 0.
> Oct 30 19:16:27 lvs-test-1 heartbeat: [3170]: debug: RscMgmtProc 'status'
> exited code 0
> Oct 30 19:16:30 lvs-test-1 heartbeat: [3170]: info: all clients are now paused
> Oct 30 19:16:30 lvs-test-1 heartbeat: [3170]: debug: hist->ackseq =13
> Oct 30 19:16:30 lvs-test-1 heartbeat: [3170]: debug: hist->lowseq =12,
> hist->hiseq=114
> Oct 30 19:16:30 lvs-test-1 heartbeat: [3170]: debug:
> Oct 30 19:16:32 lvs-test-1 heartbeat: [3170]: debug: hist->ackseq =13
> Oct 30 19:16:32 lvs-test-1 heartbeat: [3170]: debug: hist->lowseq =12,
> hist->hiseq=115
> Oct 30 19:16:32 lvs-test-1 heartbeat: [3170]: debug:
> Oct 30 19:16:34 lvs-test-1 heartbeat: [3170]: debug: hist->ackseq =13
> Oct 30 19:16:34 lvs-test-1 heartbeat: [3170]: debug: hist->lowseq =12,
> hist->hiseq=116
> Oct 30 19:16:34 lvs-test-1 heartbeat: [3170]: debug:
> Oct 30 19:16:35 lvs-test-1 heartbeat: [3170]: info: all clients are now
> resumed
> # EOF
>
> lvs-test-2 logfile
> ---
> Oct 30 23:13:08 lvs-test-2 heartbeat: [3248]: debug: add_option(keepalive,2)
> Oct 30 23:13:08 lvs-test-2 heartbeat: [3248]: debug: add_option(initdead,20)
> Oct 30 23:13:08 lvs-test-2 heartbeat: [3248]: debug: add_option(deadtime,10)
> Oct 30 23:13:08 lvs-test-2 heartbeat: [3248]: debug: add_option(warntime,5)
> Oct 30 23:13:08 lvs-test-2 heartbeat: [3248]: debug:
> add_option(auto_failback,off)
> Oct 30 23:13:08 lvs-test-2 heartbeat: [3248]: debug:
> add_option(msgfmt,netstring)
> Oct 30 23:13:08 lvs-test-2 heartbeat: [3248]: debug:
> add_option(node,lvs-test-1)
> Oct 30 23:13:08 lvs-test-2 heartbeat: [3248]: debug:
> add_option(node,lvs-test-2)
> Oct 30 23:13:08 lvs-test-2 heartbeat: [3248]: debug: uid=hacluster, gid=<null>
> Oct 30 23:13:08 lvs-test-2 heartbeat: [3248]: debug: uid=hacluster, gid=<null>
> Oct 30 23:13:08 lvs-test-2 heartbeat: [3248]: debug: uid=<null>, gid=haclient
> Oct 30 23:13:08 lvs-test-2 heartbeat: [3248]: debug: uid=root, gid=<null>
> Oct 30 23:13:08 lvs-test-2 heartbeat: [3248]: debug: uid=<null>, gid=haclient
> Oct 30 23:13:08 lvs-test-2 heartbeat: [3248]: debug: Beginning authentication
> parsing
> Oct 30 23:13:08 lvs-test-2 heartbeat: [3248]: debug: 16 max authentication
> methods
> Oct 30 23:13:08 lvs-test-2 heartbeat: [3248]: debug: Keyfile opened
> Oct 30 23:13:08 lvs-test-2 heartbeat: [3248]: debug: Keyfile perms OK
> Oct 30 23:13:08 lvs-test-2 heartbeat: [3248]: debug: 16 max authentication
> methods
> Oct 30 23:13:08 lvs-test-2 heartbeat: [3248]: debug: Found authentication
> method [crc]
> Oct 30 23:13:08 lvs-test-2 heartbeat: [3248]: info: AUTH: i=1: key =
> 0x5d5218, auth=0x2adad07a7550, authname=crc
> Oct 30 23:13:08 lvs-test-2 heartbeat: [3248]: debug: Outbound signing method
> is 1
> Oct 30 23:13:08 lvs-test-2 heartbeat: [3248]: debug: Authentication parsing
> complete [1]
> Oct 30 23:13:08 lvs-test-2 heartbeat: [3248]: debug:
> add_option(cluster,linux-ha)
> Oct 30 23:13:08 lvs-test-2 heartbeat: [3248]: debug: add_option(hopfudge,1)
> Oct 30 23:13:08 lvs-test-2 heartbeat: [3248]: debug: add_option(baud,19200)
> Oct 30 23:13:08 lvs-test-2 heartbeat: [3248]: debug:
> add_option(hbgenmethod,file)
> Oct 30 23:13:08 lvs-test-2 heartbeat: [3248]: debug: add_option(realtime,true)
> Oct 30 23:13:08 lvs-test-2 heartbeat: [3248]: debug:
> add_option(conn_logd_time,60)
> Oct 30 23:13:08 lvs-test-2 heartbeat: [3248]: debug:
> add_option(log_badpack,true)
> Oct 30 23:13:08 lvs-test-2 heartbeat: [3248]: debug:
> add_option(coredumps,true)
> Oct 30 23:13:08 lvs-test-2 heartbeat: [3248]: WARN: Core dumps could be lost
> if multiple dumps occur
> Oct 30 23:13:08 lvs-test-2 heartbeat: [3248]: WARN: Consider setting
> /proc/sys/kernel/core_uses_pid (or equivalent) to 1 for maximum supportability
> Oct 30 23:13:08 lvs-test-2 heartbeat: [3248]: debug: add_option(crm,false)
> Oct 30 23:13:08 lvs-test-2 heartbeat: [3248]: debug: add_option(autojoin,none)
> Oct 30 23:13:08 lvs-test-2 heartbeat: [3248]: debug: add_option(uuidfrom,file)
> Oct 30 23:13:08 lvs-test-2 heartbeat: [3248]: debug:
> add_option(compression,zlib)
> Oct 30 23:13:08 lvs-test-2 heartbeat: [3248]: debug:
> add_option(compression_threshold,2)
> Oct 30 23:13:08 lvs-test-2 heartbeat: [3248]: debug:
> add_option(traditional_compression,no)
> Oct 30 23:13:08 lvs-test-2 heartbeat: [3248]: debug:
> add_option(max_rexmit_delay,250)
> Oct 30 23:13:08 lvs-test-2 heartbeat: [3248]: debug: Setting max_rexmit_delay
> to 250 ms
> Oct 30 23:13:08 lvs-test-2 heartbeat: [3248]: debug:
> add_option(record_config_changes,on)
> Oct 30 23:13:08 lvs-test-2 heartbeat: [3248]: debug:
> add_option(record_pengine_inputs,on)
> Oct 30 23:13:08 lvs-test-2 heartbeat: [3248]: debug:
> add_option(enable_config_writes,on)
> Oct 30 23:13:08 lvs-test-2 heartbeat: [3248]: debug:
> add_option(memreserve,6500)
> Oct 30 23:13:08 lvs-test-2 heartbeat: [3248]: debug: add_option(debug,1)
> Oct 30 23:13:08 lvs-test-2 heartbeat: [3248]: WARN: Logging daemon is
> disabled --enabling logging daemon is recommended
> Oct 30 23:13:08 lvs-test-2 heartbeat: [3248]: info: **************************
> Oct 30 23:13:08 lvs-test-2 heartbeat: [3248]: info: Configuration validated.
> Starting heartbeat 2.0.7
> Oct 30 23:13:08 lvs-test-2 heartbeat: [3248]: debug: HA configuration OK.
> Heartbeat starting.
> Oct 30 23:13:08 lvs-test-2 heartbeat: [3249]: info: heartbeat: version 2.0.7
> Oct 30 23:13:09 lvs-test-2 heartbeat: [3249]: info: Heartbeat generation: 11
> Oct 30 23:13:10 lvs-test-2 heartbeat: [3249]: debug: uuid
> is:6097c427-ad10-4ead-95eb-179b0a3ac540
> Oct 30 23:13:10 lvs-test-2 heartbeat: [3249]: info:
> G_main_add_TriggerHandler: Added signal manual handler
> Oct 30 23:13:10 lvs-test-2 heartbeat: [3249]: info:
> G_main_add_TriggerHandler: Added signal manual handler
> Oct 30 23:13:10 lvs-test-2 heartbeat: [3249]: info: Removing
> /var/run/heartbeat/rsctmp failed, recreating.
> Oct 30 23:13:10 lvs-test-2 heartbeat: [3249]: debug: opening ucast eth2
> (UDP/IP unicast)
> Oct 30 23:13:10 lvs-test-2 heartbeat: [3249]: debug: FIFO process pid: 3314
> Oct 30 23:13:10 lvs-test-2 heartbeat: [3249]: info: glib: ucast: write socket
> priority set to IPTOS_LOWDELAY on eth2
> Oct 30 23:13:10 lvs-test-2 heartbeat: [3249]: info: glib: ucast: bound send
> socket to device: eth2
> Oct 30 23:13:10 lvs-test-2 heartbeat: [3249]: info: glib: ucast: bound
> receive socket to device: eth2
> Oct 30 23:13:10 lvs-test-2 heartbeat: [3249]: info: glib: ucast: started on
> port 694 interface eth2 to 192.168.199.11
> Oct 30 23:13:10 lvs-test-2 heartbeat: [3249]: debug: write process pid: 3315
> Oct 30 23:13:10 lvs-test-2 heartbeat: [3249]: debug: read child process pid:
> 3316
> Oct 30 23:13:10 lvs-test-2 heartbeat: [3249]: info: G_main_add_SignalHandler:
> Added signal handler for signal 17
> Oct 30 23:13:10 lvs-test-2 heartbeat: [3249]: debug: Limiting CPU: 42 CPU
> seconds every 60000 milliseconds
> Oct 30 23:13:10 lvs-test-2 heartbeat: [3249]: debug: pid 3249 locked in
> memory.
> Oct 30 23:13:10 lvs-test-2 heartbeat: [3249]: debug: Waiting for child
> processes to start
> Oct 30 23:13:10 lvs-test-2 heartbeat: [3249]: info: Local status now set to:
> 'up'
> Oct 30 23:13:10 lvs-test-2 heartbeat: [3249]: debug: All your child process
> are belong to us
> Oct 30 23:13:10 lvs-test-2 heartbeat: [3249]: debug: Starting local status
> message @ 2000 ms intervals
> Oct 30 23:13:11 lvs-test-2 heartbeat: [3314]: debug: pid 3314 locked in
> memory.
> Oct 30 23:13:11 lvs-test-2 heartbeat: [3314]: debug: Limiting CPU: 6 CPU
> seconds every 60000 milliseconds
> Oct 30 23:13:11 lvs-test-2 heartbeat: [3315]: debug: pid 3315 locked in
> memory.
> Oct 30 23:13:11 lvs-test-2 heartbeat: [3315]: debug: Limiting CPU: 24 CPU
> seconds every 60000 milliseconds
> Oct 30 23:13:11 lvs-test-2 heartbeat: [3316]: debug: pid 3316 locked in
> memory.
> Oct 30 23:13:11 lvs-test-2 heartbeat: [3316]: debug: Limiting CPU: 6 CPU
> seconds every 60000 milliseconds
> Oct 30 23:13:30 lvs-test-2 heartbeat: [3249]: WARN: node lvs-test-1: is dead
> Oct 30 23:13:30 lvs-test-2 heartbeat: [3249]: info: Comm_now_up(): updating
> status to active
> Oct 30 23:13:30 lvs-test-2 heartbeat: [3249]: info: Local status now set to:
> 'active'
> Oct 30 23:13:30 lvs-test-2 heartbeat: [3249]: debug: Sending local starting
> msg: resourcestate = 0
> Oct 30 23:13:30 lvs-test-2 heartbeat: [3249]: debug: hb_rsc_isstable:
> ResourceMgmt_child_count: 0, other_is_stable: 0, takeover_in_progress: 0,
> going_standby: 0, standby running(ms): 0, resourcestate: 0
> Oct 30 23:13:30 lvs-test-2 heartbeat: [3249]: debug: CreateInitialFilter:
> hb_takeover
> Oct 30 23:13:30 lvs-test-2 heartbeat: [3249]: debug: CreateInitialFilter:
> ip-request-resp
> Oct 30 23:13:30 lvs-test-2 heartbeat: [3249]: debug: CreateInitialFilter:
> status
> Oct 30 23:13:30 lvs-test-2 heartbeat: [3249]: debug: CreateInitialFilter:
> ip-request
> Oct 30 23:13:30 lvs-test-2 heartbeat: [3249]: debug: CreateInitialFilter:
> ask_resources
> Oct 30 23:13:30 lvs-test-2 heartbeat: [3249]: WARN: No STONITH device
> configured.
> Oct 30 23:13:30 lvs-test-2 heartbeat: [3249]: WARN: Shared disks are not
> protected.
> Oct 30 23:13:30 lvs-test-2 heartbeat: [3249]: info: Resources being acquired
> from lvs-test-1.
> Oct 30 23:13:30 lvs-test-2 heartbeat: [3249]: debug: StartNextRemoteRscReq()
> - calling hook
> Oct 30 23:13:30 lvs-test-2 heartbeat: [3249]: debug: notify_world: invoking
> harc: OLD status: active
> Oct 30 23:13:30 lvs-test-2 heartbeat: [3249]: debug: Process [status] started
> pid 3318
> Oct 30 23:13:30 lvs-test-2 heartbeat: [3249]: debug: Starting notify process
> [status]
> Oct 30 23:13:30 lvs-test-2 heartbeat: [3249]: debug: takeover_from_node:
> other now stable
> Oct 30 23:13:30 lvs-test-2 heartbeat: [3249]: debug: Process
> [req_our_resources] started pid 3319
> Oct 30 23:13:30 lvs-test-2 heartbeat: [3318]: debug: notify_world: setting
> SIGCHLD Handler to SIG_DFL
> Oct 30 23:13:30 lvs-test-2 heartbeat: [3318]: debug: notify_world: Running
> harc status
> Oct 30 23:13:30 lvs-test-2 heartbeat: [3319]: debug:
> req_our_resources(/usr/lib/heartbeat/ResourceManager listkeys lvs-test-2)
> Oct 30 23:13:30 lvs-test-2 harc[3318]: info: Running /etc/ha.d/rc.d/status
> status
> Oct 30 23:13:30 lvs-test-2 heartbeat: [3319]: info: No local resources
> [/usr/lib/heartbeat/ResourceManager listkeys lvs-test-2] to acquire.
> Oct 30 23:13:30 lvs-test-2 heartbeat: [3319]: debug: Sending hold resources
> msg: all, stable=1 # req_our_resources()
> Oct 30 23:13:30 lvs-test-2 heartbeat: [3319]: info: FIFO message [type
> resource] written rc=108
> Oct 30 23:13:30 lvs-test-2 heartbeat: [3249]: info: AnnounceTakeover(local 0,
> foreign 1, reason 'T_RESOURCES' (0))
> Oct 30 23:13:30 lvs-test-2 heartbeat: [3249]: info: AnnounceTakeover(local 1,
> foreign 1, reason 'T_RESOURCES(us)' (0))
> Oct 30 23:13:30 lvs-test-2 heartbeat: [3249]: info: Initial resource
> acquisition complete (T_RESOURCES(us))
> Oct 30 23:13:30 lvs-test-2 heartbeat: [3249]: info: STATE 1 => 3
> Oct 30 23:13:30 lvs-test-2 heartbeat: [3249]: debug: hb_rsc_isstable:
> ResourceMgmt_child_count: 2, other_is_stable: 1, takeover_in_progress: 1,
> going_standby: 0, standby running(ms): 0, resourcestate: 3
> Oct 30 23:13:30 lvs-test-2 heartbeat: [3249]: info: Exiting req_our_resources
> process 3319 returned rc 0.
> Oct 30 23:13:30 lvs-test-2 heartbeat: [3249]: debug: RscMgmtProc
> 'req_our_resources' exited code 0
> Oct 30 23:13:30 lvs-test-2 heartbeat: [3249]: info: AnnounceTakeover(local 1,
> foreign 1, reason 'req_our_resources' (1))
> Oct 30 23:13:30 lvs-test-2 heartbeat: [3249]: debug: StartNextRemoteRscReq():
> child count 1
> Oct 30 23:13:31 lvs-test-2 mach_down[3338]: info: Taking over resource group
> IPaddr2::10.204.6.25/24/eth0
> Oct 30 23:13:31 lvs-test-2 ResourceManager[3358]: info: Acquiring resource
> group: lvs-test-1 IPaddr2::10.204.6.25/24/eth0
> Oct 30 23:13:31 lvs-test-2 IPaddr2[3382]: INFO: IPaddr2 Resource is stopped
> Oct 30 23:13:31 lvs-test-2 ResourceManager[3358]: info: Running
> /etc/ha.d/resource.d/IPaddr2 10.204.6.25/24/eth0 start
> Oct 30 23:13:31 lvs-test-2 ResourceManager[3358]: debug: Starting
> /etc/ha.d/resource.d/IPaddr2 10.204.6.25/24/eth0 start
> Oct 30 23:13:31 lvs-test-2 IPaddr2[3596]: INFO: /sbin/ip -f inet addr add
> 10.204.6.25/24 brd 10.204.6.255 dev eth0
> Oct 30 23:13:31 lvs-test-2 IPaddr2[3596]: INFO: /sbin/ip link set eth0 up
> Oct 30 23:13:31 lvs-test-2 IPaddr2[3596]: INFO: /usr/lib/heartbeat/send_arp
> -i 200 -r 5 -p /var/run/heartbeat/rsctmp/send_arp/send_arp-10.204.6.25 eth0
> 10.204.6.25 auto 10.204.6.25 ffffffffffff
> Oct 30 23:13:31 lvs-test-2 IPaddr2[3514]: INFO: IPaddr2 Success
> Oct 30 23:13:31 lvs-test-2 ResourceManager[3358]: debug:
> /etc/ha.d/resource.d/IPaddr2 10.204.6.25/24/eth0 start done. RC=0
> Oct 30 23:13:31 lvs-test-2 mach_down[3338]: info:
> /usr/lib/heartbeat/mach_down: nice_failback: foreign resources acquired
> Oct 30 23:13:31 lvs-test-2 mach_down[3338]: info: mach_down takeover complete
> for node lvs-test-1.
> Oct 30 23:13:31 lvs-test-2 heartbeat: [3249]: info: Exiting status process
> 3318 returned rc 0.
> Oct 30 23:13:31 lvs-test-2 heartbeat: [3249]: debug: RscMgmtProc 'status'
> exited code 0
> Oct 30 23:13:41 lvs-test-2 heartbeat: [3249]: info: Local Resource
> acquisition completed. (none)
> Oct 30 23:13:41 lvs-test-2 heartbeat: [3249]: info: local resource transition
> completed.
> Oct 30 23:13:41 lvs-test-2 heartbeat: [3249]: debug: Sending hold resources
> msg: all, stable=1 # <none>
> Oct 30 23:13:41 lvs-test-2 heartbeat: [3249]: info: AnnounceTakeover(local 1,
> foreign 1, reason 'T_RESOURCES(us)' (1))
> Oct 30 23:13:41 lvs-test-2 heartbeat: [3249]: debug: hb_rsc_isstable:
> ResourceMgmt_child_count: 0, other_is_stable: 1, takeover_in_progress: 1,
> going_standby: 0, standby running(ms): 0, resourcestate: 4
> Oct 30 23:13:41 lvs-test-2 heartbeat: [3249]: debug: hb_rsc_isstable:
> ResourceMgmt_child_count: 0, other_is_stable: 1, takeover_in_progress: 1,
> going_standby: 0, standby running(ms): 0, resourcestate: 4
> Oct 30 23:16:22 lvs-test-2 heartbeat: [3249]: info: Link lvs-test-1:eth2 up.
> Oct 30 23:16:22 lvs-test-2 heartbeat: [3249]: info: Status update for node
> lvs-test-1: status active
> Oct 30 23:16:22 lvs-test-2 heartbeat: [3249]: debug: Status seqno: 113
> msgtime: 1225408588
> Oct 30 23:16:22 lvs-test-2 heartbeat: [3249]: debug: StartNextRemoteRscReq()
> - calling hook
> Oct 30 23:16:22 lvs-test-2 heartbeat: [3249]: debug: notify_world: invoking
> harc: OLD status: active
> Oct 30 23:16:22 lvs-test-2 heartbeat: [3249]: debug: Process [status] started
> pid 3674
> Oct 30 23:16:22 lvs-test-2 heartbeat: [3249]: debug: Starting notify process
> [status]
> Oct 30 23:16:22 lvs-test-2 heartbeat: [3674]: debug: notify_world: setting
> SIGCHLD Handler to SIG_DFL
> Oct 30 23:16:22 lvs-test-2 heartbeat: [3674]: debug: notify_world: Running
> harc status
> Oct 30 23:16:22 lvs-test-2 harc[3674]: info: Running /etc/ha.d/rc.d/status
> status
> Oct 30 23:16:22 lvs-test-2 heartbeat: [3249]: info: Exiting status process
> 3674 returned rc 0.
> Oct 30 23:16:22 lvs-test-2 heartbeat: [3249]: debug: RscMgmtProc 'status'
> exited code 0
> Oct 30 23:16:26 lvs-test-2 heartbeat: [3249]: info: all clients are now paused
> Oct 30 23:16:26 lvs-test-2 heartbeat: [3249]: debug: hist->ackseq =6
> Oct 30 23:16:26 lvs-test-2 heartbeat: [3249]: debug: hist->lowseq =5,
> hist->hiseq=107
> Oct 30 23:16:26 lvs-test-2 heartbeat: [3249]: debug: expecting from lvs-test-1
> Oct 30 23:16:26 lvs-test-2 heartbeat: [3249]: debug: it's ackseq=0
> Oct 30 23:16:26 lvs-test-2 heartbeat: [3249]: debug:
> Oct 30 23:16:28 lvs-test-2 heartbeat: [3249]: debug: hist->ackseq =6
> Oct 30 23:16:28 lvs-test-2 heartbeat: [3249]: debug: hist->lowseq =5,
> hist->hiseq=108
> Oct 30 23:16:28 lvs-test-2 heartbeat: [3249]: debug: expecting from lvs-test-1
> Oct 30 23:16:28 lvs-test-2 heartbeat: [3249]: debug: it's ackseq=0
> Oct 30 23:16:28 lvs-test-2 heartbeat: [3249]: debug:
> Oct 30 23:16:30 lvs-test-2 heartbeat: [3249]: debug: hist->ackseq =6
> Oct 30 23:16:30 lvs-test-2 heartbeat: [3249]: debug: hist->lowseq =5,
> hist->hiseq=109
> Oct 30 23:16:30 lvs-test-2 heartbeat: [3249]: debug: expecting from lvs-test-1
> Oct 30 23:16:30 lvs-test-2 heartbeat: [3249]: debug: it's ackseq=0
> Oct 30 23:16:30 lvs-test-2 heartbeat: [3249]: debug:
> Oct 30 23:16:32 lvs-test-2 heartbeat: [3249]: debug: hist->ackseq =6
> Oct 30 23:16:32 lvs-test-2 heartbeat: [3249]: debug: hist->lowseq =5,
> hist->hiseq=110
> Oct 30 23:16:32 lvs-test-2 heartbeat: [3249]: debug: expecting from lvs-test-1
> Oct 30 23:16:32 lvs-test-2 heartbeat: [3249]: debug: it's ackseq=0
> Oct 30 23:16:32 lvs-test-2 heartbeat: [3249]: debug:
> Oct 30 23:16:34 lvs-test-2 heartbeat: [3249]: debug: hist->ackseq =6
> Oct 30 23:16:34 lvs-test-2 heartbeat: [3249]: debug: hist->lowseq =5,
> hist->hiseq=111
> Oct 30 23:16:34 lvs-test-2 heartbeat: [3249]: debug: expecting from lvs-test-1
> Oct 30 23:16:34 lvs-test-2 heartbeat: [3249]: debug: it's ackseq=0
> Oct 30 23:16:34 lvs-test-2 heartbeat: [3249]: debug:
> Oct 30 23:16:36 lvs-test-2 heartbeat: [3249]: debug: hist->ackseq =6
> Oct 30 23:16:36 lvs-test-2 heartbeat: [3249]: debug: hist->lowseq =5,
> hist->hiseq=112
> Oct 30 23:16:36 lvs-test-2 heartbeat: [3249]: debug: expecting from lvs-test-1
> Oct 30 23:16:36 lvs-test-2 heartbeat: [3249]: debug: it's ackseq=0
> Oct 30 23:16:36 lvs-test-2 heartbeat: [3249]: debug:
> Oct 30 23:16:36 lvs-test-2 heartbeat: [3249]: info: all clients are now
> resumed
> Oct 30 23:17:16 lvs-test-2 heartbeat: [3249]: debug:
> APIregistration_dispatch() {
> Oct 30 23:17:16 lvs-test-2 heartbeat: [3249]: debug: process_registerevent() {
> Oct 30 23:17:16 lvs-test-2 heartbeat: [3249]: debug: client->gsource =
> 0x637408
> Oct 30 23:17:16 lvs-test-2 heartbeat: [3249]: debug:
> }/*process_registerevent*/;
> Oct 30 23:17:16 lvs-test-2 heartbeat: [3249]: debug:
> }/*APIregistration_dispatch*/;
> Oct 30 23:17:16 lvs-test-2 heartbeat: [3249]: debug: Checking client
> authorization for client 3689 (0:105)
> Oct 30 23:17:16 lvs-test-2 heartbeat: [3249]: debug:
> create_seq_snapshot_table:no missing packets found for node lvs-test-1
> Oct 30 23:17:16 lvs-test-2 heartbeat: [3249]: debug:
> create_seq_snapshot_table:no missing packets found for node lvs-test-2
> Oct 30 23:17:16 lvs-test-2 heartbeat: [3249]: debug: Signing on API client
> 3689 ('casual')
> Oct 30 23:17:16 lvs-test-2 heartbeat: [3249]: debug: hb_rsc_isstable:
> ResourceMgmt_child_count: 0, other_is_stable: 1, takeover_in_progress: 1,
> going_standby: 0, standby running(ms): 0, resourcestate: 4
> Oct 30 23:17:16 lvs-test-2 heartbeat: [3249]: debug: Signing client 3689 off
> Oct 30 23:17:16 lvs-test-2 heartbeat: [3249]: debug:
> G_remove_client(pid=3689, reason='signoff' gsource=0x637408) {
> Oct 30 23:17:16 lvs-test-2 heartbeat: [3249]: debug: api_remove_client_int:
> removing pid [3689] reason: signoff
> Oct 30 23:17:16 lvs-test-2 heartbeat: [3249]: debug: }/*G_remove_client;*/
> Oct 30 23:23:10 lvs-test-2 heartbeat: [3249]: info: Daily informational
> memory statistics
> Oct 30 23:23:10 lvs-test-2 heartbeat: [3249]: info: MSG stats: 9/619 ms age 0
> [pid3249/MST_CONTROL]
> Oct 30 23:23:10 lvs-test-2 heartbeat: [3249]: info: ha_malloc stats:
> 634/21506 87224/41510 [pid3249/MST_CONTROL]
> Oct 30 23:23:10 lvs-test-2 heartbeat: [3249]: info: RealMalloc stats: 364356
> total malloc bytes. pid [3249/MST_CONTROL]
> Oct 30 23:23:10 lvs-test-2 heartbeat: [3249]: info: Current arena value: 0
> Oct 30 23:23:10 lvs-test-2 heartbeat: [3249]: info: MSG stats: 0/1 ms age
> 579430 [pid3314/HBFIFO]
> Oct 30 23:23:10 lvs-test-2 heartbeat: [3249]: info: ha_malloc stats: 341/404
> 40556/18454 [pid3314/HBFIFO]
> Oct 30 23:23:10 lvs-test-2 heartbeat: [3249]: info: RealMalloc stats: 43132
> total malloc bytes. pid [3314/HBFIFO]
> Oct 30 23:23:10 lvs-test-2 heartbeat: [3249]: info: Current arena value: 0
> Oct 30 23:23:10 lvs-test-2 heartbeat: [3249]: info: MSG stats: 0/0 ms age
> 17181271090 [pid3315/HBWRITE]
> Oct 30 23:23:10 lvs-test-2 heartbeat: [3249]: info: ha_malloc stats: 338/766
> 40632/18627 [pid3315/HBWRITE]
> Oct 30 23:23:10 lvs-test-2 heartbeat: [3249]: info: RealMalloc stats: 49128
> total malloc bytes. pid [3315/HBWRITE]
> Oct 30 23:23:10 lvs-test-2 heartbeat: [3249]: info: Current arena value: 0
> Oct 30 23:23:10 lvs-test-2 heartbeat: [3249]: info: MSG stats: 0/0 ms age
> 17181271090 [pid3316/HBREAD]
> Oct 30 23:23:10 lvs-test-2 heartbeat: [3249]: info: ha_malloc stats: 339/878
> 40724/18691 [pid3316/HBREAD]
> Oct 30 23:23:10 lvs-test-2 heartbeat: [3249]: info: RealMalloc stats: 41100
> total malloc bytes. pid [3316/HBREAD]
> Oct 30 23:23:10 lvs-test-2 heartbeat: [3249]: info: Current arena value: 0
> Oct 30 23:23:10 lvs-test-2 heartbeat: [3249]: info: These are nothing to
> worry about.
>
>
>
>
>
> _______________________________________________
> Linux-HA mailing list
> [email protected]
> http://lists.linux-ha.org/mailman/listinfo/linux-ha
> See also: http://linux-ha.org/ReportingProblems
_______________________________________________
Linux-HA mailing list
[email protected]
http://lists.linux-ha.org/mailman/listinfo/linux-ha
See also: http://linux-ha.org/ReportingProblems
