Feb 20 08:57:39 linux cib: [3103]: info: cib_stats: Processed 87 operations (4482.00us average, 0% utilization) in the last 10min
Feb 20 09:01:01 linux crond(pam_unix)[3376]: session opened for user root by (uid=0)
Feb 20 09:01:01 linux crond(pam_unix)[3376]: session closed for user root
Feb 20 09:24:06 linux sshd(pam_unix)[3396]: session opened for user root by (uid=0)
Feb 20 09:24:13 linux sshd(pam_unix)[3415]: session opened for user root by (uid=0)
Feb 20 09:24:14 linux logger: Mark:HB_REPORT:1203517431
Feb 20 09:24:24 linux sshd(pam_unix)[3449]: session opened for user root by (uid=0)
Feb 20 09:24:38 linux crmadmin: [3687]: info: Invoked: crmadmin -D 
Feb 20 09:24:39 linux cibadmin: [3691]: info: Invoked: cibadmin -Ql 
Feb 20 09:24:39 linux ccm_tool: [3692]: info: mem_handle_event: Got an event OC_EV_MS_NEW_MEMBERSHIP from ccm
Feb 20 09:24:39 linux ccm_tool: [3692]: info: mem_handle_event: instance=7, nodes=2, new=2, lost=0, n_idx=0, new_idx=0, old_idx=4
Feb 20 09:24:39 linux ccm: [3102]: info: client (pid=3692) removed from ccm
Feb 20 09:24:39 linux crm_verify: [3695]: WARN: unpack_rsc_op: Processing failed op Apache_start_0 on postit.mtu.edu: Error
Feb 20 09:24:39 linux crm_verify: [3695]: WARN: unpack_rsc_op: Compatability handling for failed op Apache_start_0 on postit.mtu.edu
Feb 20 09:24:39 linux crm_verify: [3695]: WARN: unpack_rsc_op: Processing failed op Apache_start_0 on linux.mtu.edu: Error
Feb 20 09:24:39 linux crm_verify: [3695]: WARN: unpack_rsc_op: Compatability handling for failed op Apache_start_0 on linux.mtu.edu
Feb 20 09:24:40 linux crmadmin: [3715]: info: Invoked: crmadmin -D 
Feb 20 09:25:04 linux crmd: [5082]: info: main: CRM Hg Version: node: 552305612591183b1628baa5bc6e903e0f1e26a3 
Feb 20 09:27:39 linux cib: [3103]: info: cib_stats: Processed 2 operations (10000.00us average, 0% utilization) in the last 10min
Feb 20 09:29:14 linux lrmd: [3104]: info: rsc:IP: start
Feb 20 09:29:14 linux crmd: [3107]: info: do_lrm_rsc_op: Performing op=IP_start_0 key=6:42:4fe91bda-9521-4be5-bcce-247dc640f773)
Feb 20 09:29:15 linux IPaddr2[5105]: [5134]: INFO: ip -f inet addr add 192.168.1.30/24 brd 192.168.1.255 dev eth0
Feb 20 09:29:15 linux IPaddr2[5105]: [5136]: INFO: ip link set eth0 up
Feb 20 09:29:15 linux IPaddr2[5105]: [5138]: INFO: /usr/lib/heartbeat/send_arp -i 200 -r 5 -p /var/run/heartbeat/rsctmp/send_arp/send_arp-192.168.1.30 eth0 192.168.1.30 auto not_used not_used
Feb 20 09:29:15 linux crmd: [3107]: info: process_lrm_event: LRM operation IP_start_0 (call=14, rc=0) complete 
Feb 20 09:29:16 linux crmd: [3107]: notice: crmd_client_status_callback: Status update: Client postit.mtu.edu/crmd now has status [offline]
Feb 20 09:29:16 linux crmd: [3107]: info: crmd_client_status_callback: Got client status callback - our DC is dead
Feb 20 09:29:16 linux crmd: [3107]: info: do_state_transition: State transition S_NOT_DC -> S_ELECTION [ input=I_ELECTION cause=C_CRMD_STATUS_CALLBACK origin=crmd_client_status_callback ]
Feb 20 09:29:16 linux crmd: [3107]: info: update_dc: Unset DC postit.mtu.edu
Feb 20 09:29:16 linux crmd: [3107]: info: do_election_count_vote: Updated voted hash for linux.mtu.edu to vote
Feb 20 09:29:16 linux crmd: [3107]: info: do_election_count_vote: Election ignore: our vote (linux.mtu.edu)
Feb 20 09:29:16 linux crmd: [3107]: info: do_election_check: Still waiting on 1 non-votes (2 total)
Feb 20 09:29:16 linux cib: [3103]: info: cib_process_shutdown_req: Shutdown REQ from postit.mtu.edu
Feb 20 09:29:17 linux cib: [3103]: info: cib_client_status_callback: Status update: Client postit.mtu.edu/cib now has status [leave]
Feb 20 09:29:17 linux cib: [3103]: info: mem_handle_event: Got an event OC_EV_MS_NOT_PRIMARY from ccm
Feb 20 09:29:17 linux cib: [3103]: info: mem_handle_event: instance=7, nodes=2, new=2, lost=0, n_idx=0, new_idx=0, old_idx=4
Feb 20 09:29:17 linux crmd: [3107]: info: mem_handle_event: Got an event OC_EV_MS_NOT_PRIMARY from ccm
Feb 20 09:29:17 linux crmd: [3107]: info: mem_handle_event: instance=7, nodes=2, new=2, lost=0, n_idx=0, new_idx=0, old_idx=4
Feb 20 09:29:17 linux crmd: [3107]: info: crmd_ccm_msg_callback: Quorum lost after event=NOT PRIMARY (id=7)
Feb 20 09:29:23 linux crmd: [3107]: info: mem_handle_event: Got an event OC_EV_MS_INVALID from ccm
Feb 20 09:29:23 linux crmd: [3107]: info: mem_handle_event: no mbr_track info
Feb 20 09:29:23 linux crmd: [3107]: info: mem_handle_event: Got an event OC_EV_MS_NEW_MEMBERSHIP from ccm
Feb 20 09:29:23 linux crmd: [3107]: info: mem_handle_event: instance=8, nodes=1, new=0, lost=1, n_idx=0, new_idx=1, old_idx=3
Feb 20 09:29:23 linux crmd: [3107]: info: crmd_ccm_msg_callback: Quorum (re)attained after event=NEW MEMBERSHIP (id=8)
Feb 20 09:29:23 linux cib: [3103]: info: mem_handle_event: Got an event OC_EV_MS_INVALID from ccm
Feb 20 09:29:23 linux ccm: [3102]: info: Break tie for 2 nodes cluster
Feb 20 09:29:23 linux crmd: [3107]: info: ccm_event_detail: NEW MEMBERSHIP: trans=8, nodes=1, new=0, lost=1 n_idx=0, new_idx=1, old_idx=3
Feb 20 09:29:23 linux cib: [3103]: info: mem_handle_event: no mbr_track info
Feb 20 09:29:23 linux crmd: [3107]: info: ccm_event_detail: 	CURRENT: linux.mtu.edu [nodeid=0, born=8]
Feb 20 09:29:23 linux cib: [3103]: info: mem_handle_event: Got an event OC_EV_MS_NEW_MEMBERSHIP from ccm
Feb 20 09:29:23 linux crmd: [3107]: info: ccm_event_detail: 	LOST:    postit.mtu.edu [nodeid=1, born=1]
Feb 20 09:29:23 linux cib: [3103]: info: mem_handle_event: instance=8, nodes=1, new=0, lost=1, n_idx=0, new_idx=1, old_idx=3
Feb 20 09:29:23 linux crmd: [3107]: info: do_state_transition: State transition S_ELECTION -> S_INTEGRATION [ input=I_ELECTION_DC cause=C_FSA_INTERNAL origin=do_election_check ]
Feb 20 09:29:23 linux cib: [3103]: info: cib_ccm_msg_callback: LOST: postit.mtu.edu
Feb 20 09:29:23 linux crmd: [3107]: info: start_subsystem: Starting sub-system "tengine"
Feb 20 09:29:23 linux cib: [3103]: info: cib_ccm_msg_callback: PEER: linux.mtu.edu
Feb 20 09:29:23 linux crmd: [3107]: info: start_subsystem: Starting sub-system "pengine"
Feb 20 09:29:23 linux crmd: [3107]: info: do_dc_takeover: Taking over DC status for this partition
Feb 20 09:29:23 linux cib: [3103]: info: cib_process_readwrite: We are now in R/W mode
Feb 20 09:29:23 linux crmd: [3107]: info: join_make_offer: Making join offers based on membership 8
Feb 20 09:29:23 linux crmd: [3107]: info: do_dc_join_offer_all: join-1: Waiting on 1 outstanding join acks
Feb 20 09:29:23 linux tengine: [5142]: info: G_main_add_SignalHandler: Added signal handler for signal 15
Feb 20 09:29:23 linux tengine: [5142]: info: G_main_add_TriggerHandler: Added signal manual handler
Feb 20 09:29:23 linux tengine: [5142]: info: G_main_add_TriggerHandler: Added signal manual handler
Feb 20 09:29:23 linux crmd: [3107]: info: update_dc: Set DC to linux.mtu.edu (2.0)
Feb 20 09:29:23 linux cib: [3103]: info: cib_null_callback: Setting cib_diff_notify callbacks for tengine: on
Feb 20 09:29:23 linux tengine: [5142]: info: te_init: Registering TE UUID: 4746c560-21ca-4bf3-93a4-2a2f7919abc9
Feb 20 09:29:23 linux tengine: [5142]: info: set_graph_functions: Setting custom graph functions
Feb 20 09:29:23 linux tengine: [5142]: info: unpack_graph: Unpacked transition -1: 0 actions in 0 synapses
Feb 20 09:29:23 linux tengine: [5142]: info: te_init: Starting tengine
Feb 20 09:29:23 linux tengine: [5142]: info: te_connect_stonith: Attempting connection to fencing daemon...
Feb 20 09:29:23 linux pengine: [5143]: info: G_main_add_SignalHandler: Added signal handler for signal 15
Feb 20 09:29:23 linux pengine: [5143]: info: pe_init: Starting pengine
Feb 20 09:29:23 linux crmd: [3107]: info: do_state_transition: State transition S_INTEGRATION -> S_FINALIZE_JOIN [ input=I_INTEGRATED cause=C_FSA_INTERNAL origin=check_join_state ]
Feb 20 09:29:23 linux crmd: [3107]: info: do_state_transition: All 1 cluster nodes responded to the join offer.
Feb 20 09:29:23 linux cib: [3103]: info: sync_our_cib: Syncing CIB to all peers
Feb 20 09:29:23 linux crmd: [3107]: info: update_attrd: Connecting to attrd...
Feb 20 09:29:23 linux attrd: [3106]: info: attrd_local_callback: Sending full refresh
Feb 20 09:29:23 linux crmd: [3107]: info: update_dc: Set DC to linux.mtu.edu (2.0)
Feb 20 09:29:24 linux crmd: [3107]: info: do_dc_join_ack: join-1: Updating node state to member for linux.mtu.edu
Feb 20 09:29:24 linux crmd: [3107]: info: do_state_transition: State transition S_FINALIZE_JOIN -> S_POLICY_ENGINE [ input=I_FINALIZED cause=C_FSA_INTERNAL origin=check_join_state ]
Feb 20 09:29:24 linux crmd: [3107]: info: do_state_transition: All 1 cluster nodes are eligible to run resources.
Feb 20 09:29:24 linux tengine: [5142]: info: te_connect_stonith: Connected
Feb 20 09:29:24 linux tengine: [5142]: info: update_abort_priority: Abort priority upgraded to 1000000
Feb 20 09:29:24 linux tengine: [5142]: info: update_abort_priority: 'DC Takeover' abort superceeded
Feb 20 09:29:24 linux tengine: [5142]: info: process_graph_event: Action Apache_monitor_0 initiated by a different transitioner
Feb 20 09:29:24 linux pengine: [5143]: info: determine_online_status: Node linux.mtu.edu is online
Feb 20 09:29:24 linux pengine: [5143]: WARN: unpack_rsc_op: Processing failed op Apache_start_0 on linux.mtu.edu: Error
Feb 20 09:29:24 linux pengine: [5143]: WARN: unpack_rsc_op: Compatability handling for failed op Apache_start_0 on linux.mtu.edu
Feb 20 09:29:24 linux pengine: [5143]: notice: group_print: Resource Group: Webserver
Feb 20 09:29:24 linux pengine: [5143]: notice: native_print:     IP	(heartbeat::ocf:IPaddr2):	Started linux.mtu.edu
Feb 20 09:29:24 linux pengine: [5143]: notice: native_print:     Apache	(heartbeat::ocf:apache):	Stopped 
Feb 20 09:29:24 linux pengine: [5143]: WARN: native_color: Resource Apache cannot run anywhere
Feb 20 09:29:24 linux pengine: [5143]: notice: NoRoleChange: Leave resource IP	(linux.mtu.edu)
Feb 20 09:29:24 linux pengine: [5143]: WARN: process_pe_message: Transition 0: WARNINGs found during PE processing. PEngine Input stored in: /var/lib/heartbeat/pengine/pe-warn-2448.bz2
Feb 20 09:29:24 linux pengine: [5143]: info: process_pe_message: Configuration WARNINGs found during PE processing.  Please run "crm_verify -L" to identify issues.
Feb 20 09:29:24 linux pengine: [5143]: info: determine_online_status: Node linux.mtu.edu is online
Feb 20 09:29:24 linux pengine: [5143]: WARN: unpack_rsc_op: Processing failed op Apache_start_0 on linux.mtu.edu: Error
Feb 20 09:29:24 linux pengine: [5143]: WARN: unpack_rsc_op: Compatability handling for failed op Apache_start_0 on linux.mtu.edu
Feb 20 09:29:24 linux pengine: [5143]: notice: group_print: Resource Group: Webserver
Feb 20 09:29:24 linux pengine: [5143]: notice: native_print:     IP	(heartbeat::ocf:IPaddr2):	Started linux.mtu.edu
Feb 20 09:29:24 linux pengine: [5143]: notice: native_print:     Apache	(heartbeat::ocf:apache):	Stopped 
Feb 20 09:29:24 linux pengine: [5143]: WARN: native_color: Resource Apache cannot run anywhere
Feb 20 09:29:24 linux pengine: [5143]: notice: NoRoleChange: Leave resource IP	(linux.mtu.edu)
Feb 20 09:29:24 linux crmd: [3107]: info: do_state_transition: State transition S_POLICY_ENGINE -> S_TRANSITION_ENGINE [ input=I_PE_SUCCESS cause=C_IPC_MESSAGE origin=route_message ]
Feb 20 09:29:24 linux tengine: [5142]: info: unpack_graph: Unpacked transition 1: 0 actions in 0 synapses
Feb 20 09:29:24 linux tengine: [5142]: info: run_graph: Transition 1: (Complete=0, Pending=0, Fired=0, Skipped=0, Incomplete=0)
Feb 20 09:29:24 linux tengine: [5142]: info: notify_crmd: Transition 1 status: te_complete - <null>
Feb 20 09:29:24 linux crmd: [3107]: info: do_state_transition: State transition S_TRANSITION_ENGINE -> S_IDLE [ input=I_TE_SUCCESS cause=C_IPC_MESSAGE origin=route_message ]
Feb 20 09:29:24 linux pengine: [5143]: WARN: process_pe_message: Transition 1: WARNINGs found during PE processing. PEngine Input stored in: /var/lib/heartbeat/pengine/pe-warn-2449.bz2
Feb 20 09:29:24 linux pengine: [5143]: info: process_pe_message: Configuration WARNINGs found during PE processing.  Please run "crm_verify -L" to identify issues.
Feb 20 09:29:28 linux heartbeat: [3094]: WARN: node postit.mtu.edu: is dead
Feb 20 09:29:28 linux heartbeat: [3094]: info: Link postit.mtu.edu:eth1 dead.
Feb 20 09:29:28 linux crmd: [3107]: notice: crmd_ha_status_callback: Status update: Node postit.mtu.edu now has status [dead]
Feb 20 09:29:41 linux heartbeat: [3094]: info: Heartbeat restart on node postit.mtu.edu
Feb 20 09:29:41 linux heartbeat: [3094]: info: Link postit.mtu.edu:eth1 up.
Feb 20 09:29:41 linux heartbeat: [3094]: info: Status update for node postit.mtu.edu: status init
Feb 20 09:29:41 linux heartbeat: [3094]: info: Status update for node postit.mtu.edu: status up
Feb 20 09:29:41 linux crmd: [3107]: notice: crmd_ha_status_callback: Status update: Node postit.mtu.edu now has status [init]
Feb 20 09:29:41 linux crmd: [3107]: notice: crmd_ha_status_callback: Status update: Node postit.mtu.edu now has status [up]
Feb 20 09:29:42 linux heartbeat: [3094]: info: Status update for node postit.mtu.edu: status active
Feb 20 09:29:42 linux crmd: [3107]: notice: crmd_ha_status_callback: Status update: Node postit.mtu.edu now has status [active]
Feb 20 09:29:42 linux cib: [3103]: info: cib_client_status_callback: Status update: Client postit.mtu.edu/cib now has status [join]
Feb 20 09:29:46 linux heartbeat: [3094]: WARN: 1 lost packet(s) for [postit.mtu.edu] [18:20]
Feb 20 09:29:46 linux heartbeat: [3094]: info: No pkts missing from postit.mtu.edu!
Feb 20 09:29:46 linux crmd: [3107]: notice: crmd_client_status_callback: Status update: Client postit.mtu.edu/crmd now has status [online]
Feb 20 09:29:46 linux cib: [3103]: info: mem_handle_event: Got an event OC_EV_MS_INVALID from ccm
Feb 20 09:29:46 linux cib: [3103]: info: mem_handle_event: no mbr_track info
Feb 20 09:29:46 linux cib: [3103]: info: mem_handle_event: Got an event OC_EV_MS_NEW_MEMBERSHIP from ccm
Feb 20 09:29:46 linux cib: [3103]: info: mem_handle_event: instance=9, nodes=2, new=1, lost=0, n_idx=0, new_idx=2, old_idx=4
Feb 20 09:29:46 linux cib: [3103]: info: cib_ccm_msg_callback: PEER: linux.mtu.edu
Feb 20 09:29:46 linux cib: [3103]: info: cib_ccm_msg_callback: PEER: postit.mtu.edu
Feb 20 09:29:46 linux crmd: [3107]: info: mem_handle_event: Got an event OC_EV_MS_INVALID from ccm
Feb 20 09:29:46 linux crmd: [3107]: info: mem_handle_event: no mbr_track info
Feb 20 09:29:46 linux crmd: [3107]: info: mem_handle_event: Got an event OC_EV_MS_NEW_MEMBERSHIP from ccm
Feb 20 09:29:46 linux crmd: [3107]: info: mem_handle_event: instance=9, nodes=2, new=1, lost=0, n_idx=0, new_idx=2, old_idx=4
Feb 20 09:29:46 linux crmd: [3107]: info: crmd_ccm_msg_callback: Quorum (re)attained after event=NEW MEMBERSHIP (id=9)
Feb 20 09:29:46 linux crmd: [3107]: info: ccm_event_detail: NEW MEMBERSHIP: trans=9, nodes=2, new=1, lost=0 n_idx=0, new_idx=2, old_idx=4
Feb 20 09:29:46 linux crmd: [3107]: info: ccm_event_detail: 	CURRENT: linux.mtu.edu [nodeid=0, born=7]
Feb 20 09:29:46 linux crmd: [3107]: info: ccm_event_detail: 	CURRENT: postit.mtu.edu [nodeid=1, born=9]
Feb 20 09:29:46 linux crmd: [3107]: info: ccm_event_detail: 	NEW:     postit.mtu.edu [nodeid=1, born=9]
Feb 20 09:29:46 linux tengine: [5142]: info: te_update_diff: Aborting on transient_attributes deletions
Feb 20 09:29:46 linux crmd: [3107]: info: do_state_transition: State transition S_IDLE -> S_POLICY_ENGINE [ input=I_PE_CALC cause=C_IPC_MESSAGE origin=route_message ]
Feb 20 09:29:46 linux tengine: [5142]: info: update_abort_priority: Abort priority upgraded to 1000000
Feb 20 09:29:46 linux crmd: [3107]: info: do_state_transition: Membership changed: 8 -> 9 - join restart
Feb 20 09:29:46 linux crmd: [3107]: info: do_state_transition: State transition S_POLICY_ENGINE -> S_INTEGRATION [ input=I_NODE_JOIN cause=C_FSA_INTERNAL origin=do_state_transition ]
Feb 20 09:29:46 linux crmd: [3107]: info: update_dc: Unset DC linux.mtu.edu
Feb 20 09:29:46 linux crmd: [3107]: info: join_make_offer: Making join offers based on membership 9
Feb 20 09:29:46 linux crmd: [3107]: info: do_dc_join_offer_all: join-2: Waiting on 2 outstanding join acks
Feb 20 09:29:46 linux heartbeat: [3094]: WARN: 1 lost packet(s) for [postit.mtu.edu] [22:24]
Feb 20 09:29:46 linux heartbeat: [3094]: info: No pkts missing from postit.mtu.edu!
Feb 20 09:29:46 linux crmd: [3107]: info: update_dc: Set DC to linux.mtu.edu (2.0)
Feb 20 09:29:49 linux crmd: [3107]: info: update_dc: Unset DC linux.mtu.edu
Feb 20 09:29:49 linux crmd: [3107]: info: do_dc_join_offer_all: join-3: Waiting on 2 outstanding join acks
Feb 20 09:29:49 linux crmd: [3107]: info: update_dc: Set DC to linux.mtu.edu (2.0)
Feb 20 09:29:50 linux crmd: [3107]: info: do_state_transition: State transition S_INTEGRATION -> S_FINALIZE_JOIN [ input=I_INTEGRATED cause=C_FSA_INTERNAL origin=check_join_state ]
Feb 20 09:29:50 linux cib: [3103]: info: sync_our_cib: Syncing CIB to all peers
Feb 20 09:29:50 linux attrd: [3106]: info: attrd_local_callback: Sending full refresh
Feb 20 09:29:50 linux crmd: [3107]: info: do_state_transition: All 2 cluster nodes responded to the join offer.
Feb 20 09:29:50 linux crmd: [3107]: info: update_dc: Set DC to linux.mtu.edu (2.0)
Feb 20 09:29:50 linux crmd: [3107]: info: do_dc_join_ack: join-3: Updating node state to member for linux.mtu.edu
Feb 20 09:29:51 linux crmd: [3107]: info: do_dc_join_ack: join-3: Updating node state to member for postit.mtu.edu
Feb 20 09:29:51 linux crmd: [3107]: info: do_state_transition: State transition S_FINALIZE_JOIN -> S_POLICY_ENGINE [ input=I_FINALIZED cause=C_FSA_INTERNAL origin=check_join_state ]
Feb 20 09:29:51 linux crmd: [3107]: info: do_state_transition: All 2 cluster nodes are eligible to run resources.
Feb 20 09:29:51 linux pengine: [5143]: info: determine_online_status: Node postit.mtu.edu is online
Feb 20 09:29:51 linux pengine: [5143]: info: determine_online_status: Node linux.mtu.edu is online
Feb 20 09:29:51 linux pengine: [5143]: WARN: unpack_rsc_op: Processing failed op Apache_start_0 on linux.mtu.edu: Error
Feb 20 09:29:51 linux pengine: [5143]: WARN: unpack_rsc_op: Compatability handling for failed op Apache_start_0 on linux.mtu.edu
Feb 20 09:29:51 linux pengine: [5143]: notice: group_print: Resource Group: Webserver
Feb 20 09:29:51 linux pengine: [5143]: notice: native_print:     IP	(heartbeat::ocf:IPaddr2):	Started linux.mtu.edu
Feb 20 09:29:51 linux pengine: [5143]: notice: native_print:     Apache	(heartbeat::ocf:apache):	Stopped 
Feb 20 09:29:51 linux pengine: [5143]: notice: NoRoleChange: Move  resource IP	(linux.mtu.edu -> postit.mtu.edu)
Feb 20 09:29:51 linux pengine: [5143]: notice: StopRsc:   linux.mtu.edu	Stop IP
Feb 20 09:29:51 linux pengine: [5143]: notice: StartRsc:  postit.mtu.edu	Start IP
Feb 20 09:29:51 linux pengine: [5143]: notice: StartRsc:  postit.mtu.edu	Start Apache
Feb 20 09:29:51 linux crmd: [3107]: info: do_state_transition: State transition S_POLICY_ENGINE -> S_TRANSITION_ENGINE [ input=I_PE_SUCCESS cause=C_IPC_MESSAGE origin=route_message ]
Feb 20 09:29:51 linux tengine: [5142]: info: unpack_graph: Unpacked transition 2: 13 actions in 13 synapses
Feb 20 09:29:51 linux tengine: [5142]: info: te_pseudo_action: Pseudo action 12 fired and confirmed
Feb 20 09:29:51 linux tengine: [5142]: info: send_rsc_command: Initiating action 3: IP_monitor_0 on postit.mtu.edu
Feb 20 09:29:51 linux tengine: [5142]: info: send_rsc_command: Initiating action 4: Apache_monitor_0 on postit.mtu.edu
Feb 20 09:29:51 linux tengine: [5142]: info: te_pseudo_action: Pseudo action 15 fired and confirmed
Feb 20 09:29:51 linux tengine: [5142]: info: send_rsc_command: Initiating action 7: IP_stop_0 on linux.mtu.edu
Feb 20 09:29:51 linux crmd: [3107]: info: do_lrm_rsc_op: Performing op=IP_stop_0 key=7:2:4746c560-21ca-4bf3-93a4-2a2f7919abc9)
Feb 20 09:29:51 linux lrmd: [3104]: info: rsc:IP: stop
Feb 20 09:29:51 linux pengine: [5143]: info: process_pe_message: Transition 2: PEngine Input stored in: /var/lib/heartbeat/pengine/pe-input-977.bz2
Feb 20 09:29:52 linux IPaddr2[5144]: [5173]: INFO: ip -f inet addr delete 192.168.1.30/24 dev eth0
Feb 20 09:29:52 linux IPaddr2[5144]: [5175]: INFO: ip -o -f inet addr show eth0
Feb 20 09:29:52 linux crmd: [3107]: info: process_lrm_event: LRM operation IP_stop_0 (call=15, rc=0) complete 
Feb 20 09:29:52 linux tengine: [5142]: info: match_graph_event: Action IP_stop_0 (7) confirmed on linux.mtu.edu (rc=0)
Feb 20 09:29:52 linux tengine: [5142]: info: te_pseudo_action: Pseudo action 13 fired and confirmed
Feb 20 09:29:52 linux tengine: [5142]: info: te_pseudo_action: Pseudo action 6 fired and confirmed
Feb 20 09:29:53 linux tengine: [5142]: info: match_graph_event: Action Apache_monitor_0 (4) confirmed on postit.mtu.edu (rc=0)
Feb 20 09:29:53 linux tengine: [5142]: info: match_graph_event: Action IP_monitor_0 (3) confirmed on postit.mtu.edu (rc=0)
Feb 20 09:29:53 linux tengine: [5142]: info: send_rsc_command: Initiating action 2: probe_complete on postit.mtu.edu
Feb 20 09:29:53 linux tengine: [5142]: info: te_pseudo_action: Pseudo action 1 fired and confirmed
Feb 20 09:29:53 linux tengine: [5142]: info: te_pseudo_action: Pseudo action 10 fired and confirmed
Feb 20 09:29:53 linux tengine: [5142]: info: send_rsc_command: Initiating action 8: IP_start_0 on postit.mtu.edu
Feb 20 09:29:53 linux tengine: [5142]: info: extract_event: Aborting on transient_attributes changes for aaed2ff6-07cd-476f-8d98-259570a5480e
Feb 20 09:29:53 linux tengine: [5142]: info: update_abort_priority: Abort priority upgraded to 1000000
Feb 20 09:29:53 linux tengine: [5142]: info: update_abort_priority: Abort action 0 superceeded by 2
Feb 20 09:29:54 linux tengine: [5142]: info: match_graph_event: Action IP_start_0 (8) confirmed on postit.mtu.edu (rc=0)
Feb 20 09:29:54 linux tengine: [5142]: info: run_graph: ====================================================
Feb 20 09:29:54 linux tengine: [5142]: notice: run_graph: Transition 2: (Complete=11, Pending=0, Fired=0, Skipped=2, Incomplete=0)
Feb 20 09:29:54 linux crmd: [3107]: info: do_state_transition: State transition S_TRANSITION_ENGINE -> S_POLICY_ENGINE [ input=I_PE_CALC cause=C_IPC_MESSAGE origin=route_message ]
Feb 20 09:29:54 linux crmd: [3107]: info: do_state_transition: All 2 cluster nodes are eligible to run resources.
Feb 20 09:29:54 linux pengine: [5143]: info: determine_online_status: Node postit.mtu.edu is online
Feb 20 09:29:54 linux pengine: [5143]: info: determine_online_status: Node linux.mtu.edu is online
Feb 20 09:29:54 linux pengine: [5143]: WARN: unpack_rsc_op: Processing failed op Apache_start_0 on linux.mtu.edu: Error
Feb 20 09:29:54 linux pengine: [5143]: WARN: unpack_rsc_op: Compatability handling for failed op Apache_start_0 on linux.mtu.edu
Feb 20 09:29:54 linux pengine: [5143]: notice: group_print: Resource Group: Webserver
Feb 20 09:29:54 linux pengine: [5143]: notice: native_print:     IP	(heartbeat::ocf:IPaddr2):	Started postit.mtu.edu
Feb 20 09:29:54 linux pengine: [5143]: notice: native_print:     Apache	(heartbeat::ocf:apache):	Stopped 
Feb 20 09:29:54 linux pengine: [5143]: notice: NoRoleChange: Leave resource IP	(postit.mtu.edu)
Feb 20 09:29:54 linux pengine: [5143]: notice: StartRsc:  postit.mtu.edu	Start Apache
Feb 20 09:29:54 linux crmd: [3107]: info: do_state_transition: State transition S_POLICY_ENGINE -> S_TRANSITION_ENGINE [ input=I_PE_SUCCESS cause=C_IPC_MESSAGE origin=route_message ]
Feb 20 09:29:54 linux tengine: [5142]: info: unpack_graph: Unpacked transition 3: 4 actions in 4 synapses
Feb 20 09:29:54 linux tengine: [5142]: info: te_pseudo_action: Pseudo action 8 fired and confirmed
Feb 20 09:29:54 linux tengine: [5142]: info: send_rsc_command: Initiating action 6: IP_start_0 on postit.mtu.edu
Feb 20 09:29:54 linux pengine: [5143]: info: process_pe_message: Transition 3: PEngine Input stored in: /var/lib/heartbeat/pengine/pe-input-978.bz2
Feb 20 09:29:55 linux tengine: [5142]: info: match_graph_event: Action IP_start_0 (6) confirmed on postit.mtu.edu (rc=0)
Feb 20 09:29:55 linux tengine: [5142]: info: send_rsc_command: Initiating action 7: Apache_start_0 on postit.mtu.edu
Feb 20 09:29:56 linux tengine: [5142]: WARN: status_from_rc: Action start on postit.mtu.edu failed (target: <null> vs. rc: 6): Error
Feb 20 09:29:56 linux tengine: [5142]: WARN: update_failcount: Updating failcount for Apache on aaed2ff6-07cd-476f-8d98-259570a5480e after failed start: rc=6
Feb 20 09:29:56 linux tengine: [5142]: info: update_abort_priority: Abort priority upgraded to 1
Feb 20 09:29:56 linux tengine: [5142]: info: update_abort_priority: Abort action 0 superceeded by 2
Feb 20 09:29:56 linux tengine: [5142]: info: match_graph_event: Action Apache_start_0 (7) confirmed on postit.mtu.edu (rc=4)
Feb 20 09:29:56 linux tengine: [5142]: info: run_graph: ====================================================
Feb 20 09:29:56 linux tengine: [5142]: notice: run_graph: Transition 3: (Complete=3, Pending=0, Fired=0, Skipped=1, Incomplete=0)
Feb 20 09:29:56 linux crmd: [3107]: info: do_state_transition: State transition S_TRANSITION_ENGINE -> S_POLICY_ENGINE [ input=I_PE_CALC cause=C_IPC_MESSAGE origin=route_message ]
Feb 20 09:29:56 linux crmd: [3107]: info: do_state_transition: All 2 cluster nodes are eligible to run resources.
Feb 20 09:29:56 linux tengine: [5142]: info: extract_event: Aborting on transient_attributes changes for aaed2ff6-07cd-476f-8d98-259570a5480e
Feb 20 09:29:56 linux tengine: [5142]: info: update_abort_priority: Abort priority upgraded to 1000000
Feb 20 09:29:56 linux pengine: [5143]: info: determine_online_status: Node postit.mtu.edu is online
Feb 20 09:29:56 linux pengine: [5143]: WARN: unpack_rsc_op: Processing failed op Apache_start_0 on postit.mtu.edu: Error
Feb 20 09:29:56 linux pengine: [5143]: WARN: unpack_rsc_op: Compatability handling for failed op Apache_start_0 on postit.mtu.edu
Feb 20 09:29:56 linux pengine: [5143]: info: determine_online_status: Node linux.mtu.edu is online
Feb 20 09:29:56 linux pengine: [5143]: WARN: unpack_rsc_op: Processing failed op Apache_start_0 on linux.mtu.edu: Error
Feb 20 09:29:56 linux pengine: [5143]: WARN: unpack_rsc_op: Compatability handling for failed op Apache_start_0 on linux.mtu.edu
Feb 20 09:29:56 linux pengine: [5143]: notice: group_print: Resource Group: Webserver
Feb 20 09:29:56 linux pengine: [5143]: notice: native_print:     IP	(heartbeat::ocf:IPaddr2):	Started postit.mtu.edu
Feb 20 09:29:56 linux pengine: [5143]: notice: native_print:     Apache	(heartbeat::ocf:apache):	Started postit.mtu.edu FAILED
Feb 20 09:29:56 linux pengine: [5143]: WARN: native_color: Resource Apache cannot run anywhere
Feb 20 09:29:56 linux pengine: [5143]: notice: NoRoleChange: Leave resource IP	(postit.mtu.edu)
Feb 20 09:29:56 linux pengine: [5143]: notice: StopRsc:   postit.mtu.edu	Stop Apache
Feb 20 09:29:56 linux pengine: [5143]: WARN: process_pe_message: Transition 4: WARNINGs found during PE processing. PEngine Input stored in: /var/lib/heartbeat/pengine/pe-warn-2450.bz2
Feb 20 09:29:56 linux pengine: [5143]: info: process_pe_message: Configuration WARNINGs found during PE processing.  Please run "crm_verify -L" to identify issues.
Feb 20 09:29:56 linux pengine: [5143]: info: determine_online_status: Node postit.mtu.edu is online
Feb 20 09:29:56 linux pengine: [5143]: WARN: unpack_rsc_op: Processing failed op Apache_start_0 on postit.mtu.edu: Error
Feb 20 09:29:56 linux pengine: [5143]: WARN: unpack_rsc_op: Compatability handling for failed op Apache_start_0 on postit.mtu.edu
Feb 20 09:29:56 linux pengine: [5143]: info: determine_online_status: Node linux.mtu.edu is online
Feb 20 09:29:56 linux pengine: [5143]: WARN: unpack_rsc_op: Processing failed op Apache_start_0 on linux.mtu.edu: Error
Feb 20 09:29:56 linux pengine: [5143]: WARN: unpack_rsc_op: Compatability handling for failed op Apache_start_0 on linux.mtu.edu
Feb 20 09:29:56 linux pengine: [5143]: notice: group_print: Resource Group: Webserver
Feb 20 09:29:56 linux pengine: [5143]: notice: native_print:     IP	(heartbeat::ocf:IPaddr2):	Started postit.mtu.edu
Feb 20 09:29:56 linux pengine: [5143]: notice: native_print:     Apache	(heartbeat::ocf:apache):	Started postit.mtu.edu FAILED
Feb 20 09:29:56 linux pengine: [5143]: WARN: native_color: Resource Apache cannot run anywhere
Feb 20 09:29:56 linux pengine: [5143]: notice: NoRoleChange: Leave resource IP	(postit.mtu.edu)
Feb 20 09:29:56 linux pengine: [5143]: notice: StopRsc:   postit.mtu.edu	Stop Apache
Feb 20 09:29:56 linux crmd: [3107]: info: do_state_transition: State transition S_POLICY_ENGINE -> S_TRANSITION_ENGINE [ input=I_PE_SUCCESS cause=C_IPC_MESSAGE origin=route_message ]
Feb 20 09:29:56 linux tengine: [5142]: info: unpack_graph: Unpacked transition 5: 4 actions in 4 synapses
Feb 20 09:29:56 linux tengine: [5142]: info: te_pseudo_action: Pseudo action 10 fired and confirmed
Feb 20 09:29:57 linux tengine: [5142]: info: send_rsc_command: Initiating action 1: Apache_stop_0 on postit.mtu.edu
Feb 20 09:29:57 linux pengine: [5143]: WARN: process_pe_message: Transition 5: WARNINGs found during PE processing. PEngine Input stored in: /var/lib/heartbeat/pengine/pe-warn-2451.bz2
Feb 20 09:29:57 linux pengine: [5143]: info: process_pe_message: Configuration WARNINGs found during PE processing.  Please run "crm_verify -L" to identify issues.
Feb 20 09:29:57 linux tengine: [5142]: info: match_graph_event: Action Apache_stop_0 (1) confirmed on postit.mtu.edu (rc=0)
Feb 20 09:29:57 linux tengine: [5142]: info: te_pseudo_action: Pseudo action 11 fired and confirmed
Feb 20 09:29:57 linux tengine: [5142]: info: te_pseudo_action: Pseudo action 5 fired and confirmed
Feb 20 09:29:57 linux tengine: [5142]: info: run_graph: Transition 5: (Complete=4, Pending=0, Fired=0, Skipped=0, Incomplete=0)
Feb 20 09:29:57 linux tengine: [5142]: info: notify_crmd: Transition 5 status: te_complete - <null>
Feb 20 09:29:57 linux crmd: [3107]: info: do_state_transition: State transition S_TRANSITION_ENGINE -> S_IDLE [ input=I_TE_SUCCESS cause=C_IPC_MESSAGE origin=route_message ]
Feb 20 09:37:20 linux crmd: [3107]: info: handle_shutdown_request: Creating shutdown request for postit.mtu.edu
Feb 20 09:37:20 linux crmd: [3107]: info: do_state_transition: State transition S_IDLE -> S_POLICY_ENGINE [ input=I_PE_CALC cause=C_IPC_MESSAGE origin=route_message ]
Feb 20 09:37:20 linux tengine: [5142]: info: extract_event: Aborting on shutdown attribute for aaed2ff6-07cd-476f-8d98-259570a5480e
Feb 20 09:37:20 linux crmd: [3107]: info: do_state_transition: All 2 cluster nodes are eligible to run resources.
Feb 20 09:37:20 linux tengine: [5142]: info: update_abort_priority: Abort priority upgraded to 1000000
Feb 20 09:37:20 linux pengine: [5143]: info: determine_online_status: Node postit.mtu.edu is shutting down
Feb 20 09:37:20 linux pengine: [5143]: WARN: unpack_rsc_op: Processing failed op Apache_start_0 on postit.mtu.edu: Error
Feb 20 09:37:20 linux pengine: [5143]: WARN: unpack_rsc_op: Compatability handling for failed op Apache_start_0 on postit.mtu.edu
Feb 20 09:37:20 linux pengine: [5143]: info: determine_online_status: Node linux.mtu.edu is online
Feb 20 09:37:20 linux pengine: [5143]: WARN: unpack_rsc_op: Processing failed op Apache_start_0 on linux.mtu.edu: Error
Feb 20 09:37:20 linux pengine: [5143]: WARN: unpack_rsc_op: Compatability handling for failed op Apache_start_0 on linux.mtu.edu
Feb 20 09:37:20 linux pengine: [5143]: notice: group_print: Resource Group: Webserver
Feb 20 09:37:20 linux pengine: [5143]: notice: native_print:     IP	(heartbeat::ocf:IPaddr2):	Started postit.mtu.edu
Feb 20 09:37:20 linux pengine: [5143]: notice: native_print:     Apache	(heartbeat::ocf:apache):	Stopped 
Feb 20 09:37:20 linux pengine: [5143]: WARN: native_color: Resource Apache cannot run anywhere
Feb 20 09:37:20 linux pengine: [5143]: notice: NoRoleChange: Move  resource IP	(postit.mtu.edu -> linux.mtu.edu)
Feb 20 09:37:20 linux pengine: [5143]: notice: StopRsc:   postit.mtu.edu	Stop IP
Feb 20 09:37:20 linux pengine: [5143]: notice: StartRsc:  linux.mtu.edu	Start IP
Feb 20 09:37:20 linux pengine: [5143]: info: stage6: Scheduling Node postit.mtu.edu for shutdown
Feb 20 09:37:20 linux crmd: [3107]: info: do_state_transition: State transition S_POLICY_ENGINE -> S_TRANSITION_ENGINE [ input=I_PE_SUCCESS cause=C_IPC_MESSAGE origin=route_message ]
Feb 20 09:37:20 linux tengine: [5142]: info: unpack_graph: Unpacked transition 6: 8 actions in 8 synapses
Feb 20 09:37:20 linux tengine: [5142]: info: te_pseudo_action: Pseudo action 9 fired and confirmed
Feb 20 09:37:20 linux tengine: [5142]: info: te_pseudo_action: Pseudo action 13 fired and confirmed
Feb 20 09:37:20 linux tengine: [5142]: info: send_rsc_command: Initiating action 5: IP_stop_0 on postit.mtu.edu
Feb 20 09:37:20 linux pengine: [5143]: WARN: process_pe_message: Transition 6: WARNINGs found during PE processing. PEngine Input stored in: /var/lib/heartbeat/pengine/pe-warn-2452.bz2
Feb 20 09:37:20 linux pengine: [5143]: info: process_pe_message: Configuration WARNINGs found during PE processing.  Please run "crm_verify -L" to identify issues.
Feb 20 09:37:21 linux tengine: [5142]: info: match_graph_event: Action IP_stop_0 (5) confirmed on postit.mtu.edu (rc=0)
Feb 20 09:37:21 linux tengine: [5142]: info: te_pseudo_action: Pseudo action 10 fired and confirmed
Feb 20 09:37:21 linux tengine: [5142]: info: te_pseudo_action: Pseudo action 4 fired and confirmed
Feb 20 09:37:21 linux tengine: [5142]: info: te_crm_command: Executing crm-event (12): do_shutdown on postit.mtu.edu
Feb 20 09:37:21 linux tengine: [5142]: info: te_pseudo_action: Pseudo action 7 fired and confirmed
Feb 20 09:37:21 linux tengine: [5142]: info: send_rsc_command: Initiating action 6: IP_start_0 on linux.mtu.edu
Feb 20 09:37:21 linux crmd: [3107]: info: do_lrm_rsc_op: Performing op=IP_start_0 key=6:6:4746c560-21ca-4bf3-93a4-2a2f7919abc9)
Feb 20 09:37:21 linux lrmd: [3104]: info: rsc:IP: start
Feb 20 09:37:21 linux IPaddr2[5177]: [5206]: INFO: ip -f inet addr add 192.168.1.30/24 brd 192.168.1.255 dev eth0
Feb 20 09:37:21 linux IPaddr2[5177]: [5208]: INFO: ip link set eth0 up
Feb 20 09:37:21 linux IPaddr2[5177]: [5210]: INFO: /usr/lib/heartbeat/send_arp -i 200 -r 5 -p /var/run/heartbeat/rsctmp/send_arp/send_arp-192.168.1.30 eth0 192.168.1.30 auto not_used not_used
Feb 20 09:37:21 linux crmd: [3107]: info: process_lrm_event: LRM operation IP_start_0 (call=16, rc=0) complete 
Feb 20 09:37:22 linux tengine: [5142]: info: match_graph_event: Action IP_start_0 (6) confirmed on linux.mtu.edu (rc=0)
Feb 20 09:37:22 linux crmd: [3107]: notice: crmd_client_status_callback: Status update: Client postit.mtu.edu/crmd now has status [offline]
Feb 20 09:37:22 linux cib: [3103]: info: cib_process_shutdown_req: Shutdown REQ from postit.mtu.edu
Feb 20 09:37:23 linux cib: [3103]: info: cib_client_status_callback: Status update: Client postit.mtu.edu/cib now has status [leave]
Feb 20 09:37:23 linux crmd: [3107]: info: mem_handle_event: Got an event OC_EV_MS_INVALID from ccm
Feb 20 09:37:23 linux cib: [3103]: info: mem_handle_event: Got an event OC_EV_MS_INVALID from ccm
Feb 20 09:37:23 linux crmd: [3107]: info: mem_handle_event: no mbr_track info
Feb 20 09:37:23 linux cib: [3103]: info: mem_handle_event: no mbr_track info
Feb 20 09:37:23 linux crmd: [3107]: info: mem_handle_event: Got an event OC_EV_MS_NEW_MEMBERSHIP from ccm
Feb 20 09:37:23 linux cib: [3103]: info: mem_handle_event: Got an event OC_EV_MS_NEW_MEMBERSHIP from ccm
Feb 20 09:37:23 linux crmd: [3107]: info: mem_handle_event: instance=10, nodes=1, new=0, lost=1, n_idx=0, new_idx=1, old_idx=3
Feb 20 09:37:23 linux cib: [3103]: info: mem_handle_event: instance=10, nodes=1, new=0, lost=1, n_idx=0, new_idx=1, old_idx=3
Feb 20 09:37:23 linux crmd: [3107]: info: crmd_ccm_msg_callback: Quorum (re)attained after event=NEW MEMBERSHIP (id=10)
Feb 20 09:37:23 linux cib: [3103]: info: cib_ccm_msg_callback: LOST: postit.mtu.edu
Feb 20 09:37:23 linux ccm: [3102]: info: Break tie for 2 nodes cluster
Feb 20 09:37:23 linux crmd: [3107]: info: ccm_event_detail: NEW MEMBERSHIP: trans=10, nodes=1, new=0, lost=1 n_idx=0, new_idx=1, old_idx=3
Feb 20 09:37:23 linux cib: [3103]: info: cib_ccm_msg_callback: PEER: linux.mtu.edu
Feb 20 09:37:23 linux crmd: [3107]: info: ccm_event_detail: 	CURRENT: linux.mtu.edu [nodeid=0, born=10]
Feb 20 09:37:23 linux crmd: [3107]: info: ccm_event_detail: 	LOST:    postit.mtu.edu [nodeid=1, born=9]
Feb 20 09:37:23 linux tengine: [5142]: info: run_graph: Transition 6: (Complete=8, Pending=0, Fired=0, Skipped=0, Incomplete=0)
Feb 20 09:37:23 linux tengine: [5142]: info: notify_crmd: Transition 6 status: te_complete - <null>
Feb 20 09:37:23 linux crmd: [3107]: info: do_state_transition: State transition S_TRANSITION_ENGINE -> S_IDLE [ input=I_TE_SUCCESS cause=C_IPC_MESSAGE origin=route_message ]
Feb 20 09:37:33 linux heartbeat: [3094]: WARN: node postit.mtu.edu: is dead
Feb 20 09:37:33 linux heartbeat: [3094]: info: Link postit.mtu.edu:eth1 dead.
Feb 20 09:37:33 linux crmd: [3107]: notice: crmd_ha_status_callback: Status update: Node postit.mtu.edu now has status [dead]
Feb 20 09:37:39 linux cib: [3103]: info: cib_stats: Processed 53 operations (25094.00us average, 0% utilization) in the last 10min
Feb 20 09:37:44 linux heartbeat: [3094]: info: Heartbeat restart on node postit.mtu.edu
Feb 20 09:37:44 linux heartbeat: [3094]: info: Link postit.mtu.edu:eth1 up.
Feb 20 09:37:44 linux heartbeat: [3094]: info: Status update for node postit.mtu.edu: status init
Feb 20 09:37:44 linux heartbeat: [3094]: info: Status update for node postit.mtu.edu: status up
Feb 20 09:37:44 linux crmd: [3107]: notice: crmd_ha_status_callback: Status update: Node postit.mtu.edu now has status [init]
Feb 20 09:37:44 linux crmd: [3107]: notice: crmd_ha_status_callback: Status update: Node postit.mtu.edu now has status [up]
Feb 20 09:37:44 linux heartbeat: [3094]: info: Status update for node postit.mtu.edu: status active
Feb 20 09:37:44 linux crmd: [3107]: notice: crmd_ha_status_callback: Status update: Node postit.mtu.edu now has status [active]
Feb 20 09:37:44 linux cib: [3103]: info: cib_client_status_callback: Status update: Client postit.mtu.edu/cib now has status [join]
Feb 20 09:37:47 linux heartbeat: [3094]: WARN: 1 lost packet(s) for [postit.mtu.edu] [17:19]
Feb 20 09:37:47 linux heartbeat: [3094]: info: No pkts missing from postit.mtu.edu!
Feb 20 09:37:48 linux cib: [3103]: info: mem_handle_event: Got an event OC_EV_MS_INVALID from ccm
Feb 20 09:37:48 linux cib: [3103]: info: mem_handle_event: no mbr_track info
Feb 20 09:37:48 linux cib: [3103]: info: mem_handle_event: Got an event OC_EV_MS_NEW_MEMBERSHIP from ccm
Feb 20 09:37:48 linux cib: [3103]: info: mem_handle_event: instance=11, nodes=2, new=1, lost=0, n_idx=0, new_idx=2, old_idx=4
Feb 20 09:37:48 linux cib: [3103]: info: cib_ccm_msg_callback: PEER: linux.mtu.edu
Feb 20 09:37:48 linux crmd: [3107]: notice: crmd_client_status_callback: Status update: Client postit.mtu.edu/crmd now has status [online]
Feb 20 09:37:48 linux cib: [3103]: info: cib_ccm_msg_callback: PEER: postit.mtu.edu
Feb 20 09:37:48 linux crmd: [3107]: info: mem_handle_event: Got an event OC_EV_MS_INVALID from ccm
Feb 20 09:37:48 linux crmd: [3107]: info: mem_handle_event: no mbr_track info
Feb 20 09:37:48 linux crmd: [3107]: info: mem_handle_event: Got an event OC_EV_MS_NEW_MEMBERSHIP from ccm
Feb 20 09:37:48 linux crmd: [3107]: info: mem_handle_event: instance=11, nodes=2, new=1, lost=0, n_idx=0, new_idx=2, old_idx=4
Feb 20 09:37:48 linux crmd: [3107]: info: crmd_ccm_msg_callback: Quorum (re)attained after event=NEW MEMBERSHIP (id=11)
Feb 20 09:37:48 linux crmd: [3107]: info: ccm_event_detail: NEW MEMBERSHIP: trans=11, nodes=2, new=1, lost=0 n_idx=0, new_idx=2, old_idx=4
Feb 20 09:37:48 linux crmd: [3107]: info: ccm_event_detail: 	CURRENT: linux.mtu.edu [nodeid=0, born=7]
Feb 20 09:37:48 linux crmd: [3107]: info: ccm_event_detail: 	CURRENT: postit.mtu.edu [nodeid=1, born=11]
Feb 20 09:37:48 linux crmd: [3107]: info: ccm_event_detail: 	NEW:     postit.mtu.edu [nodeid=1, born=11]
Feb 20 09:37:48 linux tengine: [5142]: info: te_update_diff: Aborting on transient_attributes deletions
Feb 20 09:37:48 linux crmd: [3107]: info: do_state_transition: State transition S_IDLE -> S_POLICY_ENGINE [ input=I_PE_CALC cause=C_IPC_MESSAGE origin=route_message ]
Feb 20 09:37:48 linux tengine: [5142]: info: update_abort_priority: Abort priority upgraded to 1000000
Feb 20 09:37:48 linux crmd: [3107]: info: do_state_transition: Membership changed: 9 -> 11 - join restart
Feb 20 09:37:48 linux crmd: [3107]: info: do_state_transition: State transition S_POLICY_ENGINE -> S_INTEGRATION [ input=I_NODE_JOIN cause=C_FSA_INTERNAL origin=do_state_transition ]
Feb 20 09:37:48 linux crmd: [3107]: info: update_dc: Unset DC linux.mtu.edu
Feb 20 09:37:48 linux crmd: [3107]: info: join_make_offer: Making join offers based on membership 11
Feb 20 09:37:48 linux crmd: [3107]: info: do_dc_join_offer_all: join-4: Waiting on 2 outstanding join acks
Feb 20 09:37:48 linux crmd: [3107]: info: update_dc: Set DC to linux.mtu.edu (2.0)
Feb 20 09:37:49 linux heartbeat: [3094]: WARN: 1 lost packet(s) for [postit.mtu.edu] [22:24]
Feb 20 09:37:49 linux heartbeat: [3094]: info: No pkts missing from postit.mtu.edu!
Feb 20 09:37:52 linux crmd: [3107]: info: update_dc: Unset DC linux.mtu.edu
Feb 20 09:37:52 linux crmd: [3107]: info: do_dc_join_offer_all: join-5: Waiting on 2 outstanding join acks
Feb 20 09:37:52 linux crmd: [3107]: info: update_dc: Set DC to linux.mtu.edu (2.0)
Feb 20 09:37:53 linux attrd: [3106]: info: attrd_local_callback: Sending full refresh
Feb 20 09:37:53 linux crmd: [3107]: info: do_state_transition: State transition S_INTEGRATION -> S_FINALIZE_JOIN [ input=I_INTEGRATED cause=C_FSA_INTERNAL origin=check_join_state ]
Feb 20 09:37:53 linux cib: [3103]: info: sync_our_cib: Syncing CIB to all peers
Feb 20 09:37:53 linux crmd: [3107]: info: do_state_transition: All 2 cluster nodes responded to the join offer.
Feb 20 09:37:53 linux crmd: [3107]: info: update_dc: Set DC to linux.mtu.edu (2.0)
Feb 20 09:37:54 linux crmd: [3107]: info: do_dc_join_ack: join-5: Updating node state to member for linux.mtu.edu
Feb 20 09:37:54 linux crmd: [3107]: info: do_dc_join_ack: join-5: Updating node state to member for postit.mtu.edu
Feb 20 09:37:54 linux crmd: [3107]: info: do_state_transition: State transition S_FINALIZE_JOIN -> S_POLICY_ENGINE [ input=I_FINALIZED cause=C_FSA_INTERNAL origin=check_join_state ]
Feb 20 09:37:54 linux crmd: [3107]: info: do_state_transition: All 2 cluster nodes are eligible to run resources.
Feb 20 09:37:54 linux pengine: [5143]: info: determine_online_status: Node postit.mtu.edu is online
Feb 20 09:37:54 linux pengine: [5143]: info: determine_online_status: Node linux.mtu.edu is online
Feb 20 09:37:54 linux pengine: [5143]: WARN: unpack_rsc_op: Processing failed op Apache_start_0 on linux.mtu.edu: Error
Feb 20 09:37:54 linux pengine: [5143]: WARN: unpack_rsc_op: Compatability handling for failed op Apache_start_0 on linux.mtu.edu
Feb 20 09:37:54 linux pengine: [5143]: notice: group_print: Resource Group: Webserver
Feb 20 09:37:54 linux pengine: [5143]: notice: native_print:     IP	(heartbeat::ocf:IPaddr2):	Started linux.mtu.edu
Feb 20 09:37:54 linux pengine: [5143]: notice: native_print:     Apache	(heartbeat::ocf:apache):	Stopped 
Feb 20 09:37:54 linux pengine: [5143]: notice: NoRoleChange: Move  resource IP	(linux.mtu.edu -> postit.mtu.edu)
Feb 20 09:37:54 linux pengine: [5143]: notice: StopRsc:   linux.mtu.edu	Stop IP
Feb 20 09:37:54 linux pengine: [5143]: notice: StartRsc:  postit.mtu.edu	Start IP
Feb 20 09:37:54 linux pengine: [5143]: notice: StartRsc:  postit.mtu.edu	Start Apache
Feb 20 09:37:55 linux crmd: [3107]: info: do_state_transition: State transition S_POLICY_ENGINE -> S_TRANSITION_ENGINE [ input=I_PE_SUCCESS cause=C_IPC_MESSAGE origin=route_message ]
Feb 20 09:37:55 linux tengine: [5142]: info: unpack_graph: Unpacked transition 7: 13 actions in 13 synapses
Feb 20 09:37:55 linux pengine: [5143]: info: process_pe_message: Transition 7: PEngine Input stored in: /var/lib/heartbeat/pengine/pe-input-979.bz2
Feb 20 09:37:55 linux tengine: [5142]: info: te_pseudo_action: Pseudo action 12 fired and confirmed
Feb 20 09:37:55 linux tengine: [5142]: info: send_rsc_command: Initiating action 3: IP_monitor_0 on postit.mtu.edu
Feb 20 09:37:55 linux tengine: [5142]: info: send_rsc_command: Initiating action 4: Apache_monitor_0 on postit.mtu.edu
Feb 20 09:37:55 linux tengine: [5142]: info: te_pseudo_action: Pseudo action 15 fired and confirmed
Feb 20 09:37:55 linux tengine: [5142]: info: send_rsc_command: Initiating action 7: IP_stop_0 on linux.mtu.edu
Feb 20 09:37:55 linux crmd: [3107]: info: do_lrm_rsc_op: Performing op=IP_stop_0 key=7:7:4746c560-21ca-4bf3-93a4-2a2f7919abc9)
Feb 20 09:37:55 linux lrmd: [3104]: info: rsc:IP: stop
Feb 20 09:37:55 linux IPaddr2[5214]: [5243]: INFO: ip -f inet addr delete 192.168.1.30/24 dev eth0
Feb 20 09:37:55 linux IPaddr2[5214]: [5245]: INFO: ip -o -f inet addr show eth0
Feb 20 09:37:55 linux crmd: [3107]: info: process_lrm_event: LRM operation IP_stop_0 (call=17, rc=0) complete 
Feb 20 09:37:55 linux tengine: [5142]: info: match_graph_event: Action IP_stop_0 (7) confirmed on linux.mtu.edu (rc=0)
Feb 20 09:37:55 linux tengine: [5142]: info: te_pseudo_action: Pseudo action 13 fired and confirmed
Feb 20 09:37:55 linux tengine: [5142]: info: te_pseudo_action: Pseudo action 6 fired and confirmed
Feb 20 09:37:56 linux tengine: [5142]: info: match_graph_event: Action Apache_monitor_0 (4) confirmed on postit.mtu.edu (rc=0)
Feb 20 09:37:56 linux tengine: [5142]: info: match_graph_event: Action IP_monitor_0 (3) confirmed on postit.mtu.edu (rc=0)
Feb 20 09:37:56 linux tengine: [5142]: info: send_rsc_command: Initiating action 2: probe_complete on postit.mtu.edu
Feb 20 09:37:56 linux tengine: [5142]: info: te_pseudo_action: Pseudo action 1 fired and confirmed
Feb 20 09:37:56 linux tengine: [5142]: info: te_pseudo_action: Pseudo action 10 fired and confirmed
Feb 20 09:37:56 linux tengine: [5142]: info: send_rsc_command: Initiating action 8: IP_start_0 on postit.mtu.edu
Feb 20 09:37:57 linux tengine: [5142]: info: extract_event: Aborting on transient_attributes changes for aaed2ff6-07cd-476f-8d98-259570a5480e
Feb 20 09:37:57 linux tengine: [5142]: info: update_abort_priority: Abort priority upgraded to 1000000
Feb 20 09:37:57 linux tengine: [5142]: info: update_abort_priority: Abort action 0 superceeded by 2
Feb 20 09:37:57 linux crmd: [3107]: info: do_state_transition: State transition S_TRANSITION_ENGINE -> S_POLICY_ENGINE [ input=I_PE_CALC cause=C_IPC_MESSAGE origin=route_message ]
Feb 20 09:37:57 linux tengine: [5142]: info: match_graph_event: Action IP_start_0 (8) confirmed on postit.mtu.edu (rc=0)
Feb 20 09:37:57 linux crmd: [3107]: info: do_state_transition: All 2 cluster nodes are eligible to run resources.
Feb 20 09:37:57 linux tengine: [5142]: info: run_graph: ====================================================
Feb 20 09:37:57 linux tengine: [5142]: notice: run_graph: Transition 7: (Complete=11, Pending=0, Fired=0, Skipped=2, Incomplete=0)
Feb 20 09:37:57 linux pengine: [5143]: info: determine_online_status: Node postit.mtu.edu is online
Feb 20 09:37:57 linux crmd: [3107]: info: do_state_transition: State transition S_POLICY_ENGINE -> S_TRANSITION_ENGINE [ input=I_PE_SUCCESS cause=C_IPC_MESSAGE origin=route_message ]
Feb 20 09:37:57 linux pengine: [5143]: info: determine_online_status: Node linux.mtu.edu is online
Feb 20 09:37:57 linux pengine: [5143]: WARN: unpack_rsc_op: Processing failed op Apache_start_0 on linux.mtu.edu: Error
Feb 20 09:37:57 linux pengine: [5143]: WARN: unpack_rsc_op: Compatability handling for failed op Apache_start_0 on linux.mtu.edu
Feb 20 09:37:57 linux pengine: [5143]: notice: group_print: Resource Group: Webserver
Feb 20 09:37:57 linux pengine: [5143]: notice: native_print:     IP	(heartbeat::ocf:IPaddr2):	Started postit.mtu.edu
Feb 20 09:37:57 linux pengine: [5143]: notice: native_print:     Apache	(heartbeat::ocf:apache):	Stopped 
Feb 20 09:37:57 linux pengine: [5143]: notice: NoRoleChange: Leave resource IP	(postit.mtu.edu)
Feb 20 09:37:57 linux pengine: [5143]: notice: StartRsc:  postit.mtu.edu	Start Apache
Feb 20 09:37:57 linux tengine: [5142]: info: unpack_graph: Unpacked transition 8: 4 actions in 4 synapses
Feb 20 09:37:57 linux pengine: [5143]: info: process_pe_message: Transition 8: PEngine Input stored in: /var/lib/heartbeat/pengine/pe-input-980.bz2
Feb 20 09:37:57 linux tengine: [5142]: info: te_pseudo_action: Pseudo action 8 fired and confirmed
Feb 20 09:37:57 linux tengine: [5142]: info: send_rsc_command: Initiating action 6: IP_start_0 on postit.mtu.edu
Feb 20 09:37:58 linux tengine: [5142]: info: match_graph_event: Action IP_start_0 (6) confirmed on postit.mtu.edu (rc=0)
Feb 20 09:37:58 linux tengine: [5142]: info: send_rsc_command: Initiating action 7: Apache_start_0 on postit.mtu.edu
Feb 20 09:37:59 linux tengine: [5142]: WARN: status_from_rc: Action start on postit.mtu.edu failed (target: <null> vs. rc: 6): Error
Feb 20 09:37:59 linux tengine: [5142]: WARN: update_failcount: Updating failcount for Apache on aaed2ff6-07cd-476f-8d98-259570a5480e after failed start: rc=6
Feb 20 09:37:59 linux tengine: [5142]: info: update_abort_priority: Abort priority upgraded to 1
Feb 20 09:37:59 linux crmd: [3107]: info: do_state_transition: State transition S_TRANSITION_ENGINE -> S_POLICY_ENGINE [ input=I_PE_CALC cause=C_IPC_MESSAGE origin=route_message ]
Feb 20 09:37:59 linux tengine: [5142]: info: update_abort_priority: Abort action 0 superceeded by 2
Feb 20 09:37:59 linux crmd: [3107]: info: do_state_transition: All 2 cluster nodes are eligible to run resources.
Feb 20 09:37:59 linux tengine: [5142]: info: match_graph_event: Action Apache_start_0 (7) confirmed on postit.mtu.edu (rc=4)
Feb 20 09:37:59 linux tengine: [5142]: info: run_graph: ====================================================
Feb 20 09:37:59 linux tengine: [5142]: notice: run_graph: Transition 8: (Complete=3, Pending=0, Fired=0, Skipped=1, Incomplete=0)
Feb 20 09:37:59 linux pengine: [5143]: info: determine_online_status: Node postit.mtu.edu is online
Feb 20 09:37:59 linux pengine: [5143]: WARN: unpack_rsc_op: Processing failed op Apache_start_0 on postit.mtu.edu: Error
Feb 20 09:37:59 linux pengine: [5143]: WARN: unpack_rsc_op: Compatability handling for failed op Apache_start_0 on postit.mtu.edu
Feb 20 09:37:59 linux pengine: [5143]: info: determine_online_status: Node linux.mtu.edu is online
Feb 20 09:37:59 linux pengine: [5143]: WARN: unpack_rsc_op: Processing failed op Apache_start_0 on linux.mtu.edu: Error
Feb 20 09:37:59 linux pengine: [5143]: WARN: unpack_rsc_op: Compatability handling for failed op Apache_start_0 on linux.mtu.edu
Feb 20 09:38:00 linux pengine: [5143]: notice: group_print: Resource Group: Webserver
Feb 20 09:38:00 linux pengine: [5143]: notice: native_print:     IP	(heartbeat::ocf:IPaddr2):	Started postit.mtu.edu
Feb 20 09:38:00 linux pengine: [5143]: notice: native_print:     Apache	(heartbeat::ocf:apache):	Started postit.mtu.edu FAILED
Feb 20 09:38:00 linux crmd: [3107]: info: do_state_transition: State transition S_POLICY_ENGINE -> S_TRANSITION_ENGINE [ input=I_PE_SUCCESS cause=C_IPC_MESSAGE origin=route_message ]
Feb 20 09:38:00 linux tengine: [5142]: info: unpack_graph: Unpacked transition 9: 4 actions in 4 synapses
Feb 20 09:38:00 linux pengine: [5143]: WARN: native_color: Resource Apache cannot run anywhere
Feb 20 09:38:00 linux tengine: [5142]: info: te_pseudo_action: Pseudo action 10 fired and confirmed
Feb 20 09:38:00 linux pengine: [5143]: notice: NoRoleChange: Leave resource IP	(postit.mtu.edu)
Feb 20 09:38:00 linux tengine: [5142]: info: send_rsc_command: Initiating action 1: Apache_stop_0 on postit.mtu.edu
Feb 20 09:38:00 linux pengine: [5143]: notice: StopRsc:   postit.mtu.edu	Stop Apache
Feb 20 09:38:00 linux pengine: [5143]: WARN: process_pe_message: Transition 9: WARNINGs found during PE processing. PEngine Input stored in: /var/lib/heartbeat/pengine/pe-warn-2453.bz2
Feb 20 09:38:00 linux pengine: [5143]: info: process_pe_message: Configuration WARNINGs found during PE processing.  Please run "crm_verify -L" to identify issues.
Feb 20 09:38:00 linux tengine: [5142]: info: extract_event: Aborting on transient_attributes changes for aaed2ff6-07cd-476f-8d98-259570a5480e
Feb 20 09:38:00 linux tengine: [5142]: info: update_abort_priority: Abort priority upgraded to 1000000
Feb 20 09:38:00 linux tengine: [5142]: info: update_abort_priority: Abort action 0 superceeded by 2
Feb 20 09:38:00 linux tengine: [5142]: info: match_graph_event: Action Apache_stop_0 (1) confirmed on postit.mtu.edu (rc=0)
Feb 20 09:38:00 linux tengine: [5142]: info: run_graph: ====================================================
Feb 20 09:38:00 linux crmd: [3107]: info: do_state_transition: State transition S_TRANSITION_ENGINE -> S_POLICY_ENGINE [ input=I_PE_CALC cause=C_IPC_MESSAGE origin=route_message ]
Feb 20 09:38:00 linux tengine: [5142]: notice: run_graph: Transition 9: (Complete=2, Pending=0, Fired=0, Skipped=2, Incomplete=0)
Feb 20 09:38:00 linux crmd: [3107]: info: do_state_transition: All 2 cluster nodes are eligible to run resources.
Feb 20 09:38:00 linux pengine: [5143]: info: determine_online_status: Node postit.mtu.edu is online
Feb 20 09:38:00 linux pengine: [5143]: WARN: unpack_rsc_op: Processing failed op Apache_start_0 on postit.mtu.edu: Error
Feb 20 09:38:00 linux pengine: [5143]: WARN: unpack_rsc_op: Compatability handling for failed op Apache_start_0 on postit.mtu.edu
Feb 20 09:38:00 linux pengine: [5143]: info: determine_online_status: Node linux.mtu.edu is online
Feb 20 09:38:00 linux pengine: [5143]: WARN: unpack_rsc_op: Processing failed op Apache_start_0 on linux.mtu.edu: Error
Feb 20 09:38:00 linux pengine: [5143]: WARN: unpack_rsc_op: Compatability handling for failed op Apache_start_0 on linux.mtu.edu
Feb 20 09:38:01 linux crmd: [3107]: info: do_state_transition: State transition S_POLICY_ENGINE -> S_TRANSITION_ENGINE [ input=I_PE_SUCCESS cause=C_IPC_MESSAGE origin=route_message ]
Feb 20 09:38:01 linux tengine: [5142]: info: unpack_graph: Unpacked transition 10: 0 actions in 0 synapses
Feb 20 09:38:01 linux pengine: [5143]: notice: group_print: Resource Group: Webserver
Feb 20 09:38:01 linux crmd: [3107]: info: do_state_transition: State transition S_TRANSITION_ENGINE -> S_IDLE [ input=I_TE_SUCCESS cause=C_IPC_MESSAGE origin=route_message ]
Feb 20 09:38:01 linux tengine: [5142]: info: run_graph: Transition 10: (Complete=0, Pending=0, Fired=0, Skipped=0, Incomplete=0)
Feb 20 09:38:01 linux pengine: [5143]: notice: native_print:     IP	(heartbeat::ocf:IPaddr2):	Started postit.mtu.edu
Feb 20 09:38:01 linux tengine: [5142]: info: notify_crmd: Transition 10 status: te_complete - <null>
Feb 20 09:38:01 linux pengine: [5143]: notice: native_print:     Apache	(heartbeat::ocf:apache):	Stopped 
Feb 20 09:38:01 linux pengine: [5143]: WARN: native_color: Resource Apache cannot run anywhere
Feb 20 09:38:01 linux pengine: [5143]: notice: NoRoleChange: Leave resource IP	(postit.mtu.edu)
Feb 20 09:38:01 linux pengine: [5143]: WARN: process_pe_message: Transition 10: WARNINGs found during PE processing. PEngine Input stored in: /var/lib/heartbeat/pengine/pe-warn-2454.bz2
Feb 20 09:38:01 linux pengine: [5143]: info: process_pe_message: Configuration WARNINGs found during PE processing.  Please run "crm_verify -L" to identify issues.
Feb 20 09:42:55 linux sshd(pam_unix)[5251]: session opened for user root by (uid=0)
Feb 20 09:43:02 linux sshd(pam_unix)[5270]: session opened for user root by (uid=0)
Feb 20 09:43:02 linux logger: Mark:HB_REPORT:1203518646
Feb 20 09:43:07 linux crmd: [3107]: info: handle_request: Current ping state: S_IDLE
Feb 20 09:43:08 linux crmd: [3107]: info: handle_request: Current ping state: S_IDLE
Feb 20 09:43:16 linux sshd(pam_unix)[5304]: session opened for user root by (uid=0)
