I found out why it didn't work, fail-count for resource httpd was at
INFINITY:
# crm_simulate -sL
[...]
<nvpair id="status-2-fail-count-httpd" name="fail-count-httpd"
value="INFINITY"/>
[...]
To reset the fail-count:
# pcs resource failcount reset httpd
Now it works as expected :)
Next step: configure STONISH !
Thanks again !
Kevin
On 06/15/2016 02:18 PM, Kevin THIERRY wrote:
Hello !
I wasn't expecting such great explanations, thanks a lot Ken ! Also
thank you for your example Dimitri ! It solved the issue I had !
I'm still having trouble though: when I make the primary node fail (I
unplug it), the secondary node starts everything as expected but the
httpd service. However when I plug back the primary node, all
resources go back to it and everything works fine, even the httpd
service.
Starting the httpd server manually on the second node works fine
(using systemctl). I suspect an issue with the allocation scores but I
don't know how to solve it.
################################################
# grep httpd /var/log/cluster/corosync.log
################################################
Jun 15 11:58:27 [27192] laa-billing-backup pengine: warning:
unpack_rsc_op_failure: Processing failed op start for httpd on
billing-backup-sync: unknown error (1)
Jun 15 11:58:27 [27192] laa-billing-backup pengine: info:
native_print: httpd (ocf::heartbeat:apache): Started
billing-primary-sync
Jun 15 11:58:27 [27192] laa-billing-backup pengine: info:
get_failcount_full: httpd has failed INFINITY times on
billing-backup-sync
Jun 15 11:58:27 [27192] laa-billing-backup pengine: warning:
common_apply_stickiness: Forcing httpd away from
billing-backup-sync after 1000000 failures (max=1000000)
Jun 15 11:58:27 [27192] laa-billing-backup pengine: info:
rsc_merge_weights: vip: Rolling back scores from httpd
Jun 15 11:58:27 [27192] laa-billing-backup pengine: info:
rsc_merge_weights: drbd:1: Rolling back scores from httpd
Jun 15 11:58:27 [27192] laa-billing-backup pengine: info:
rsc_merge_weights: drbd-master: Rolling back scores from httpd
Jun 15 11:58:27 [27192] laa-billing-backup pengine: info:
rsc_merge_weights: fs: Rolling back scores from httpd
Jun 15 11:58:27 [27192] laa-billing-backup pengine: info:
rsc_merge_weights: pgsql: Rolling back scores from httpd
Jun 15 11:58:27 [27192] laa-billing-backup pengine: info:
native_color: Resource http cannot run anywhere
Jun 15 11:58:27 [27192] laa-billing-backup pengine: notice:
LogActions: Stop httpd(billing-primary-sync)
Jun 15 11:58:27 [27193] laa-billing-backup crmd: notice:
te_rsc_command: Initiating action 54: stop httpd_stop_0 on
billing-primary-sync
Jun 15 11:58:29 [27188] laa-billing-backup cib: info:
cib_perform_op: +
/cib/status/node_state[@id='1']/lrm[@id='1']/lrm_resources/lrm_resource[@id='httpd']/lrm_rsc_op[@id='httpd_last_0']:
@operation_key=httpd_stop_0, @operation=stop,
@transition-key=54:75:0:4fdfd97f-42dd-4f04-bf86-294d300df414,
@transition-magic=0:0;54:75:0:4fdfd97f-42dd-4f04-bf86-294d300df414,
@call-id=166, @last-run=1465966707, @last-rc-change=1465966707,
@exec-time=2118
Jun 15 11:58:29 [27193] laa-billing-backup crmd: info:
match_graph_event: Action httpd_stop_0 (54) confirmed on
billing-primary-sync (rc=0)
On 06/14/2016 09:30 PM, Ken Gaillot wrote:
################################################
# crm_simulate -sL
################################################
Current cluster status:
Online: [ billing-backup-sync billing-primary-sync ]
vip (ocf::heartbeat:IPaddr2): Started billing-primary-sync
Master/Slave Set: drbd-master [drbd]
Masters: [ billing-primary-sync ]
Slaves: [ billing-backup-sync ]
fs (ocf::heartbeat:Filesystem): Started billing-primary-sync
pgsql (ocf::heartbeat:pgsql): Started billing-primary-sync
httpd (ocf::heartbeat:apache): Started billing-primary-sync
Clone Set: ping-clone [ping]
Started: [ billing-backup-sync billing-primary-sync ]
Allocation scores:
native_color: vip allocation score on billing-backup-sync: -INFINITY
native_color: vip allocation score on billing-primary-sync: 400
clone_color: drbd-master allocation score on billing-backup-sync: 0
clone_color: drbd-master allocation score on billing-primary-sync: 300
clone_color: drbd:0 allocation score on billing-backup-sync: 0
clone_color: drbd:0 allocation score on billing-primary-sync: 10100
clone_color: drbd:1 allocation score on billing-backup-sync: 10100
clone_color: drbd:1 allocation score on billing-primary-sync: 0
native_color: drbd:0 allocation score on billing-backup-sync: 0
native_color: drbd:0 allocation score on billing-primary-sync: 10100
native_color: drbd:1 allocation score on billing-backup-sync: 10100
native_color: drbd:1 allocation score on billing-primary-sync: -INFINITY
drbd:0 promotion score on billing-primary-sync: INFINITY
drbd:1 promotion score on billing-backup-sync: -INFINITY
native_color: fs allocation score on billing-backup-sync: -INFINITY
native_color: fs allocation score on billing-primary-sync: 10400
native_color: pgsql allocation score on billing-backup-sync: -INFINITY
native_color: pgsql allocation score on billing-primary-sync: 200
native_color: httpd allocation score on billing-backup-sync: -INFINITY
native_color: httpd allocation score on billing-primary-sync: 100
clone_color: ping-clone allocation score on billing-backup-sync: 0
clone_color: ping-clone allocation score on billing-primary-sync: 0
clone_color: ping:0 allocation score on billing-backup-sync: 0
clone_color: ping:0 allocation score on billing-primary-sync: 100
clone_color: ping:1 allocation score on billing-backup-sync: 100
clone_color: ping:1 allocation score on billing-primary-sync: 0
native_color: ping:1 allocation score on billing-backup-sync: 100
native_color: ping:1 allocation score on billing-primary-sync: 0
native_color: ping:0 allocation score on billing-backup-sync: -INFINITY
native_color: ping:0 allocation score on billing-primary-sync: 100
Transition Summary:
################################################
# Full config
################################################
# VIP
pcs cluster cib vip_cfg
pcs -f vip_cfg resource create vip ocf:heartbeat:IPaddr2 \
ip=10.5.200.30 cidr_netmask=24 op monitor interval=30s
pcs -f vip_cfg constraint location vip prefers billing-primary=100
pcs cluster cib-push vip_cfg
# DRBD
pcs cluster cib drbd_cfg
pcs -f drbd_cfg resource create drbd ocf:linbit:drbd \
drbd_resource=drbd0 \
op monitor interval=29s role="Master" \
op monitor interval=31s role="Slave"
pcs -f drbd_cfg resource master drbd-master drbd \
master-max=1 master-node-max=1 clone-max=2 clone-node-max=1
notify=true
pcs -f drbd_cfg constraint location drbd-master prefers
billing-primary=100
pcs -f drbd_cfg constraint colocation add master drbd-master with vip
pcs -f drbd_cfg constraint order start vip then promote drbd-master
pcs cluster cib-push drbd_cfg
# FS
pcs cluster cib fs_cfg
pcs -f fs_cfg resource create fs Filesystem \
device="/dev/drbd0" directory="/data" fstype="ext4"
pcs -f fs_cfg constraint location fs prefers billing-primary=100
pcs -f fs_cfg constraint colocation add fs with drbd-master INFINITY
with-rsc-role=Master
pcs -f fs_cfg constraint order promote drbd-master then start fs
pcs cluster cib-push fs_cfg
# PGSQL
pcs cluster cib pgsql_cfg
pcs -f pgsql_cfg resource create pgsql pgsql \
pgctl="/usr/bin/pg_ctl" \
psql="/usr/bin/psql" \
pgdata="/data/pgsql/data/" \
node_list="billing-primary billing-backup" \
restart_on_promote='true'
pcs -f pgsql_cfg constraint location pgsql prefers billing-primary=100
pcs -f pgsql_cfg constraint colocation add pgsql with fs INFINITY
pcs -f pgsql_cfg constraint order start fs then start pgsql
pcs cluster cib-push pgsql_cfg
# HTTPD
pcs cluster cib httpd_cfg
pcs -f httpd_cfg resource create httpd ocf:heartbeat:apache \
configfile=/etc/httpd/conf/httpd.conf \
statusurl="http://localhost/welcome"; \
op monitor interval=1min
pcs -f httpd_cfg constraint location httpd prefers billing-primary=100
pcs -f httpd_cfg constraint colocation add httpd with pgsql INFINITY
pcs -f httpd_cfg constraint order start pgsql then start httpd
pcs cluster cib-push httpd_cfg
# PING
pcs cluster cib ping_cfg
pcs -f ping_cfg resource create ping ocf:pacemaker:ping dampen=5s
multiplier=1000 host_list="10.5.200.254" --clone
pcs -f ping_cfg constraint location vip rule score=-INFINITY pingd lt
1 or not_defined pingd
pcs cluster cib-push ping_cfg
_______________________________________________
Users mailing list: Users@clusterlabs.org
http://clusterlabs.org/mailman/listinfo/users
Project Home: http://www.clusterlabs.org
Getting started: http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf
Bugs: http://bugs.clusterlabs.org
