Il 17/10/2019 18:08, Ken Gaillot ha scritto:
This does sound odd, possibly a bug. Can you provide the output of "pcs
cluster cib" when one of the unexpected results is happening? (Strip
out any passwords or other sensitive information, and you can e-mail it
to me privately if you don't want it on the list.)
There's no problem at all in sharing the informations. I'm working on a
sandbox to test Pacemaker and then use it on a production set to achieve
high availability and fault tolerance, so I'm working on dummy machines
inside our internal network.
(meanwhile I added three nodes more, but the behaviour has not changed)
Thanks
On Thu, 2019-10-17 at 16:55 +0200, Raffaele Pantaleoni wrote:
Il 17/10/2019 16:47, Raffaele Pantaleoni ha scritto:
Il 17/10/2019 14:51, Jan Pokorný ha scritto:
On 17/10/19 08:22 +0200, Raffaele Pantaleoni wrote:
I'm rather new to Pacemaker, I'm performing early tests on a
set of
three virtual machines.
I am configuring the cluster in the following way:
3 nodes configured
4 resources configured
Online: [ SRVDRSW01 SRVDRSW02 SRVDRSW03 ]
ClusterIP (ocf::heartbeat:IPaddr2): Started
SRVDRSW01
CouchIP (ocf::heartbeat:IPaddr2): Started
SRVDRSW03
FrontEnd (ocf::heartbeat:nginx): Started SRVDRSW01
ITATESTSERVER-DIP (ocf::nodejs:pm2): Started
SRVDRSW01
with the following constraints:
Location Constraints:
Resource: ClusterIP
Enabled on: SRVRDSW01 (score:200)
Enabled on: SRVRDSW02 (score:100)
Resource: CouchIP
Enabled on: SRVRDSW02 (score:10)
Enabled on: SRVRDSW03 (score:100)
Disabled on: SRVRDSW01 (score:-INFINITY)
Resource: FrontEnd
Enabled on: SRVRDSW01 (score:200)
Enabled on: SRVRDSW02 (score:100)
Resource: ITATESTSERVER-DIP
Enabled on: SRVRDSW01 (score:200)
Enabled on: SRVRDSW02 (score:100)
Ordering Constraints:
start ClusterIP then start FrontEnd (kind:Mandatory)
start ClusterIP then start ITATESTSERVER-DIP
(kind:Mandatory)
Colocation Constraints:
FrontEnd with ClusterIP (score:INFINITY)
FrontEnd with ITATESTSERVER-DIP (score:INFINITY)
I've configured the cluster with an opt in strategy using the
following
commands:
crm_attribute --name symmectric-cluster --update false
pcs resource create ClusterIP ocf:heartbeat:IPaddr2
ip=172.16.10.126
cidr_netmask=16 op monitor interval=30s
pcs resource create CouchIP ocf:heartbeat:IPaddr2
ip=172.16.10.128
cidr_netmask=16 op monitor interval=30s
pcs resource create FrontEnd ocf:heartbeat:nginx
configfile=/etc/nginx/nginx.conf
pcs resource create ITATESTSERVER-DIP ocf:nodejs:pm2
user=ITATESTSERVER
--force
pcs constraint colocation add FrontEnd with ClusterIP INFINITY
pcs constraint colocation add FrontEnd with ITATESTSERVER-DIP
INFINITY
pcs constraint order ClusterIP then FrontEnd
pcs constraint order ClusterIP then ITATESTSERVER-DIP
pcs constraint location ClusterIP prefers SRVRDSW01=200
pcs constraint location ClusterIP prefers SRVRDSW02=100
pcs constraint location CouchIP prefers SRVRDSW02=10
pcs constraint location CouchIP prefers SRVRDSW03=100
pcs constraint location FrontEnd prefers SRVRDSW01=200
pcs constraint location FrontEnd prefers SRVRDSW02=100
pcs constraint location ITATESTSERVER-DIP prefers SRVRDSW01=200
pcs constraint location ITATESTSERVER-DIP prefers SRVRDSW02=100
Everything seems to be ok but when I put vm 02 and 03 in
standby I'd
expect
CouchIP not be assigned to vm 01 beacuse of the constraint.
The IPaddr2 resource gets assigned to vm 01 no matter what.
Node SRVDRSW02: standby
Node SRVDRSW03: standby
Online: [ SRVDRSW01 ]
Full list of resources:
ClusterIP (ocf::heartbeat:IPaddr2): Started
SRVDRSW01
CouchIP (ocf::heartbeat:IPaddr2): Started
SRVDRSW01
FrontEnd (ocf::heartbeat:nginx): Started SRVDRSW01
ITATESTSERVER-DIP (ocf::nodejs:pm2): Started
SRVDRSW01
crm_simulate -sL returns the follwoing
---cut---
native_color: CouchIP allocation score on SRVDRSW01: 0
native_color: CouchIP allocation score on SRVDRSW02: 0
native_color: CouchIP allocation score on SRVDRSW03: 0
---cut---
Why is that? I have explicitly assigned -INFINITY to CouchIP
resource
related to node SRVDRSW01 (as stated by pcs constraint:
Disabled on:
SRVRDSW01 (score:-INFINITY) ).
What am I missing or doing wrong?
I am not that deep into these relationships, proper design
documentation with guided examples is non-existent[*].
But it occurs to me that the situation might be the inverse of
what's
been confusing for typical opt-out clusters:
https://lists.clusterlabs.org/pipermail/users/2017-April/005463.html
Have you tried avoiding:
Resource: CouchIP
Disabled on: SRVRDSW01 (score:-INFINITY)
Yes, I already tried that, but I did it again nevertheless since I
am a
newbie. I deleted the whole set of resources and commented out the
constraint from the creation script.
The cluster was running, then I put all the nodes in standby and
brought
SRVRDSW01 back. The CouchIP resource has been bound to the
"forbidden"
node.
crm_simulate -sL still gives a score of 0 to the three nodes when
it
should be something like -INFINITY 100 and 200 respectively.
Just to make the whole thing more confusing: I noticed that
SRVRDSW03,
that is (implicitly) not allowed to get the ClusterIP resource is
marked
(correctly) as -INFINITY from crm_simulate. So the opt in
configuration would seem to be correct, but for the CouchIP
resource
that is no special or different from the ClusterIP resource.
I am really disoriented.
Just another bit of information: I put the whole set in stand by
then
brought back SRVRDSW03... surprise surprise the ClusterIP resource
has
been bound to it even if it shouldn't.
What's wrong?
altogether, since when such explicit edge would be missing,
implicit
"cannot" (for opt-in cluster) would apply anyway, and perhaps
even
reliably, then?
[*] as far as I know, except for
https://wiki.clusterlabs.org/w/images/a/ae/Ordering_Explained_-_White.pdf
https://wiki.clusterlabs.org/w/images/8/8a/Colocation_Explained_-_White.pdf
probably not revised for giving a truthful model in all
details
for years,
and not demonstrating the effect of symmetry requested or
avoided
within
the cluster on those, amongst others
(shameless plug: there will be such coverage for upcoming
group
based
access control addition [those facilities haven't been
terminated in
back-end so far] as a first foray in this area -- also the
correct
understanding is rather important here)
_______________________________________________
Manage your subscription:
https://lists.clusterlabs.org/mailman/listinfo/users
ClusterLabs home: https://www.clusterlabs.org/
Thank you for your reply.
--
Raffaele Pantaleoni
<cib crm_feature_set="3.0.11" validate-with="pacemaker-2.6" epoch="14360" num_updates="9" admin_epoch="0" cib-last-written="Fri Oct 18 08:04:15 2019" update-origin="SRVDRSW01" update-client="crm_attribute" update-user="root" have-quorum="1" dc-uuid="739248760">
<configuration>
<crm_config>
<cluster_property_set id="cib-bootstrap-options">
<nvpair id="cib-bootstrap-options-have-watchdog" name="have-watchdog" value="false"/>
<nvpair id="cib-bootstrap-options-dc-version" name="dc-version" value="1.1.16-94ff4df"/>
<nvpair id="cib-bootstrap-options-cluster-infrastructure" name="cluster-infrastructure" value="corosync"/>
<nvpair id="cib-bootstrap-options-cluster-name" name="cluster-name" value="debian"/>
<nvpair id="cib-bootstrap-options-stonith-enabled" name="stonith-enabled" value="false"/>
<nvpair id="cib-bootstrap-options-symmetric-cluster" name="symmetric-cluster" value="true"/>
<nvpair id="cib-bootstrap-options-symmectric-cluster" name="symmectric-cluster" value="true"/>
</cluster_property_set>
</crm_config>
<nodes>
<node id="739248759" uname="SRVDRSW01">
<instance_attributes id="nodes-739248759"/>
</node>
<node id="739248760" uname="SRVDRSW02">
<instance_attributes id="nodes-739248760">
<nvpair id="nodes-739248760-standby" name="standby" value="on"/>
</instance_attributes>
</node>
<node id="739248761" uname="SRVDRSW03">
<instance_attributes id="nodes-739248761">
<nvpair id="nodes-739248761-standby" name="standby" value="on"/>
</instance_attributes>
</node>
<node id="739248771" uname="SRVDRSW04.seko.com">
<instance_attributes id="nodes-739248771">
<nvpair id="nodes-739248771-standby" name="standby" value="on"/>
</instance_attributes>
</node>
<node id="739248772" uname="SRVRDSW05.seko.com">
<instance_attributes id="nodes-739248772">
<nvpair id="nodes-739248772-standby" name="standby" value="on"/>
</instance_attributes>
</node>
<node id="739248773" uname="srvdrsw06.seko.com">
<instance_attributes id="nodes-739248773">
<nvpair id="nodes-739248773-standby" name="standby" value="on"/>
</instance_attributes>
</node>
</nodes>
<resources>
<primitive class="ocf" id="ClusterIP" provider="heartbeat" type="IPaddr2">
<instance_attributes id="ClusterIP-instance_attributes">
<nvpair id="ClusterIP-instance_attributes-ip" name="ip" value="172.16.10.126"/>
<nvpair id="ClusterIP-instance_attributes-cidr_netmask" name="cidr_netmask" value="16"/>
</instance_attributes>
<operations>
<op id="ClusterIP-start-interval-0s" interval="0s" name="start" timeout="20s"/>
<op id="ClusterIP-stop-interval-0s" interval="0s" name="stop" timeout="20s"/>
<op id="ClusterIP-monitor-interval-30s" interval="30s" name="monitor"/>
</operations>
</primitive>
<primitive class="ocf" id="CouchIP" provider="heartbeat" type="IPaddr2">
<instance_attributes id="CouchIP-instance_attributes">
<nvpair id="CouchIP-instance_attributes-ip" name="ip" value="172.16.10.128"/>
<nvpair id="CouchIP-instance_attributes-cidr_netmask" name="cidr_netmask" value="16"/>
</instance_attributes>
<operations>
<op id="CouchIP-start-interval-0s" interval="0s" name="start" timeout="20s"/>
<op id="CouchIP-stop-interval-0s" interval="0s" name="stop" timeout="20s"/>
<op id="CouchIP-monitor-interval-30s" interval="30s" name="monitor"/>
</operations>
</primitive>
<primitive class="ocf" id="FrontEnd" provider="heartbeat" type="nginx">
<instance_attributes id="FrontEnd-instance_attributes">
<nvpair id="FrontEnd-instance_attributes-configfile" name="configfile" value="/etc/nginx/nginx.conf"/>
</instance_attributes>
<operations>
<op id="FrontEnd-start-interval-0s" interval="0s" name="start" timeout="40s"/>
<op id="FrontEnd-stop-interval-0s" interval="0s" name="stop" timeout="60s"/>
<op id="FrontEnd-monitor-interval-10s" interval="10s" name="monitor" timeout="30s"/>
<op id="FrontEnd-monitor-interval-30s" interval="30s" name="monitor" timeout="30s"/>
<op id="FrontEnd-monitor-interval-60s" interval="60s" name="monitor" timeout="45s"/>
<op id="FrontEnd-monitor-interval-60s-1" interval="60s" name="monitor" timeout="60s"/>
</operations>
</primitive>
<primitive class="ocf" id="ITATESTSERVER-DIP" provider="nodejs" type="pm2">
<instance_attributes id="ITATESTSERVER-DIP-instance_attributes">
<nvpair id="ITATESTSERVER-DIP-instance_attributes-user" name="user" value="ITATESTSERVER"/>
</instance_attributes>
<operations>
<op id="ITATESTSERVER-DIP-start-interval-0s" interval="0s" name="start" timeout="20s"/>
<op id="ITATESTSERVER-DIP-stop-interval-0s" interval="0s" name="stop" timeout="20s"/>
<op id="ITATESTSERVER-DIP-monitor-interval-10" interval="10" name="monitor" timeout="20s"/>
</operations>
</primitive>
</resources>
<constraints>
<rsc_colocation id="colocation-FrontEnd-ClusterIP-INFINITY" rsc="FrontEnd" score="INFINITY" with-rsc="ClusterIP"/>
<rsc_colocation id="colocation-FrontEnd-ITATESTSERVER-DIP-INFINITY" rsc="FrontEnd" score="INFINITY" with-rsc="ITATESTSERVER-DIP"/>
<rsc_order first="ClusterIP" first-action="start" id="order-ClusterIP-FrontEnd-mandatory" then="FrontEnd" then-action="start"/>
<rsc_order first="ClusterIP" first-action="start" id="order-ClusterIP-ITATESTSERVER-DIP-mandatory" then="ITATESTSERVER-DIP" then-action="start"/>
<rsc_location id="location-ClusterIP-SRVRDSW01-200" node="SRVRDSW01" rsc="ClusterIP" score="200"/>
<rsc_location id="location-ClusterIP-SRVRDSW02-100" node="SRVRDSW02" rsc="ClusterIP" score="100"/>
<rsc_location id="location-ClusterIP-SRVRDSW03--INFINITY" node="SRVRDSW03" rsc="ClusterIP" score="-INFINITY"/>
<rsc_location id="location-CouchIP-SRVRDSW01--INFINITY" node="SRVRDSW01" rsc="CouchIP" score="-INFINITY"/>
<rsc_location id="location-CouchIP-SRVRDSW02-10" node="SRVRDSW02" rsc="CouchIP" score="10"/>
<rsc_location id="location-CouchIP-SRVRDSW03-100" node="SRVRDSW03" rsc="CouchIP" score="100"/>
<rsc_location id="location-FrontEnd-SRVRDSW01-200" node="SRVRDSW01" rsc="FrontEnd" score="200"/>
<rsc_location id="location-FrontEnd-SRVRDSW02-100" node="SRVRDSW02" rsc="FrontEnd" score="100"/>
<rsc_location id="location-FrontEnd-SRVRDSW03--INFINITY" node="SRVRDSW03" rsc="FrontEnd" score="-INFINITY"/>
<rsc_location id="location-ITATESTSERVER-DIP-SRVRDSW01-200" node="SRVRDSW01" rsc="ITATESTSERVER-DIP" score="200"/>
<rsc_location id="location-ITATESTSERVER-DIP-SRVRDSW02-100" node="SRVRDSW02" rsc="ITATESTSERVER-DIP" score="100"/>
<rsc_location id="location-ITATESTSERVER-DIP-SRVRDSW03--INFINITY" node="SRVRDSW03" rsc="ITATESTSERVER-DIP" score="-INFINITY"/>
</constraints>
</configuration>
<status>
<node_state id="739248760" uname="SRVDRSW02" in_ccm="true" crmd="online" crm-debug-origin="do_update_resource" join="member" expected="member">
<transient_attributes id="739248760">
<instance_attributes id="status-739248760">
<nvpair id="status-739248760-shutdown" name="shutdown" value="0"/>
</instance_attributes>
</transient_attributes>
<lrm id="739248760">
<lrm_resources>
<lrm_resource id="CouchIP" type="IPaddr2" class="ocf" provider="heartbeat">
<lrm_rsc_op id="CouchIP_last_0" operation_key="CouchIP_stop_0" operation="stop" crm-debug-origin="do_update_resource" crm_feature_set="3.0.11" transition-key="10:639:0:ed3d0c11-dff2-42e1-a1ba-f22890cdf734" transition-magic="0:0;10:639:0:ed3d0c11-dff2-42e1-a1ba-f22890cdf734" on_node="SRVDRSW02" call-id="237" rc-code="0" op-status="0" interval="0" last-run="1571378456" last-rc-change="1571378456" exec-time="128" queue-time="0" op-digest="e9ff5a78ec19403862eecf0c0fc782a2"/>
<lrm_rsc_op id="CouchIP_monitor_30000" operation_key="CouchIP_monitor_30000" operation="monitor" crm-debug-origin="do_update_resource" crm_feature_set="3.0.11" transition-key="10:580:0:ed3d0c11-dff2-42e1-a1ba-f22890cdf734" transition-magic="0:0;10:580:0:ed3d0c11-dff2-42e1-a1ba-f22890cdf734" on_node="SRVDRSW02" call-id="235" rc-code="0" op-status="0" interval="30000" last-rc-change="1571325338" exec-time="55" queue-time="0" op-digest="54ce7fcd0b3f868e5646a2b4dc9bf0c0"/>
</lrm_resource>
<lrm_resource id="FrontEnd" type="nginx" class="ocf" provider="heartbeat">
<lrm_rsc_op id="FrontEnd_last_0" operation_key="FrontEnd_monitor_0" operation="monitor" crm-debug-origin="build_active_RAs" crm_feature_set="3.0.11" transition-key="5:513:7:ed3d0c11-dff2-42e1-a1ba-f22890cdf734" transition-magic="0:7;5:513:7:ed3d0c11-dff2-42e1-a1ba-f22890cdf734" on_node="SRVDRSW02" call-id="215" rc-code="7" op-status="0" interval="0" last-run="1571322749" last-rc-change="1571322749" exec-time="51" queue-time="0" op-digest="5611a661a872392d48752d304f6fd013" op-force-restart=" configfile " op-restart-digest="5611a661a872392d48752d304f6fd013"/>
</lrm_resource>
<lrm_resource id="ClusterIP" type="IPaddr2" class="ocf" provider="heartbeat">
<lrm_rsc_op id="ClusterIP_last_0" operation_key="ClusterIP_monitor_0" operation="monitor" crm-debug-origin="build_active_RAs" crm_feature_set="3.0.11" transition-key="3:511:7:ed3d0c11-dff2-42e1-a1ba-f22890cdf734" transition-magic="0:7;3:511:7:ed3d0c11-dff2-42e1-a1ba-f22890cdf734" on_node="SRVDRSW02" call-id="205" rc-code="7" op-status="0" interval="0" last-run="1571322748" last-rc-change="1571322748" exec-time="23" queue-time="0" op-digest="8c0cc6a2f21addd6868e2865905746ba"/>
</lrm_resource>
<lrm_resource id="ITATESTSERVER-DIP" type="pm2" class="ocf" provider="nodejs">
<lrm_rsc_op id="ITATESTSERVER-DIP_last_0" operation_key="ITATESTSERVER-DIP_monitor_0" operation="monitor" crm-debug-origin="build_active_RAs" crm_feature_set="3.0.11" transition-key="5:514:7:ed3d0c11-dff2-42e1-a1ba-f22890cdf734" transition-magic="0:7;5:514:7:ed3d0c11-dff2-42e1-a1ba-f22890cdf734" on_node="SRVDRSW02" call-id="219" rc-code="7" op-status="0" interval="0" last-run="1571322749" last-rc-change="1571322749" exec-time="5581" queue-time="0" op-digest="a35ca3e47b2b14a0bf6f2bca5ce3d63a" op-secure-params=" user " op-secure-digest="f2317cad3d54cec5d7d7aa7d0bf35cf8"/>
</lrm_resource>
</lrm_resources>
</lrm>
</node_state>
<node_state id="739248761" uname="SRVDRSW03" in_ccm="true" crmd="online" join="member" expected="member" crm-debug-origin="do_update_resource">
<transient_attributes id="739248761">
<instance_attributes id="status-739248761">
<nvpair id="status-739248761-shutdown" name="shutdown" value="0"/>
</instance_attributes>
</transient_attributes>
<lrm id="739248761">
<lrm_resources>
<lrm_resource id="ClusterIP" type="IPaddr2" class="ocf" provider="heartbeat">
<lrm_rsc_op id="ClusterIP_last_0" operation_key="ClusterIP_stop_0" operation="stop" crm-debug-origin="do_update_resource" crm_feature_set="3.0.11" transition-key="7:535:0:ed3d0c11-dff2-42e1-a1ba-f22890cdf734" transition-magic="0:0;7:535:0:ed3d0c11-dff2-42e1-a1ba-f22890cdf734" on_node="SRVDRSW03" call-id="108" rc-code="0" op-status="0" interval="0" last-run="1571324515" last-rc-change="1571324515" exec-time="37" queue-time="0" op-digest="8c0cc6a2f21addd6868e2865905746ba"/>
<lrm_rsc_op id="ClusterIP_monitor_30000" operation_key="ClusterIP_monitor_30000" operation="monitor" crm-debug-origin="do_update_resource" crm_feature_set="3.0.11" transition-key="3:529:0:ed3d0c11-dff2-42e1-a1ba-f22890cdf734" transition-magic="0:0;3:529:0:ed3d0c11-dff2-42e1-a1ba-f22890cdf734" on_node="SRVDRSW03" call-id="94" rc-code="0" op-status="0" interval="30000" last-rc-change="1571323896" exec-time="94" queue-time="0" op-digest="3dcdeb02ce2a74fc6fc8ec8da841a615"/>
</lrm_resource>
<lrm_resource id="CouchIP" type="IPaddr2" class="ocf" provider="heartbeat">
<lrm_rsc_op id="CouchIP_last_0" operation_key="CouchIP_stop_0" operation="stop" crm-debug-origin="do_update_resource" crm_feature_set="3.0.11" transition-key="9:573:0:ed3d0c11-dff2-42e1-a1ba-f22890cdf734" transition-magic="0:0;9:573:0:ed3d0c11-dff2-42e1-a1ba-f22890cdf734" on_node="SRVDRSW03" call-id="112" rc-code="0" op-status="0" interval="0" last-run="1571325190" last-rc-change="1571325190" exec-time="32" queue-time="0" op-digest="e9ff5a78ec19403862eecf0c0fc782a2"/>
<lrm_rsc_op id="CouchIP_monitor_30000" operation_key="CouchIP_monitor_30000" operation="monitor" crm-debug-origin="do_update_resource" crm_feature_set="3.0.11" transition-key="11:572:0:ed3d0c11-dff2-42e1-a1ba-f22890cdf734" transition-magic="0:0;11:572:0:ed3d0c11-dff2-42e1-a1ba-f22890cdf734" on_node="SRVDRSW03" call-id="110" rc-code="0" op-status="0" interval="30000" last-rc-change="1571325187" exec-time="64" queue-time="0" op-digest="54ce7fcd0b3f868e5646a2b4dc9bf0c0"/>
</lrm_resource>
<lrm_resource id="FrontEnd" type="nginx" class="ocf" provider="heartbeat">
<lrm_rsc_op id="FrontEnd_last_0" operation_key="FrontEnd_stop_0" operation="stop" crm-debug-origin="do_update_resource" crm_feature_set="3.0.11" transition-key="13:535:0:ed3d0c11-dff2-42e1-a1ba-f22890cdf734" transition-magic="0:0;13:535:0:ed3d0c11-dff2-42e1-a1ba-f22890cdf734" on_node="SRVDRSW03" call-id="104" rc-code="0" op-status="0" interval="0" last-run="1571324514" last-rc-change="1571324514" exec-time="1255" queue-time="0" op-digest="5611a661a872392d48752d304f6fd013" op-force-restart=" configfile " op-restart-digest="5611a661a872392d48752d304f6fd013"/>
<lrm_rsc_op id="FrontEnd_monitor_30000" operation_key="FrontEnd_monitor_30000" operation="monitor" crm-debug-origin="do_update_resource" crm_feature_set="3.0.11" transition-key="8:529:0:ed3d0c11-dff2-42e1-a1ba-f22890cdf734" transition-magic="0:0;8:529:0:ed3d0c11-dff2-42e1-a1ba-f22890cdf734" on_node="SRVDRSW03" call-id="97" rc-code="0" op-status="0" interval="30000" last-rc-change="1571323896" exec-time="119" queue-time="0" op-digest="d70fbbb0c830371f29d91efc2ed7a51a"/>
<lrm_rsc_op id="FrontEnd_monitor_10000" operation_key="FrontEnd_monitor_10000" operation="monitor" crm-debug-origin="do_update_resource" crm_feature_set="3.0.11" transition-key="7:529:0:ed3d0c11-dff2-42e1-a1ba-f22890cdf734" transition-magic="0:0;7:529:0:ed3d0c11-dff2-42e1-a1ba-f22890cdf734" on_node="SRVDRSW03" call-id="98" rc-code="0" op-status="0" interval="10000" last-rc-change="1571323896" exec-time="153" queue-time="117" op-digest="d70fbbb0c830371f29d91efc2ed7a51a"/>
</lrm_resource>
<lrm_resource id="ITATESTSERVER-DIP" type="pm2" class="ocf" provider="nodejs">
<lrm_rsc_op id="ITATESTSERVER-DIP_last_0" operation_key="ITATESTSERVER-DIP_stop_0" operation="stop" crm-debug-origin="do_update_resource" crm_feature_set="3.0.11" transition-key="17:535:0:ed3d0c11-dff2-42e1-a1ba-f22890cdf734" transition-magic="0:0;17:535:0:ed3d0c11-dff2-42e1-a1ba-f22890cdf734" on_node="SRVDRSW03" call-id="106" rc-code="0" op-status="0" interval="0" last-run="1571324514" last-rc-change="1571324514" exec-time="1034" queue-time="0" op-digest="a35ca3e47b2b14a0bf6f2bca5ce3d63a" op-secure-params=" user " op-secure-digest="f2317cad3d54cec5d7d7aa7d0bf35cf8"/>
<lrm_rsc_op id="ITATESTSERVER-DIP_monitor_10000" operation_key="ITATESTSERVER-DIP_monitor_10000" operation="monitor" crm-debug-origin="do_update_resource" crm_feature_set="3.0.11" transition-key="10:529:0:ed3d0c11-dff2-42e1-a1ba-f22890cdf734" transition-magic="0:0;10:529:0:ed3d0c11-dff2-42e1-a1ba-f22890cdf734" on_node="SRVDRSW03" call-id="99" rc-code="0" op-status="0" interval="10000" last-rc-change="1571323898" exec-time="387" queue-time="0" op-digest="3692cedc2f4621c820f8f276381a52c5" op-secure-params=" user " op-secure-digest="f2317cad3d54cec5d7d7aa7d0bf35cf8"/>
</lrm_resource>
</lrm_resources>
</lrm>
</node_state>
<node_state id="739248772" uname="SRVRDSW05.seko.com" in_ccm="true" crmd="online" join="member" expected="member" crm-debug-origin="do_update_resource">
<transient_attributes id="739248772">
<instance_attributes id="status-739248772">
<nvpair id="status-739248772-shutdown" name="shutdown" value="0"/>
</instance_attributes>
</transient_attributes>
<lrm id="739248772">
<lrm_resources>
<lrm_resource id="ClusterIP" type="IPaddr2" class="ocf" provider="heartbeat">
<lrm_rsc_op id="ClusterIP_last_0" operation_key="ClusterIP_monitor_0" operation="monitor" crm-debug-origin="do_update_resource" crm_feature_set="3.0.11" transition-key="6:511:7:ed3d0c11-dff2-42e1-a1ba-f22890cdf734" transition-magic="0:7;6:511:7:ed3d0c11-dff2-42e1-a1ba-f22890cdf734" on_node="SRVRDSW05.seko.com" call-id="30" rc-code="7" op-status="0" interval="0" last-run="1571322748" last-rc-change="1571322748" exec-time="152" queue-time="0" op-digest="8c0cc6a2f21addd6868e2865905746ba"/>
</lrm_resource>
<lrm_resource id="CouchIP" type="IPaddr2" class="ocf" provider="heartbeat">
<lrm_rsc_op id="CouchIP_last_0" operation_key="CouchIP_stop_0" operation="stop" crm-debug-origin="do_update_resource" crm_feature_set="3.0.11" transition-key="9:575:0:ed3d0c11-dff2-42e1-a1ba-f22890cdf734" transition-magic="0:0;9:575:0:ed3d0c11-dff2-42e1-a1ba-f22890cdf734" on_node="SRVRDSW05.seko.com" call-id="51" rc-code="0" op-status="0" interval="0" last-run="1571325218" last-rc-change="1571325218" exec-time="75" queue-time="0" op-digest="e9ff5a78ec19403862eecf0c0fc782a2"/>
<lrm_rsc_op id="CouchIP_monitor_30000" operation_key="CouchIP_monitor_30000" operation="monitor" crm-debug-origin="do_update_resource" crm_feature_set="3.0.11" transition-key="11:574:0:ed3d0c11-dff2-42e1-a1ba-f22890cdf734" transition-magic="0:0;11:574:0:ed3d0c11-dff2-42e1-a1ba-f22890cdf734" on_node="SRVRDSW05.seko.com" call-id="49" rc-code="0" op-status="0" interval="30000" last-rc-change="1571325197" exec-time="56" queue-time="0" op-digest="54ce7fcd0b3f868e5646a2b4dc9bf0c0"/>
</lrm_resource>
<lrm_resource id="FrontEnd" type="nginx" class="ocf" provider="heartbeat">
<lrm_rsc_op id="FrontEnd_last_0" operation_key="FrontEnd_monitor_0" operation="monitor" crm-debug-origin="do_update_resource" crm_feature_set="3.0.11" transition-key="8:513:7:ed3d0c11-dff2-42e1-a1ba-f22890cdf734" transition-magic="0:7;8:513:7:ed3d0c11-dff2-42e1-a1ba-f22890cdf734" on_node="SRVRDSW05.seko.com" call-id="38" rc-code="7" op-status="0" interval="0" last-run="1571322749" last-rc-change="1571322749" exec-time="511" queue-time="0" op-digest="5611a661a872392d48752d304f6fd013" op-force-restart=" configfile " op-restart-digest="5611a661a872392d48752d304f6fd013"/>
</lrm_resource>
<lrm_resource id="ITATESTSERVER-DIP" type="pm2" class="ocf" provider="nodejs">
<lrm_rsc_op id="ITATESTSERVER-DIP_last_0" operation_key="ITATESTSERVER-DIP_stop_0" operation="stop" crm-debug-origin="do_update_resource" crm_feature_set="3.0.11" transition-key="14:515:0:ed3d0c11-dff2-42e1-a1ba-f22890cdf734" transition-magic="0:0;14:515:0:ed3d0c11-dff2-42e1-a1ba-f22890cdf734" on_node="SRVRDSW05.seko.com" call-id="43" rc-code="0" op-status="0" interval="0" last-run="1571322755" last-rc-change="1571322755" exec-time="1313" queue-time="0" op-digest="a35ca3e47b2b14a0bf6f2bca5ce3d63a" op-secure-params=" user " op-secure-digest="f2317cad3d54cec5d7d7aa7d0bf35cf8"/>
</lrm_resource>
</lrm_resources>
</lrm>
</node_state>
<node_state id="739248771" uname="SRVDRSW04.seko.com" in_ccm="true" crmd="online" crm-debug-origin="do_update_resource" join="member" expected="member">
<lrm id="739248771">
<lrm_resources>
<lrm_resource id="ClusterIP" type="IPaddr2" class="ocf" provider="heartbeat">
<lrm_rsc_op id="ClusterIP_last_0" operation_key="ClusterIP_stop_0" operation="stop" crm-debug-origin="do_update_resource" crm_feature_set="3.0.11" transition-key="3:537:0:ed3d0c11-dff2-42e1-a1ba-f22890cdf734" transition-magic="0:0;3:537:0:ed3d0c11-dff2-42e1-a1ba-f22890cdf734" on_node="SRVDRSW04.seko.com" call-id="57" rc-code="0" op-status="0" interval="0" last-run="1571324842" last-rc-change="1571324842" exec-time="33" queue-time="0" op-digest="8c0cc6a2f21addd6868e2865905746ba"/>
<lrm_rsc_op id="ClusterIP_monitor_30000" operation_key="ClusterIP_monitor_30000" operation="monitor" crm-debug-origin="do_update_resource" crm_feature_set="3.0.11" transition-key="9:535:0:ed3d0c11-dff2-42e1-a1ba-f22890cdf734" transition-magic="0:0;9:535:0:ed3d0c11-dff2-42e1-a1ba-f22890cdf734" on_node="SRVDRSW04.seko.com" call-id="45" rc-code="0" op-status="0" interval="30000" last-rc-change="1571324515" exec-time="123" queue-time="0" op-digest="3dcdeb02ce2a74fc6fc8ec8da841a615"/>
</lrm_resource>
<lrm_resource id="CouchIP" type="IPaddr2" class="ocf" provider="heartbeat">
<lrm_rsc_op id="CouchIP_last_0" operation_key="CouchIP_stop_0" operation="stop" crm-debug-origin="do_update_resource" crm_feature_set="3.0.11" transition-key="9:574:0:ed3d0c11-dff2-42e1-a1ba-f22890cdf734" transition-magic="0:0;9:574:0:ed3d0c11-dff2-42e1-a1ba-f22890cdf734" on_node="SRVDRSW04.seko.com" call-id="61" rc-code="0" op-status="0" interval="0" last-run="1571325197" last-rc-change="1571325197" exec-time="29" queue-time="0" op-digest="e9ff5a78ec19403862eecf0c0fc782a2"/>
<lrm_rsc_op id="CouchIP_monitor_30000" operation_key="CouchIP_monitor_30000" operation="monitor" crm-debug-origin="do_update_resource" crm_feature_set="3.0.11" transition-key="11:573:0:ed3d0c11-dff2-42e1-a1ba-f22890cdf734" transition-magic="0:0;11:573:0:ed3d0c11-dff2-42e1-a1ba-f22890cdf734" on_node="SRVDRSW04.seko.com" call-id="59" rc-code="0" op-status="0" interval="30000" last-rc-change="1571325190" exec-time="22" queue-time="0" op-digest="54ce7fcd0b3f868e5646a2b4dc9bf0c0"/>
</lrm_resource>
<lrm_resource id="FrontEnd" type="nginx" class="ocf" provider="heartbeat">
<lrm_rsc_op id="FrontEnd_last_0" operation_key="FrontEnd_stop_0" operation="stop" crm-debug-origin="do_update_resource" crm_feature_set="3.0.11" transition-key="13:536:0:ed3d0c11-dff2-42e1-a1ba-f22890cdf734" transition-magic="0:0;13:536:0:ed3d0c11-dff2-42e1-a1ba-f22890cdf734" on_node="SRVDRSW04.seko.com" call-id="53" rc-code="0" op-status="0" interval="0" last-run="1571324841" last-rc-change="1571324841" exec-time="1478" queue-time="0" op-digest="5611a661a872392d48752d304f6fd013" op-force-restart=" configfile " op-restart-digest="5611a661a872392d48752d304f6fd013"/>
<lrm_rsc_op id="FrontEnd_monitor_30000" operation_key="FrontEnd_monitor_30000" operation="monitor" crm-debug-origin="do_update_resource" crm_feature_set="3.0.11" transition-key="16:535:0:ed3d0c11-dff2-42e1-a1ba-f22890cdf734" transition-magic="0:0;16:535:0:ed3d0c11-dff2-42e1-a1ba-f22890cdf734" on_node="SRVDRSW04.seko.com" call-id="48" rc-code="0" op-status="0" interval="30000" last-rc-change="1571324516" exec-time="124" queue-time="1" op-digest="d70fbbb0c830371f29d91efc2ed7a51a"/>
<lrm_rsc_op id="FrontEnd_monitor_10000" operation_key="FrontEnd_monitor_10000" operation="monitor" crm-debug-origin="do_update_resource" crm_feature_set="3.0.11" transition-key="15:535:0:ed3d0c11-dff2-42e1-a1ba-f22890cdf734" transition-magic="0:0;15:535:0:ed3d0c11-dff2-42e1-a1ba-f22890cdf734" on_node="SRVDRSW04.seko.com" call-id="49" rc-code="0" op-status="0" interval="10000" last-rc-change="1571324516" exec-time="123" queue-time="123" op-digest="d70fbbb0c830371f29d91efc2ed7a51a"/>
</lrm_resource>
<lrm_resource id="ITATESTSERVER-DIP" type="pm2" class="ocf" provider="nodejs">
<lrm_rsc_op id="ITATESTSERVER-DIP_last_0" operation_key="ITATESTSERVER-DIP_stop_0" operation="stop" crm-debug-origin="do_update_resource" crm_feature_set="3.0.11" transition-key="17:536:0:ed3d0c11-dff2-42e1-a1ba-f22890cdf734" transition-magic="0:0;17:536:0:ed3d0c11-dff2-42e1-a1ba-f22890cdf734" on_node="SRVDRSW04.seko.com" call-id="55" rc-code="0" op-status="0" interval="0" last-run="1571324841" last-rc-change="1571324841" exec-time="711" queue-time="0" op-digest="a35ca3e47b2b14a0bf6f2bca5ce3d63a" op-secure-params=" user " op-secure-digest="f2317cad3d54cec5d7d7aa7d0bf35cf8"/>
<lrm_rsc_op id="ITATESTSERVER-DIP_monitor_10000" operation_key="ITATESTSERVER-DIP_monitor_10000" operation="monitor" crm-debug-origin="do_update_resource" crm_feature_set="3.0.11" transition-key="19:535:0:ed3d0c11-dff2-42e1-a1ba-f22890cdf734" transition-magic="0:0;19:535:0:ed3d0c11-dff2-42e1-a1ba-f22890cdf734" on_node="SRVDRSW04.seko.com" call-id="50" rc-code="0" op-status="0" interval="10000" last-rc-change="1571324518" exec-time="458" queue-time="0" op-digest="3692cedc2f4621c820f8f276381a52c5" op-secure-params=" user " op-secure-digest="f2317cad3d54cec5d7d7aa7d0bf35cf8"/>
</lrm_resource>
</lrm_resources>
</lrm>
<transient_attributes id="739248771">
<instance_attributes id="status-739248771">
<nvpair id="status-739248771-shutdown" name="shutdown" value="0"/>
</instance_attributes>
</transient_attributes>
</node_state>
<node_state id="739248759" uname="SRVDRSW01" in_ccm="true" crmd="online" join="member" expected="member" crm-debug-origin="do_update_resource">
<lrm id="739248759">
<lrm_resources>
<lrm_resource id="ClusterIP" type="IPaddr2" class="ocf" provider="heartbeat">
<lrm_rsc_op id="ClusterIP_last_0" operation_key="ClusterIP_start_0" operation="start" crm-debug-origin="do_update_resource" crm_feature_set="3.0.11" transition-key="2:641:0:ed3d0c11-dff2-42e1-a1ba-f22890cdf734" transition-magic="0:0;2:641:0:ed3d0c11-dff2-42e1-a1ba-f22890cdf734" on_node="SRVDRSW01" call-id="431" rc-code="0" op-status="0" interval="0" last-run="1571378655" last-rc-change="1571378655" exec-time="56" queue-time="0" op-digest="8c0cc6a2f21addd6868e2865905746ba"/>
<lrm_rsc_op id="ClusterIP_monitor_30000" operation_key="ClusterIP_monitor_30000" operation="monitor" crm-debug-origin="do_update_resource" crm_feature_set="3.0.11" transition-key="3:641:0:ed3d0c11-dff2-42e1-a1ba-f22890cdf734" transition-magic="0:0;3:641:0:ed3d0c11-dff2-42e1-a1ba-f22890cdf734" on_node="SRVDRSW01" call-id="434" rc-code="0" op-status="0" interval="30000" last-rc-change="1571378655" exec-time="46" queue-time="0" op-digest="3dcdeb02ce2a74fc6fc8ec8da841a615"/>
</lrm_resource>
<lrm_resource id="CouchIP" type="IPaddr2" class="ocf" provider="heartbeat">
<lrm_rsc_op id="CouchIP_last_0" operation_key="CouchIP_start_0" operation="start" crm-debug-origin="do_update_resource" crm_feature_set="3.0.11" transition-key="4:641:0:ed3d0c11-dff2-42e1-a1ba-f22890cdf734" transition-magic="0:0;4:641:0:ed3d0c11-dff2-42e1-a1ba-f22890cdf734" on_node="SRVDRSW01" call-id="432" rc-code="0" op-status="0" interval="0" last-run="1571378655" last-rc-change="1571378655" exec-time="50" queue-time="1" op-digest="e9ff5a78ec19403862eecf0c0fc782a2"/>
<lrm_rsc_op id="CouchIP_monitor_30000" operation_key="CouchIP_monitor_30000" operation="monitor" crm-debug-origin="do_update_resource" crm_feature_set="3.0.11" transition-key="5:641:0:ed3d0c11-dff2-42e1-a1ba-f22890cdf734" transition-magic="0:0;5:641:0:ed3d0c11-dff2-42e1-a1ba-f22890cdf734" on_node="SRVDRSW01" call-id="433" rc-code="0" op-status="0" interval="30000" last-rc-change="1571378655" exec-time="39" queue-time="0" op-digest="54ce7fcd0b3f868e5646a2b4dc9bf0c0"/>
</lrm_resource>
<lrm_resource id="FrontEnd" type="nginx" class="ocf" provider="heartbeat">
<lrm_rsc_op id="FrontEnd_last_0" operation_key="FrontEnd_start_0" operation="start" crm-debug-origin="do_update_resource" crm_feature_set="3.0.11" transition-key="6:641:0:ed3d0c11-dff2-42e1-a1ba-f22890cdf734" transition-magic="0:0;6:641:0:ed3d0c11-dff2-42e1-a1ba-f22890cdf734" on_node="SRVDRSW01" call-id="435" rc-code="0" op-status="0" interval="0" last-run="1571378655" last-rc-change="1571378655" exec-time="125" queue-time="0" op-digest="5611a661a872392d48752d304f6fd013" op-force-restart=" configfile " op-restart-digest="5611a661a872392d48752d304f6fd013"/>
<lrm_rsc_op id="FrontEnd_monitor_30000" operation_key="FrontEnd_monitor_30000" operation="monitor" crm-debug-origin="do_update_resource" crm_feature_set="3.0.11" transition-key="8:641:0:ed3d0c11-dff2-42e1-a1ba-f22890cdf734" transition-magic="0:0;8:641:0:ed3d0c11-dff2-42e1-a1ba-f22890cdf734" on_node="SRVDRSW01" call-id="437" rc-code="0" op-status="0" interval="30000" last-rc-change="1571378656" exec-time="44" queue-time="1" op-digest="d70fbbb0c830371f29d91efc2ed7a51a"/>
<lrm_rsc_op id="FrontEnd_monitor_10000" operation_key="FrontEnd_monitor_10000" operation="monitor" crm-debug-origin="do_update_resource" crm_feature_set="3.0.11" transition-key="7:641:0:ed3d0c11-dff2-42e1-a1ba-f22890cdf734" transition-magic="0:0;7:641:0:ed3d0c11-dff2-42e1-a1ba-f22890cdf734" on_node="SRVDRSW01" call-id="438" rc-code="0" op-status="0" interval="10000" last-rc-change="1571378656" exec-time="46" queue-time="44" op-digest="d70fbbb0c830371f29d91efc2ed7a51a"/>
</lrm_resource>
<lrm_resource id="ITATESTSERVER-DIP" type="pm2" class="ocf" provider="nodejs">
<lrm_rsc_op id="ITATESTSERVER-DIP_last_0" operation_key="ITATESTSERVER-DIP_start_0" operation="start" crm-debug-origin="do_update_resource" crm_feature_set="3.0.11" transition-key="9:641:0:ed3d0c11-dff2-42e1-a1ba-f22890cdf734" transition-magic="0:0;9:641:0:ed3d0c11-dff2-42e1-a1ba-f22890cdf734" on_node="SRVDRSW01" call-id="436" rc-code="0" op-status="0" interval="0" last-run="1571378655" last-rc-change="1571378655" exec-time="6261" queue-time="0" op-digest="a35ca3e47b2b14a0bf6f2bca5ce3d63a" op-secure-params=" user " op-secure-digest="f2317cad3d54cec5d7d7aa7d0bf35cf8"/>
<lrm_rsc_op id="ITATESTSERVER-DIP_monitor_10000" operation_key="ITATESTSERVER-DIP_monitor_10000" operation="monitor" crm-debug-origin="do_update_resource" crm_feature_set="3.0.11" transition-key="10:641:0:ed3d0c11-dff2-42e1-a1ba-f22890cdf734" transition-magic="0:0;10:641:0:ed3d0c11-dff2-42e1-a1ba-f22890cdf734" on_node="SRVDRSW01" call-id="439" rc-code="0" op-status="0" interval="10000" last-rc-change="1571378662" exec-time="407" queue-time="0" op-digest="3692cedc2f4621c820f8f276381a52c5" op-secure-params=" user " op-secure-digest="f2317cad3d54cec5d7d7aa7d0bf35cf8"/>
</lrm_resource>
</lrm_resources>
</lrm>
<transient_attributes id="739248759">
<instance_attributes id="status-739248759"/>
</transient_attributes>
</node_state>
<node_state id="739248773" in_ccm="true" crmd="online" crm-debug-origin="do_update_resource" uname="srvdrsw06.seko.com" join="member" expected="member">
<lrm id="739248773">
<lrm_resources>
<lrm_resource id="CouchIP" type="IPaddr2" class="ocf" provider="heartbeat">
<lrm_rsc_op id="CouchIP_last_0" operation_key="CouchIP_stop_0" operation="stop" crm-debug-origin="do_update_resource" crm_feature_set="3.0.11" transition-key="9:576:0:ed3d0c11-dff2-42e1-a1ba-f22890cdf734" transition-magic="0:0;9:576:0:ed3d0c11-dff2-42e1-a1ba-f22890cdf734" on_node="srvdrsw06.seko.com" call-id="26" rc-code="0" op-status="0" interval="0" last-run="1571325225" last-rc-change="1571325225" exec-time="347" queue-time="0" op-digest="e9ff5a78ec19403862eecf0c0fc782a2"/>
<lrm_rsc_op id="CouchIP_monitor_30000" operation_key="CouchIP_monitor_30000" operation="monitor" crm-debug-origin="do_update_resource" crm_feature_set="3.0.11" transition-key="11:575:0:ed3d0c11-dff2-42e1-a1ba-f22890cdf734" transition-magic="0:0;11:575:0:ed3d0c11-dff2-42e1-a1ba-f22890cdf734" on_node="srvdrsw06.seko.com" call-id="24" rc-code="0" op-status="0" interval="30000" last-rc-change="1571325219" exec-time="149" queue-time="0" op-digest="54ce7fcd0b3f868e5646a2b4dc9bf0c0"/>
</lrm_resource>
<lrm_resource id="ClusterIP" type="IPaddr2" class="ocf" provider="heartbeat">
<lrm_rsc_op id="ClusterIP_last_0" operation_key="ClusterIP_monitor_0" operation="monitor" crm-debug-origin="do_update_resource" crm_feature_set="3.0.11" transition-key="7:519:7:ed3d0c11-dff2-42e1-a1ba-f22890cdf734" transition-magic="0:7;7:519:7:ed3d0c11-dff2-42e1-a1ba-f22890cdf734" on_node="srvdrsw06.seko.com" call-id="5" rc-code="7" op-status="0" interval="0" last-run="1571323002" last-rc-change="1571323002" exec-time="1394" queue-time="0" op-digest="8c0cc6a2f21addd6868e2865905746ba"/>
</lrm_resource>
<lrm_resource id="FrontEnd" type="nginx" class="ocf" provider="heartbeat">
<lrm_rsc_op id="FrontEnd_last_0" operation_key="FrontEnd_stop_0" operation="stop" crm-debug-origin="do_update_resource" crm_feature_set="3.0.11" transition-key="11:520:0:ed3d0c11-dff2-42e1-a1ba-f22890cdf734" transition-magic="0:0;11:520:0:ed3d0c11-dff2-42e1-a1ba-f22890cdf734" on_node="srvdrsw06.seko.com" call-id="19" rc-code="0" op-status="0" interval="0" last-run="1571323007" last-rc-change="1571323007" exec-time="1181" queue-time="0" op-digest="5611a661a872392d48752d304f6fd013" op-force-restart=" configfile " op-restart-digest="5611a661a872392d48752d304f6fd013"/>
</lrm_resource>
<lrm_resource id="ITATESTSERVER-DIP" type="pm2" class="ocf" provider="nodejs">
<lrm_rsc_op id="ITATESTSERVER-DIP_last_0" operation_key="ITATESTSERVER-DIP_monitor_0" operation="monitor" crm-debug-origin="do_update_resource" crm_feature_set="3.0.11" transition-key="10:519:7:ed3d0c11-dff2-42e1-a1ba-f22890cdf734" transition-magic="0:5;10:519:7:ed3d0c11-dff2-42e1-a1ba-f22890cdf734" on_node="srvdrsw06.seko.com" call-id="17" rc-code="5" op-status="0" interval="0" last-run="1571323003" last-rc-change="1571323003" exec-time="3148" queue-time="1" op-digest="a35ca3e47b2b14a0bf6f2bca5ce3d63a" op-secure-params=" user " op-secure-digest="f2317cad3d54cec5d7d7aa7d0bf35cf8"/>
</lrm_resource>
</lrm_resources>
</lrm>
<transient_attributes id="739248773">
<instance_attributes id="status-739248773"/>
</transient_attributes>
</node_state>
</status>
</cib>
_______________________________________________
Manage your subscription:
https://lists.clusterlabs.org/mailman/listinfo/users
ClusterLabs home: https://www.clusterlabs.org/
