07.03.2014 10:30, Vladislav Bogdanov wrote:
> 07.03.2014 05:43, Andrew Beekhof wrote:
>>
>> On 6 Mar 2014, at 10:39 pm, Vladislav Bogdanov <bub...@hoster-ok.com> wrote:
>>
>>> 18.02.2014 03:49, Andrew Beekhof wrote:
>>>>
>>>> On 31 Jan 2014, at 6:20 pm, yusuke iida <yusk.i...@gmail.com> wrote:
>>>>
>>>>> Hi, all
>>>>>
>>>>> I measure the performance of Pacemaker in the following combinations.
>>>>> Pacemaker-1.1.11.rc1
>>>>> libqb-0.16.0
>>>>> corosync-2.3.2
>>>>>
>>>>> All nodes are KVM virtual machines.
>>>>>
>>>>> stopped the node of vm01 compulsorily from the inside, after starting 14
>>>>> nodes.
>>>>> "virsh destroy vm01" was used for the stop.
>>>>> Then, in addition to the compulsorily stopped node, other nodes are
>>>>> separated from a cluster.
>>>>>
>>>>> The log of "Retransmit List:" is then outputted in large quantities from
>>>>> corosync.
>>>>
>>>> Probably best to poke the corosync guys about this.
>>>>
>>>> However, <= .11 is known to cause significant CPU usage with that many
>>>> nodes.
>>>> I can easily imagine this staving corosync of resources and causing
>>>> breakage.
>>>>
>>>> I would _highly_ recommend retesting with the current git master of
>>>> pacemaker.
>>>> I merged the new cib code last week which is faster by _two_ orders of
>>>> magnitude and uses significantly less CPU.
>>>
>>> Andrew, current git master (ee094a2) almost works, the only issue is
>>> that crm_diff calculates incorrect diff digest. If I replace digest in
>>> diff by hands with what cib calculates as "expected". it applies
>>> correctly. Otherwise - -206.
>>
>> More details?
>
> Hmmm...
> seems to be crmsh-specific,
> Cannot reproduce with pure-XML editing.
> Kristoffer, does
> http://hg.savannah.gnu.org/hgweb/crmsh/rev/c42d9361a310 address this?
The problem seems to be caused by the fact that crmsh does not provide
<status> section in both orig and new XMLs to crm_diff, and digest
generation seems to rely on that, so crm_diff and cib daemon produce
different digests.
Attached are two sets of XML files, one (orig.xml, new.xml, patch.xml)
are related to the full CIB operation (with status section included),
another (orig-edited.xml, new-edited.xml, patch-edited.xml) have that
section removed like crmsh does do.
Resulting diffs differ only by digest, and that seems to be the exact issue.
<cib epoch="4" num_updates="5" admin_epoch="0" validate-with="pacemaker-1.2" cib-last-written="Tue Mar 11 06:57:54 2014" update-origin="booter-0" update-client="crmd" update-user="hacluster" crm_feature_set="3.0.9" have-quorum="1" dc-uuid="1">
<configuration>
<crm_config>
<cluster_property_set id="cib-bootstrap-options">
<nvpair id="cib-bootstrap-options-dc-version" name="dc-version" value="1.1.11-1.3.el6-b75a9bd"/>
<nvpair id="cib-bootstrap-options-cluster-infrastructure" name="cluster-infrastructure" value="corosync"/>
<nvpair name="symmetric-cluster" value="true" id="cib-bootstrap-options-symmetric-cluster"/>
</cluster_property_set>
</crm_config>
<nodes>
<node id="1" uname="booter-0"/>
<node id="2" uname="booter-1"/>
</nodes>
<resources/>
<constraints/>
</configuration>
<status>
<node_state id="1" uname="booter-0" in_ccm="true" crmd="online" crm-debug-origin="do_state_transition" join="member" expected="member">
<lrm id="1">
<lrm_resources/>
</lrm>
<transient_attributes id="1">
<instance_attributes id="status-1">
<nvpair id="status-1-shutdown" name="shutdown" value="0"/>
<nvpair id="status-1-probe_complete" name="probe_complete" value="true"/>
</instance_attributes>
</transient_attributes>
</node_state>
</status>
</cib>
<cib epoch="4" num_updates="5" admin_epoch="0" validate-with="pacemaker-1.2" cib-last-written="Tue Mar 11 06:57:54 2014" update-origin="booter-0" update-client="crmd" update-user="hacluster" crm_feature_set="3.0.9" have-quorum="1" dc-uuid="1">
<configuration>
<crm_config>
<cluster_property_set id="cib-bootstrap-options">
<nvpair id="cib-bootstrap-options-dc-version" name="dc-version" value="1.1.11-1.3.el6-b75a9bd"/>
<nvpair id="cib-bootstrap-options-cluster-infrastructure" name="cluster-infrastructure" value="corosync"/>
<nvpair name="symmetric-cluster" value="true" id="cib-bootstrap-options-symmetric-cluster"/>
</cluster_property_set>
</crm_config>
<nodes>
<node id="1" uname="booter-0"/>
<node id="2" uname="booter-1"/>
</nodes>
<resources/>
<constraints/>
</configuration>
</cib>
<cib epoch="3" num_updates="5" admin_epoch="0" validate-with="pacemaker-1.2" cib-last-written="Tue Mar 11 06:57:54 2014" update-origin="booter-0" update-client="crmd" update-user="hacluster" crm_feature_set="3.0.9" have-quorum="1" dc-uuid="1">
<configuration>
<crm_config>
<cluster_property_set id="cib-bootstrap-options">
<nvpair id="cib-bootstrap-options-dc-version" name="dc-version" value="1.1.11-1.3.el6-b75a9bd"/>
<nvpair id="cib-bootstrap-options-cluster-infrastructure" name="cluster-infrastructure" value="corosync"/>
</cluster_property_set>
</crm_config>
<nodes>
<node id="1" uname="booter-0"/>
<node id="2" uname="booter-1"/>
</nodes>
<resources/>
<constraints/>
</configuration>
<status>
<node_state id="1" uname="booter-0" in_ccm="true" crmd="online" crm-debug-origin="do_state_transition" join="member" expected="member">
<lrm id="1">
<lrm_resources/>
</lrm>
<transient_attributes id="1">
<instance_attributes id="status-1">
<nvpair id="status-1-shutdown" name="shutdown" value="0"/>
<nvpair id="status-1-probe_complete" name="probe_complete" value="true"/>
</instance_attributes>
</transient_attributes>
</node_state>
</status>
</cib>
<cib epoch="3" num_updates="5" admin_epoch="0" validate-with="pacemaker-1.2" cib-last-written="Tue Mar 11 06:57:54 2014" update-origin="booter-0" update-client="crmd" update-user="hacluster" crm_feature_set="3.0.9" have-quorum="1" dc-uuid="1">
<configuration>
<crm_config>
<cluster_property_set id="cib-bootstrap-options">
<nvpair id="cib-bootstrap-options-dc-version" name="dc-version" value="1.1.11-1.3.el6-b75a9bd"/>
<nvpair id="cib-bootstrap-options-cluster-infrastructure" name="cluster-infrastructure" value="corosync"/>
</cluster_property_set>
</crm_config>
<nodes>
<node id="1" uname="booter-0"/>
<node id="2" uname="booter-1"/>
</nodes>
<resources/>
<constraints/>
</configuration>
</cib>
<diff format="2" digest="e0ac92d71cd76363342274102cdd2d92">
<version>
<source admin_epoch="0" epoch="3" num_updates="5"/>
<target admin_epoch="0" epoch="4" num_updates="5"/>
</version>
<change operation="modify" path="/cib">
<change-list>
<change-attr name="epoch" operation="set" value="4"/>
</change-list>
<change-result>
<cib epoch="4" num_updates="5" admin_epoch="0" validate-with="pacemaker-1.2" cib-last-written="Tue Mar 11 06:57:54 2014" update-origin="booter-0" update-client="crmd" update-user="hacluster" crm_feature_set="3.0.9" have-quorum="1" dc-uuid="1"/>
</change-result>
</change>
<change operation="create" path="/cib/configuration/crm_config/cluster_property_set[@id='cib-bootstrap-options']" position="2">
<nvpair name="symmetric-cluster" value="true" id="cib-bootstrap-options-symmetric-cluster"/>
</change>
</diff>
<diff format="2" digest="0fecc5c0a38bffaf42319e620c1070e4">
<version>
<source admin_epoch="0" epoch="3" num_updates="5"/>
<target admin_epoch="0" epoch="4" num_updates="5"/>
</version>
<change operation="modify" path="/cib">
<change-list>
<change-attr name="epoch" operation="set" value="4"/>
</change-list>
<change-result>
<cib epoch="4" num_updates="5" admin_epoch="0" validate-with="pacemaker-1.2" cib-last-written="Tue Mar 11 06:57:54 2014" update-origin="booter-0" update-client="crmd" update-user="hacluster" crm_feature_set="3.0.9" have-quorum="1" dc-uuid="1"/>
</change-result>
</change>
<change operation="create" path="/cib/configuration/crm_config/cluster_property_set[@id='cib-bootstrap-options']" position="2">
<nvpair name="symmetric-cluster" value="true" id="cib-bootstrap-options-symmetric-cluster"/>
</change>
</diff>
_______________________________________________
Pacemaker mailing list: Pacemaker@oss.clusterlabs.org
http://oss.clusterlabs.org/mailman/listinfo/pacemaker
Project Home: http://www.clusterlabs.org
Getting started: http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf
Bugs: http://bugs.clusterlabs.org
