On 10/17/2016 09:55 AM, Nikhil Utane wrote: > I see these prints. > > pengine: info: rsc_merge_weights:cu_4: Rolling back scores from cu_3 > pengine: debug: native_assign_node:Assigning Redun_CU4_Wb30 to cu_4 > pengine: info: rsc_merge_weights:cu_3: Rolling back scores from cu_2 > pengine: debug: native_assign_node:Assigning Redund_CU5_WB30 to cu_3 > > Looks like rolling back the scores is causing the new decision to > relocate the resources. > Am I using the scores incorrectly?
No, I think this is expected. Your anti-colocation constraints place cu_2 and cu_3 relative to cu_4, so that means the cluster will place cu_4 first if possible, before deciding where the others should go. Similarly, cu_2 has a constraint relative to cu_3, so cu_3 gets placed next, and cu_2 is the one left out. The anti-colocation scores of -INFINITY outweigh the stickiness of 100. I'm not sure whether setting stickiness to INFINITY would change anything; hopefully, it would stop cu_3 from moving, but cu_2 would still be stopped. I don't see a good way around this. The cluster has to place some resource first, in order to know not to place some other resource on the same node. I don't think there's a way to make them "equal", because then none of them could be placed to begin with -- unless you went with utilization attributes, as someone else suggested, with placement-strategy=balanced: http://clusterlabs.org/doc/en-US/Pacemaker/1.1-pcs/html-single/Pacemaker_Explained/index.html#idm140521708557280 > > [root@Redund_CU5_WB30 root]# pcs constraint > Location Constraints: > Resource: cu_2 > Enabled on: Redun_CU4_Wb30 (score:0) > Enabled on: Redund_CU5_WB30 (score:0) > Enabled on: Redund_CU3_WB30 (score:0) > Enabled on: Redund_CU1_WB30 (score:0) > Resource: cu_3 > Enabled on: Redun_CU4_Wb30 (score:0) > Enabled on: Redund_CU5_WB30 (score:0) > Enabled on: Redund_CU3_WB30 (score:0) > Enabled on: Redund_CU1_WB30 (score:0) > Resource: cu_4 > Enabled on: Redun_CU4_Wb30 (score:0) > Enabled on: Redund_CU5_WB30 (score:0) > Enabled on: Redund_CU3_WB30 (score:0) > Enabled on: Redund_CU1_WB30 (score:0) > Ordering Constraints: > Colocation Constraints: > cu_2 with cu_4 (score:-INFINITY) > cu_3 with cu_4 (score:-INFINITY) > cu_2 with cu_3 (score:-INFINITY) > > > On Mon, Oct 17, 2016 at 8:16 PM, Nikhil Utane > <nikhil.subscri...@gmail.com <mailto:nikhil.subscri...@gmail.com>> wrote: > > This is driving me insane. > > This is how the resources were started. Redund_CU1_WB30 was the DC > which I rebooted. > cu_4(ocf::redundancy:RedundancyRA):Started Redund_CU1_WB30 > cu_2(ocf::redundancy:RedundancyRA):Started Redund_CU5_WB30 > cu_3(ocf::redundancy:RedundancyRA):Started Redun_CU4_Wb30 > > Since the standby node was not UP. I was expecting resource cu_4 to > be waiting to be scheduled. > But then it re-arranged everything as below. > cu_4(ocf::redundancy:RedundancyRA):Started Redun_CU4_Wb30 > cu_2(ocf::redundancy:RedundancyRA):Stopped > cu_3(ocf::redundancy:RedundancyRA):Started Redund_CU5_WB30 > > There is not much information available in the logs on new DC. It > just shows what it has decided to do but nothing to suggest why it > did it that way. > > notice: Start cu_4(Redun_CU4_Wb30) > notice: Stop cu_2(Redund_CU5_WB30) > notice: Move cu_3(Started Redun_CU4_Wb30 -> Redund_CU5_WB30) > > I have default stickiness set to 100 which is higher than any score > that I have configured. > I have migration_threshold set to 1. Should I bump that up instead? > > -Thanks > Nikhil > > On Sat, Oct 15, 2016 at 12:36 AM, Ken Gaillot <kgail...@redhat.com > <mailto:kgail...@redhat.com>> wrote: > > On 10/14/2016 06:56 AM, Nikhil Utane wrote: > > Hi, > > > > Thank you for the responses so far. > > I added reverse colocation as well. However seeing some other issue > in > > resource movement that I am analyzing. > > > > Thinking further on this, why doesn't "/a not with b" does not > imply "b > > not with a"?/ > > Coz wouldn't putting "b with a" violate "a not with b"? > > > > Can someone confirm that colocation is required to be configured > both ways? > > The anti-colocation should only be defined one-way. Otherwise, > you get a > dependency loop (as seen in logs you showed elsewhere). > > The one-way constraint is enough to keep the resources apart. > However, > the question is whether the cluster might move resources around > unnecessarily. > > For example, "A not with B" means that the cluster will place B > first, > then place A somewhere else. So, if B's node fails, can the cluster > decide that A's node is now the best place for B, and move A to > a free > node, rather than simply start B on the free node? > > The cluster does take dependencies into account when placing a > resource, > so I would hope that wouldn't happen. But I'm not sure. Having some > stickiness might help, so that A has some preference against moving. > > > -Thanks > > Nikhil > > > > / > > / > > > > On Fri, Oct 14, 2016 at 1:09 PM, Vladislav Bogdanov > > <bub...@hoster-ok.com <mailto:bub...@hoster-ok.com> > <mailto:bub...@hoster-ok.com <mailto:bub...@hoster-ok.com>>> wrote: > > > > On October 14, 2016 10:13:17 AM GMT+03:00, Ulrich Windl > > <ulrich.wi...@rz.uni-regensburg.de > <mailto:ulrich.wi...@rz.uni-regensburg.de> > > <mailto:ulrich.wi...@rz.uni-regensburg.de > <mailto:ulrich.wi...@rz.uni-regensburg.de>>> wrote: > > >>>> Nikhil Utane <nikhil.subscri...@gmail.com > <mailto:nikhil.subscri...@gmail.com> > > <mailto:nikhil.subscri...@gmail.com > <mailto:nikhil.subscri...@gmail.com>>> schrieb am 13.10.2016 um > > >16:43 in > > >Nachricht > > > ><cagnwmjubpucnbgxrohkhsbq0lxovwslfpkupg1r8gjqrfqm...@mail.gmail.com > > <mailto:cagnwmjubpucnbgxrohkhsbq0lxovwslfpkupg1r8gjqrfqm...@mail.gmail.com> > > > > <mailto:cagnwmjubpucnbgxrohkhsbq0lxovwslfpkupg1r8gjqrfqm...@mail.gmail.com > > <mailto:cagnwmjubpucnbgxrohkhsbq0lxovwslfpkupg1r8gjqrfqm...@mail.gmail.com>>>: > > >> Ulrich, > > >> > > >> I have 4 resources only (not 5, nodes are 5). So then I only > need 6 > > >> constraints, right? > > >> > > >> [,1] [,2] [,3] [,4] [,5] [,6] > > >> [1,] "A" "A" "A" "B" "B" "C" > > >> [2,] "B" "C" "D" "C" "D" "D" > > > > > >Sorry for my confusion. As Andrei Borzenkovsaid in > > > ><caa91j0w+epahflg9u6vx_x8lgfkf9rp55g3nocy4ozna9bb...@mail.gmail.com > > <mailto:caa91j0w%2bepahflg9u6vx_x8lgfkf9rp55g3nocy4ozna9bbz%...@mail.gmail.com> > > > > <mailto:caa91j0w%2bepahflg9u6vx_x8lgfkf9rp55g3nocy4ozna9bbz%...@mail.gmail.com > > <mailto:caa91j0w%252bepahflg9u6vx_x8lgfkf9rp55g3nocy4ozna9bbz%25...@mail.gmail.com>>> > > >you probably have to add (A, B) _and_ (B, A)! Thinking about > it, I > > >wonder whether an easier solution would be using > "utilization": If > > >every node has one token to give, and every resource needs on > token, no > > >two resources will run on one node. Sounds like an easier > solution to > > >me. > > > > > >Regards, > > >Ulrich > > > > > > > > >> > > >> I understand that if I configure constraint of R1 with R2 > score as > > >> -infinity, then the same applies for R2 with R1 score as > -infinity > > >(don't > > >> have to configure it explicitly). > > >> I am not having a problem of multiple resources getting > schedule on > > >the > > >> same node. Rather, one working resource is unnecessarily > getting > > >relocated. > > >> > > >> -Thanks > > >> Nikhil > > >> > > >> > > >> On Thu, Oct 13, 2016 at 7:45 PM, Ulrich Windl < > > >> ulrich.wi...@rz.uni-regensburg.de > <mailto:ulrich.wi...@rz.uni-regensburg.de> > > <mailto:ulrich.wi...@rz.uni-regensburg.de > <mailto:ulrich.wi...@rz.uni-regensburg.de>>> wrote: > > >> > > >>> Hi! > > >>> > > >>> Don't you need 10 constraints, excluding every possible > pair of your > > >5 > > >>> resources (named A-E here), like in this table (produced > with R): > > >>> > > >>> [,1] [,2] [,3] [,4] [,5] [,6] [,7] [,8] [,9] [,10] > > >>> [1,] "A" "A" "A" "A" "B" "B" "B" "C" "C" "D" > > >>> [2,] "B" "C" "D" "E" "C" "D" "E" "D" "E" "E" > > >>> > > >>> Ulrich > > >>> > > >>> >>> Nikhil Utane <nikhil.subscri...@gmail.com > <mailto:nikhil.subscri...@gmail.com> > > <mailto:nikhil.subscri...@gmail.com > <mailto:nikhil.subscri...@gmail.com>>> schrieb am 13.10.2016 > > >um > > >>> 15:59 in > > >>> Nachricht > > >>> > > > ><cagnwmjw0cwmr3bvr3l9xzcacjuzyczqbzezuzpajxi+pn7o...@mail.gmail.com > > <mailto:cagnwmjw0cwmr3bvr3l9xzcacjuzyczqbzezuzpajxi%2bpn7o...@mail.gmail.com> > > > > <mailto:cagnwmjw0cwmr3bvr3l9xzcacjuzyczqbzezuzpajxi%2bpn7o...@mail.gmail.com > > <mailto:cagnwmjw0cwmr3bvr3l9xzcacjuzyczqbzezuzpajxi%252bpn7o...@mail.gmail.com>>>: > > >>> > Hi, > > >>> > > > >>> > I have 5 nodes and 4 resources configured. > > >>> > I have configured constraint such that no two > resources can be > > >>> co-located. > > >>> > I brought down a node (which happened to be DC). I > was expecting > > >the > > >>> > resource on the failed node would be migrated to the > 5th waiting > > >node > > >>> (that > > >>> > is not running any resource). > > >>> > However what happened was the failed node resource > was started on > > >another > > >>> > active node (after stopping it's existing resource) > and that > > >node's > > >>> > resource was moved to the waiting node. > > >>> > > > >>> > What could I be doing wrong? > > >>> > > > >>> > <nvpair id="cib-bootstrap-options-have-watchdog" > value="true" > > >>> > name="have-watchdog"/> > > >>> > <nvpair id="cib-bootstrap-options-dc-version" > > >value="1.1.14-5a6cdd1" > > >>> > name="dc-version"/> > > >>> > <nvpair > id="cib-bootstrap-options-cluster-infrastructure" > > >>> value="corosync" > > >>> > name="cluster-infrastructure"/> > > >>> > <nvpair id="cib-bootstrap-options-stonith-enabled" > value="false" > > >>> > name="stonith-enabled"/> > > >>> > <nvpair id="cib-bootstrap-options-no-quorum-policy" > value="ignore" > > >>> > name="no-quorum-policy"/> > > >>> > <nvpair > id="cib-bootstrap-options-default-action-timeout" > > >value="240" > > >>> > name="default-action-timeout"/> > > >>> > <nvpair id="cib-bootstrap-options-symmetric-cluster" > value="false" > > >>> > name="symmetric-cluster"/> > > >>> > > > >>> > # pcs constraint > > >>> > Location Constraints: > > >>> > Resource: cu_2 > > >>> > Enabled on: Redun_CU4_Wb30 (score:0) > > >>> > Enabled on: Redund_CU2_WB30 (score:0) > > >>> > Enabled on: Redund_CU3_WB30 (score:0) > > >>> > Enabled on: Redund_CU5_WB30 (score:0) > > >>> > Enabled on: Redund_CU1_WB30 (score:0) > > >>> > Resource: cu_3 > > >>> > Enabled on: Redun_CU4_Wb30 (score:0) > > >>> > Enabled on: Redund_CU2_WB30 (score:0) > > >>> > Enabled on: Redund_CU3_WB30 (score:0) > > >>> > Enabled on: Redund_CU5_WB30 (score:0) > > >>> > Enabled on: Redund_CU1_WB30 (score:0) > > >>> > Resource: cu_4 > > >>> > Enabled on: Redun_CU4_Wb30 (score:0) > > >>> > Enabled on: Redund_CU2_WB30 (score:0) > > >>> > Enabled on: Redund_CU3_WB30 (score:0) > > >>> > Enabled on: Redund_CU5_WB30 (score:0) > > >>> > Enabled on: Redund_CU1_WB30 (score:0) > > >>> > Resource: cu_5 > > >>> > Enabled on: Redun_CU4_Wb30 (score:0) > > >>> > Enabled on: Redund_CU2_WB30 (score:0) > > >>> > Enabled on: Redund_CU3_WB30 (score:0) > > >>> > Enabled on: Redund_CU5_WB30 (score:0) > > >>> > Enabled on: Redund_CU1_WB30 (score:0) > > >>> > Ordering Constraints: > > >>> > Colocation Constraints: > > >>> > cu_3 with cu_2 (score:-INFINITY) > > >>> > cu_4 with cu_2 (score:-INFINITY) > > >>> > cu_4 with cu_3 (score:-INFINITY) > > >>> > cu_5 with cu_2 (score:-INFINITY) > > >>> > cu_5 with cu_3 (score:-INFINITY) > > >>> > cu_5 with cu_4 (score:-INFINITY) > > >>> > > > >>> > -Thanks > > >>> > Nikhil > > >>> > > >>> > > >>> > > > > Hi, > > > > use of utilization (balanced strategy) has one caveat: > resources are > > not moved just because of utilization of one node is less, > when > > nodes have the same allocation score for the resource. > > So, after the simultaneus outage of two nodes in a 5-node > cluster, > > it may appear that one node runs two resources and two > recovered > > nodes run nothing. > > > > Original 'utilization' strategy only limits resource > placement, it > > is not considered when choosing a node for a resource. > > > > Vladislav _______________________________________________ Users mailing list: Users@clusterlabs.org http://clusterlabs.org/mailman/listinfo/users Project Home: http://www.clusterlabs.org Getting started: http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf Bugs: http://bugs.clusterlabs.org
