[ClusterLabs] [Problem] Thel fail-over is completed without the stop of the resource being carried out.

2016-09-26 Thread renayama19661014
Hi All,

We discovered a problem in the cluster which Quorum control and STONITH did not 
have.

We can confirm the problem in the next procedure.

Step1) Constitute a cluster.

[root@rh72-01 ~]# crm configure load update trac3437.crm 

[root@rh72-01 ~]# crm_mon -1 -Af
Stack: corosync
Current DC: rh72-01 (version 1.1.15-e174ec8) - partition with quorum
Last updated: Mon Sep 26 13:00:22 2016  Last change: Mon Sep 26 
12:59:52 2016 by root via cibadmin on rh72-01

2 nodes and 1 resource configured

Online: [ rh72-01 rh72-02 ]

Resource Group: grpDummy
prmDummy   (ocf::pacemaker:Dummy): Started rh72-01

Node Attributes:
* Node rh72-01:
* Node rh72-02:

Migration Summary:
* Node rh72-01:
* Node rh72-02:


Step2) Edit Dummy resource to cause stop trouble.

(snip)
dummy_stop() {
return $OCF_ERR_GENERIC
dummy_monitorif [ $? -eq $OCF_SUCCESS ]; thenrm ${OCF_RESKEY_state} 
   fi
rm -f "${VERIFY_SERIALIZED_FILE}"
return $OCF_SUCCESS
}
(snip)

Step3) Stop Pacemaker of the node. Stop trouble happens.
[root@rh72-01 ~]# systemctl stop pacemaker

[root@rh72-01 ~]# crm_mon -1 -Af
Stack: corosync
Current DC: rh72-01 (version 1.1.15-e174ec8) - partition with quorum
Last updated: Mon Sep 26 13:01:33 2016  Last change: Mon Sep 26 
12:59:52 2016 by root via cibadmin on rh72-01

2 nodes and 1 resource configured

Online: [ rh72-01 rh72-02 ]

Resource Group: grpDummy
prmDummy   (ocf::pacemaker:Dummy): FAILED rh72-01 (blocked)

Node Attributes:
* Node rh72-01:
* Node rh72-02:

Migration Summary:
* Node rh72-01:
prmDummy: migration-threshold=1 fail-count=100 last-failure='Mon Sep 26 
13:01:18 2016'
* Node rh72-02:

Failed Actions:
* prmDummy_stop_0 on rh72-01 'unknown error' (1): call=8, status=complete, 
exitreason='none',
last-rc-change='Mon Sep 26 13:01:18 2016', queued=0ms, exec=33ms

Step4) Correct Dummy resource in the original.
(snip)
dummy_stop() {
dummy_monitor
if [ $? -eq $OCF_SUCCESS ]; then
rm ${OCF_RESKEY_state}
fi
rm -f "${VERIFY_SERIALIZED_FILE}"
return $OCF_SUCCESS
}
(snip)

Step5) Clean up does the trouble of the Dummy resource.

[root@rh72-01 ~]# crm_resource -C -r prmDummy -H rh72-01 -f
Cleaning up prmDummy on rh72-01, removing fail-count-prmDummy
Waiting for 1 replies from the CRMd. OK

Step6) Fail-over is completed. However, the stop of the Dummy resource is not 
carried out in rh72-01 node.

[root@rh72-02 ~]# crm_mon -1 -Af
Stack: corosync
Current DC: rh72-02 (version 1.1.15-e174ec8) - partition WITHOUT quorum
Last updated: Mon Sep 26 13:02:32 2016  Last change: Mon Sep 26 
13:02:20 2016 by hacluster via crmd on rh72-01

2 nodes and 1 resource configured

Online: [ rh72-02 ]
OFFLINE: [ rh72-01 ]

Resource Group: grpDummy
prmDummy   (ocf::pacemaker:Dummy): Started rh72-02

Node Attributes:
* Node rh72-02:

Migration Summary:
* Node rh72-02:

[root@rh72-01 ~]# ls -lt /var/run/Dummy-prmDummy.state 
-rw-r-. 1 root root 0  9月 26  2016 /var/run/Dummy-prmDummy.state
-
Sep 26 13:02:21 rh72-01 crmd[1584]: warning: Action 2 (prmDummy_monitor_0) on 
rh72-01 failed (target: 7 vs. rc: 0): Error
Sep 26 13:02:21 rh72-01 crmd[1584]: notice: Transition aborted by operation 
prmDummy_monitor_0 'create' on rh72-01: Event failed | 
magic=0:0;2:6:7:196faae4-4faf-42a5-9ffb-9dcf6272e3fb cib=0.6.2 
source=match_graph_event:310 complete=false
Sep 26 13:02:21 rh72-01 crmd[1584]: info: Action prmDummy_monitor_0 (2) 
confirmed on rh72-01 (rc=0)
Sep 26 13:02:21 rh72-01 crmd[1584]: info: Detected action (6.2) 
prmDummy_monitor_0.13=ok: failed
Sep 26 13:02:21 rh72-01 crmd[1584]: warning: Action 2 (prmDummy_monitor_0) on 
rh72-01 failed (target: 7 vs. rc: 0): Error
Sep 26 13:02:21 rh72-01 crmd[1584]: info: Transition aborted by operation 
prmDummy_monitor_0 'create' on rh72-01: Event failed | 
magic=0:0;2:6:7:196faae4-4faf-42a5-9ffb-9dcf6272e3fb cib=0.6.2 
source=match_graph_event:310 complete=false
Sep 26 13:02:21 rh72-01 crmd[1584]: info: Action prmDummy_monitor_0 (2) 
confirmed on rh72-01 (rc=0)
Sep 26 13:02:21 rh72-01 crmd[1584]: info: Detected action (6.2) 
prmDummy_monitor_0.13=ok: failed
Sep 26 13:02:21 rh72-01 crmd[1584]: notice: Transition 6 (Complete=3, 
Pending=0, Fired=0, Skipped=0, Incomplete=3, 
Source=/var/lib/pacemaker/pengine/pe-input-6.bz2): Complete
Sep 26 13:02:21 rh72-01 crmd[1584]: info: Input I_STOP received in state 
S_TRANSITION_ENGINE from notify_crmd
Sep 26 13:02:21 rh72-01 crmd[1584]: info: State transition S_TRANSITION_ENGINE 
-> S_STOPPING | input=I_STOP cause=C_FSA_INTERNAL origin=notify_crmd
Sep 26 13:02:21 rh72-01 crmd[1584]: info: DC role released
Sep 26 13:02:21 rh72-01 crmd[1584]: info: Connection to the Policy Engine 
released
Sep 26 13:02:21 rh72-01 cib[1579]: info: Forwarding cib_modify operation for 
section status to all (origin=local/crmd/56)
Sep 26 13:02:21 rh72-01 cib[1579]: info: Diff: --- 0.6.2 2
Sep 26 13:02:21 rh72-01 cib[1579]: info: Diff: +++ 0.6.3 (null)
Sep 26 13:02:21 rh72-01 cib[1579]: info: +  /cib:  @num_updates=3Sep 26 
13:02:21 

Re: [ClusterLabs] Is it possible to sign up for cluster events from Pacemaker?

2016-09-26 Thread Kostiantyn Ponomarenko
Yes, DBus would be one of the ways.

Thank you,
Kostia

On Mon, Sep 26, 2016 at 3:33 PM, Klaus Wenninger 
wrote:

> On 09/26/2016 02:29 PM, Kostiantyn Ponomarenko wrote:
>
> Correcting a typo.
> * the same -> I also was hoping to hear that I can do the same from c++
> code.
>
>
> There is no direct-API or so just the Agent-Interface.
> But of course the Agent can be a binary compiled from c++ code
> that communicates via some IPC with whoever.
>
> Where you thinking of e.g. a DBus interface?
>
> Klaus
>
>
> Thank you,
> Kostia
>
> On Mon, Sep 26, 2016 at 3:28 PM, Kostiantyn Ponomarenko <
> konstantin.ponomare...@gmail.com> wrote:
>
>> Thanks for the answer.
>>
>> I also was hoping to hear that I can do the case from c++ code.
>>
>> Thank you,
>> Kostia
>>
>> On Mon, Sep 26, 2016 at 1:59 PM, Klaus Wenninger 
>> wrote:
>>
>>> On 09/26/2016 12:29 PM, Kostiantyn Ponomarenko wrote:
>>> > Hi,
>>> >
>>> > I am wondering if it is possible to sing up for cluster events from
>>> > Pacemaker? Something like:
>>> >  - a node joins/leaves the cluster,
>>> >  - a resource fails,
>>> >  - a resources moves,
>>> >  - etc.
>>> Sounds like a use case for the alerts-feature coming with
>>> pacemaker-1.1.15.
>>> You can configure alert-agents to be called in the cases mentioned.
>>> Meanwhile there should be high-level-tooling support from crmsh & pcs.
>>> Legacy features for a similar purpose are ClusterMon-RA &
>>> cluster-properties
>>> notification-agent/recipient.
>>> For further reading see:
>>> http://clusterlabs.org/doc/en-US/Pacemaker/1.1/html/Pacemake
>>> r_Explained/ch07.html
>>>
>>> >
>>> > Thank you,
>>> > Kostia
>>> >
>>> >
>>> > ___
>>> > Users mailing list: Users@clusterlabs.org
>>> > http://clusterlabs.org/mailman/listinfo/users
>>> >
>>> > Project Home: http://www.clusterlabs.org
>>> > Getting started: http://www.clusterlabs.org/doc
>>> /Cluster_from_Scratch.pdf
>>> > Bugs: http://bugs.clusterlabs.org
>>>
>>>
>>>
>>> ___
>>> Users mailing list: Users@clusterlabs.org
>>> http://clusterlabs.org/mailman/listinfo/users
>>>
>>> Project Home: http://www.clusterlabs.org
>>> Getting started: http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf
>>> Bugs: http://bugs.clusterlabs.org
>>>
>>
>>
>
>
___
Users mailing list: Users@clusterlabs.org
http://clusterlabs.org/mailman/listinfo/users

Project Home: http://www.clusterlabs.org
Getting started: http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf
Bugs: http://bugs.clusterlabs.org


Re: [ClusterLabs] Is it possible to sign up for cluster events from Pacemaker?

2016-09-26 Thread Klaus Wenninger
On 09/26/2016 02:29 PM, Kostiantyn Ponomarenko wrote:
> Correcting a typo.
> * the same -> I also was hoping to hear that I can do the same from
> c++ code.

There is no direct-API or so just the Agent-Interface.
But of course the Agent can be a binary compiled from c++ code
that communicates via some IPC with whoever.

Where you thinking of e.g. a DBus interface?

Klaus
>
> Thank you,
> Kostia
>
> On Mon, Sep 26, 2016 at 3:28 PM, Kostiantyn Ponomarenko
>  > wrote:
>
> Thanks for the answer.
>
> I also was hoping to hear that I can do the case from c++ code.
>
> Thank you,
> Kostia
>
> On Mon, Sep 26, 2016 at 1:59 PM, Klaus Wenninger
> > wrote:
>
> On 09/26/2016 12:29 PM, Kostiantyn Ponomarenko wrote:
> > Hi,
> >
> > I am wondering if it is possible to sing up for cluster
> events from
> > Pacemaker? Something like:
> >  - a node joins/leaves the cluster,
> >  - a resource fails,
> >  - a resources moves,
> >  - etc.
> Sounds like a use case for the alerts-feature coming with
> pacemaker-1.1.15.
> You can configure alert-agents to be called in the cases
> mentioned.
> Meanwhile there should be high-level-tooling support from
> crmsh & pcs.
> Legacy features for a similar purpose are ClusterMon-RA &
> cluster-properties
> notification-agent/recipient.
> For further reading see:
> 
> http://clusterlabs.org/doc/en-US/Pacemaker/1.1/html/Pacemaker_Explained/ch07.html
> 
> 
>
> >
> > Thank you,
> > Kostia
> >
> >
> > ___
> > Users mailing list: Users@clusterlabs.org
> 
> > http://clusterlabs.org/mailman/listinfo/users
> 
> >
> > Project Home: http://www.clusterlabs.org
> > Getting started:
> http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf
> 
> > Bugs: http://bugs.clusterlabs.org
>
>
>
> ___
> Users mailing list: Users@clusterlabs.org
> 
> http://clusterlabs.org/mailman/listinfo/users
> 
>
> Project Home: http://www.clusterlabs.org
> Getting started:
> http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf
> 
> Bugs: http://bugs.clusterlabs.org
>
>
>

___
Users mailing list: Users@clusterlabs.org
http://clusterlabs.org/mailman/listinfo/users

Project Home: http://www.clusterlabs.org
Getting started: http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf
Bugs: http://bugs.clusterlabs.org


Re: [ClusterLabs] Is it possible to sign up for cluster events from Pacemaker?

2016-09-26 Thread Kostiantyn Ponomarenko
Correcting a typo.
* the same -> I also was hoping to hear that I can do the same from c++
code.

Thank you,
Kostia

On Mon, Sep 26, 2016 at 3:28 PM, Kostiantyn Ponomarenko <
konstantin.ponomare...@gmail.com> wrote:

> Thanks for the answer.
>
> I also was hoping to hear that I can do the case from c++ code.
>
> Thank you,
> Kostia
>
> On Mon, Sep 26, 2016 at 1:59 PM, Klaus Wenninger 
> wrote:
>
>> On 09/26/2016 12:29 PM, Kostiantyn Ponomarenko wrote:
>> > Hi,
>> >
>> > I am wondering if it is possible to sing up for cluster events from
>> > Pacemaker? Something like:
>> >  - a node joins/leaves the cluster,
>> >  - a resource fails,
>> >  - a resources moves,
>> >  - etc.
>> Sounds like a use case for the alerts-feature coming with
>> pacemaker-1.1.15.
>> You can configure alert-agents to be called in the cases mentioned.
>> Meanwhile there should be high-level-tooling support from crmsh & pcs.
>> Legacy features for a similar purpose are ClusterMon-RA &
>> cluster-properties
>> notification-agent/recipient.
>> For further reading see:
>> http://clusterlabs.org/doc/en-US/Pacemaker/1.1/html/Pacemake
>> r_Explained/ch07.html
>>
>> >
>> > Thank you,
>> > Kostia
>> >
>> >
>> > ___
>> > Users mailing list: Users@clusterlabs.org
>> > http://clusterlabs.org/mailman/listinfo/users
>> >
>> > Project Home: http://www.clusterlabs.org
>> > Getting started: http://www.clusterlabs.org/doc
>> /Cluster_from_Scratch.pdf
>> > Bugs: http://bugs.clusterlabs.org
>>
>>
>>
>> ___
>> Users mailing list: Users@clusterlabs.org
>> http://clusterlabs.org/mailman/listinfo/users
>>
>> Project Home: http://www.clusterlabs.org
>> Getting started: http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf
>> Bugs: http://bugs.clusterlabs.org
>>
>
>
___
Users mailing list: Users@clusterlabs.org
http://clusterlabs.org/mailman/listinfo/users

Project Home: http://www.clusterlabs.org
Getting started: http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf
Bugs: http://bugs.clusterlabs.org


Re: [ClusterLabs] Is it possible to sign up for cluster events from Pacemaker?

2016-09-26 Thread Kostiantyn Ponomarenko
Thanks for the answer.

I also was hoping to hear that I can do the case from c++ code.

Thank you,
Kostia

On Mon, Sep 26, 2016 at 1:59 PM, Klaus Wenninger 
wrote:

> On 09/26/2016 12:29 PM, Kostiantyn Ponomarenko wrote:
> > Hi,
> >
> > I am wondering if it is possible to sing up for cluster events from
> > Pacemaker? Something like:
> >  - a node joins/leaves the cluster,
> >  - a resource fails,
> >  - a resources moves,
> >  - etc.
> Sounds like a use case for the alerts-feature coming with pacemaker-1.1.15.
> You can configure alert-agents to be called in the cases mentioned.
> Meanwhile there should be high-level-tooling support from crmsh & pcs.
> Legacy features for a similar purpose are ClusterMon-RA &
> cluster-properties
> notification-agent/recipient.
> For further reading see:
> http://clusterlabs.org/doc/en-US/Pacemaker/1.1/html/
> Pacemaker_Explained/ch07.html
>
> >
> > Thank you,
> > Kostia
> >
> >
> > ___
> > Users mailing list: Users@clusterlabs.org
> > http://clusterlabs.org/mailman/listinfo/users
> >
> > Project Home: http://www.clusterlabs.org
> > Getting started: http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf
> > Bugs: http://bugs.clusterlabs.org
>
>
>
> ___
> Users mailing list: Users@clusterlabs.org
> http://clusterlabs.org/mailman/listinfo/users
>
> Project Home: http://www.clusterlabs.org
> Getting started: http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf
> Bugs: http://bugs.clusterlabs.org
>
___
Users mailing list: Users@clusterlabs.org
http://clusterlabs.org/mailman/listinfo/users

Project Home: http://www.clusterlabs.org
Getting started: http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf
Bugs: http://bugs.clusterlabs.org


Re: [ClusterLabs] Is it possible to sign up for cluster events from Pacemaker?

2016-09-26 Thread Klaus Wenninger
On 09/26/2016 12:29 PM, Kostiantyn Ponomarenko wrote:
> Hi,
>
> I am wondering if it is possible to sing up for cluster events from
> Pacemaker? Something like:
>  - a node joins/leaves the cluster,
>  - a resource fails,
>  - a resources moves,
>  - etc.
Sounds like a use case for the alerts-feature coming with pacemaker-1.1.15.
You can configure alert-agents to be called in the cases mentioned.
Meanwhile there should be high-level-tooling support from crmsh & pcs.
Legacy features for a similar purpose are ClusterMon-RA & cluster-properties
notification-agent/recipient.
For further reading see:
http://clusterlabs.org/doc/en-US/Pacemaker/1.1/html/Pacemaker_Explained/ch07.html

>
> Thank you,
> Kostia
>
>
> ___
> Users mailing list: Users@clusterlabs.org
> http://clusterlabs.org/mailman/listinfo/users
>
> Project Home: http://www.clusterlabs.org
> Getting started: http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf
> Bugs: http://bugs.clusterlabs.org



___
Users mailing list: Users@clusterlabs.org
http://clusterlabs.org/mailman/listinfo/users

Project Home: http://www.clusterlabs.org
Getting started: http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf
Bugs: http://bugs.clusterlabs.org