Hi Steffen, > Unfortunately not sure about the exact scenario. But I have been doing > some recent experiments with node standby/unstandby stop/start. This > to get procedures right for updating node rpms etc. > > Later I noticed the uncomforting "pending fencing actions" status msg.
Okay! Repeat the standby and unstandby steps in the same way to check. We will start checking after tomorrow, so I think it will take some time until next week. Many thanks, Hideo Yamauchi. ----- Original Message ----- > From: "renayama19661...@ybb.ne.jp" <renayama19661...@ybb.ne.jp> > To: Reid Wahl <nw...@redhat.com>; Cluster Labs - All topics related to > open-source clustering welcomed <users@clusterlabs.org> > Cc: > Date: 2021/1/7, Thu 17:51 > Subject: Re: [ClusterLabs] Pending Fencing Actions shown in pcs status > > Hi Steffen, > Hi Reid, > > The fencing history is kept inside stonith-ng and is not written to cib. > However, getting the entire cib and getting it sent will help you to > reproduce > the problem. > > Best Regards, > Hideo Yamauchi. > > > ----- Original Message ----- >> From: Reid Wahl <nw...@redhat.com> >> To: renayama19661...@ybb.ne.jp; Cluster Labs - All topics related to > open-source clustering welcomed <users@clusterlabs.org> >> Date: 2021/1/7, Thu 17:39 >> Subject: Re: [ClusterLabs] Pending Fencing Actions shown in pcs status >> >> >> Hi, Steffen. Those attachments don't contain the CIB. They contain the > `pcs config` output. You can get the cib with `pcs cluster cib > > $(hostname).cib.xml`. >> >> >> Granted, it's possible that this fence action information wouldn't > be in the CIB at all. It might be stored in fencer memory. >> >> >> On Thu, Jan 7, 2021 at 12:26 AM <renayama19661...@ybb.ne.jp> wrote: >> >> Hi Steffen, >>> >>>> Here CIB settings attached (pcs config show) for all 3 of my nodes >>>> (all 3 seems 100% identical), node03 is the DC. >>> >>> >>> Thank you for the attachment. >>> >>> What is the scenario when this situation occurs? >>> In what steps did the problem appear when fencing was performed (or > failed)? >>> >>> >>> Best Regards, >>> Hideo Yamauchi. >>> >>> >>> ----- Original Message ----- >>>> From: Steffen Vinther Sørensen <svint...@gmail.com> >>>> To: renayama19661...@ybb.ne.jp; Cluster Labs - All topics related > to open-source clustering welcomed <users@clusterlabs.org> >>>> Cc: >>>> Date: 2021/1/7, Thu 17:05 >>>> Subject: Re: [ClusterLabs] Pending Fencing Actions shown in pcs > status >>>> >>>> Hi Hideo, >>>> >>>> Here CIB settings attached (pcs config show) for all 3 of my nodes >>>> (all 3 seems 100% identical), node03 is the DC. >>>> >>>> Regards >>>> Steffen >>>> >>>> On Thu, Jan 7, 2021 at 8:06 AM <renayama19661...@ybb.ne.jp> > wrote: >>>>> >>>>> Hi Steffen, >>>>> Hi Reid, >>>>> >>>>> I also checked the Centos source rpm and it seems to include a > fix for the >>>> problem. >>>>> >>>>> As Steffen suggested, if you share your CIB settings, I might > know >>>> something. >>>>> >>>>> If this issue is the same as the fix, the display will only be > displayed on >>>> the DC node and will not affect the operation. >>>>> The pending actions shown will remain for a long time, but > will not have a >>>> negative impact on the cluster. >>>>> >>>>> Best Regards, >>>>> Hideo Yamauchi. >>>>> >>>>> >>>>> ----- Original Message ----- >>>>> > From: Reid Wahl <nw...@redhat.com> >>>>> > To: Cluster Labs - All topics related to open-source > clustering >>>> welcomed <users@clusterlabs.org> >>>>> > Cc: >>>>> > Date: 2021/1/7, Thu 15:58 >>>>> > Subject: Re: [ClusterLabs] Pending Fencing Actions shown > in pcs status >>>>> > >>>>> > It's supposedly fixed in that version. >>>>> > - https://bugzilla.redhat.com/show_bug.cgi?id=1787749 >>>>> > - https://access.redhat.com/solutions/4713471 >>>>> > >>>>> > So you may be hitting a different issue (unless > there's a bug in >>>> the >>>>> > pcmk 1.1 backport of the fix). >>>>> > >>>>> > I may be a little bit out of my area of knowledge here, > but can you >>>>> > share the CIBs from nodes 1 and 3? Maybe Hideo, Klaus, or > Ken has some >>>>> > insight. >>>>> > >>>>> > On Wed, Jan 6, 2021 at 10:53 PM Steffen Vinther Sørensen >>>>> > <svint...@gmail.com> wrote: >>>>> >> >>>>> >> Hi Hideo, >>>>> >> >>>>> >> If the fix is not going to make it into the CentOS7 > pacemaker >>>> version, >>>>> >> I guess the stable approach to take advantage of it > is to build >>>> the >>>>> >> cluster on another OS than CentOS7 ? A little late > for that in >>>> this >>>>> >> case though :) >>>>> >> >>>>> >> Regards >>>>> >> Steffen >>>>> >> >>>>> >> >>>>> >> >>>>> >> >>>>> >> On Thu, Jan 7, 2021 at 7:27 AM > <renayama19661...@ybb.ne.jp> >>>> wrote: >>>>> >> > >>>>> >> > Hi Steffen, >>>>> >> > >>>>> >> > The fix pointed out by Reid is affecting it. >>>>> >> > >>>>> >> > Since the fencing action requested by the DC > node exists >>>> only in the >>>>> > DC node, such an event occurs. >>>>> >> > You will need to take advantage of the modified > pacemaker to >>>> resolve >>>>> > the issue. >>>>> >> > >>>>> >> > Best Regards, >>>>> >> > Hideo Yamauchi. >>>>> >> > >>>>> >> > >>>>> >> > >>>>> >> > ----- Original Message ----- >>>>> >> > > From: Reid Wahl <nw...@redhat.com> >>>>> >> > > To: Cluster Labs - All topics related to > open-source >>>> clustering >>>>> > welcomed <users@clusterlabs.org> >>>>> >> > > Cc: >>>>> >> > > Date: 2021/1/7, Thu 15:07 >>>>> >> > > Subject: Re: [ClusterLabs] Pending Fencing > Actions >>>> shown in pcs >>>>> > status >>>>> >> > > >>>>> >> > > Hi, Steffen. Are your cluster nodes all > running the >>>> same >>>>> > Pacemaker >>>>> >> > > versions? This looks like Bug 5401[1], > which is fixed >>>> by upstream >>>>> >> > > commit df71a07[2]. I'm a little bit > confused about >>>> why it >>>>> > only shows >>>>> >> > > up on one out of three nodes though. >>>>> >> > > >>>>> >> > > [1] > https://bugs.clusterlabs.org/show_bug.cgi?id=5401 >>>>> >> > > [2] >>>> https://github.com/ClusterLabs/pacemaker/commit/df71a07 >>>>> >> > > >>>>> >> > > On Tue, Jan 5, 2021 at 8:31 AM Steffen > Vinther Sørensen >>>>> >> > > <svint...@gmail.com> wrote: >>>>> >> > >> >>>>> >> > >> Hello >>>>> >> > >> >>>>> >> > >> node 1 is showing this in 'pcs > status' >>>>> >> > >> >>>>> >> > >> Pending Fencing Actions: >>>>> >> > >> * reboot of > kvm03-node02.avigol-gcs.dk pending: >>>>> > client=crmd.37819, >>>>> >> > >> origin=kvm03-node03.avigol-gcs.dk >>>>> >> > >> >>>>> >> > >> node 2 and node 3 outputs no such > thing (node 3 is >>>> DC) >>>>> >> > >> >>>>> >> > >> Google is not much help, how to > investigate this >>>> further and >>>>> > get rid >>>>> >> > >> of such terrifying status message ? >>>>> >> > >> >>>>> >> > >> Regards >>>>> >> > >> Steffen >>>>> >> > >> > _______________________________________________ >>>>> >> > >> Manage your subscription: >>>>> >> > >> >>>> https://lists.clusterlabs.org/mailman/listinfo/users >>>>> >> > >> >>>>> >> > >> ClusterLabs home: > https://www.clusterlabs.org/ >>>>> >> > >> >>>>> >> > > >>>>> >> > > >>>>> >> > > -- >>>>> >> > > Regards, >>>>> >> > > >>>>> >> > > Reid Wahl, RHCA >>>>> >> > > Senior Software Maintenance Engineer, Red > Hat >>>>> >> > > CEE - Platform Support Delivery - > ClusterHA >>>>> >> > > >>>>> >> > > > _______________________________________________ >>>>> >> > > Manage your subscription: >>>>> >> > > > https://lists.clusterlabs.org/mailman/listinfo/users >>>>> >> > > >>>>> >> > > ClusterLabs home: > https://www.clusterlabs.org/ >>>>> >> > > >>>>> >> > >>>>> >> > _______________________________________________ >>>>> >> > Manage your subscription: >>>>> >> > > https://lists.clusterlabs.org/mailman/listinfo/users >>>>> >> > >>>>> >> > ClusterLabs home: https://www.clusterlabs.org/ >>>>> >> _______________________________________________ >>>>> >> Manage your subscription: >>>>> >> https://lists.clusterlabs.org/mailman/listinfo/users > >>>>> >> >>>>> >> ClusterLabs home: https://www.clusterlabs.org/ >>>>> > >>>>> > >>>>> > >>>>> > -- >>>>> > Regards, >>>>> > >>>>> > Reid Wahl, RHCA >>>>> > Senior Software Maintenance Engineer, Red Hat >>>>> > CEE - Platform Support Delivery - ClusterHA >>>>> > >>>>> > _______________________________________________ >>>>> > Manage your subscription: >>>>> > https://lists.clusterlabs.org/mailman/listinfo/users >>>>> > >>>>> > ClusterLabs home: https://www.clusterlabs.org/ >>>>> > >>>>> >>>>> _______________________________________________ >>>>> Manage your subscription: >>>>> https://lists.clusterlabs.org/mailman/listinfo/users >>>>> >>>>> ClusterLabs home: https://www.clusterlabs.org/ >>>> >>> >>> _______________________________________________ >>> Manage your subscription: >>> https://lists.clusterlabs.org/mailman/listinfo/users >>> >>> ClusterLabs home: https://www.clusterlabs.org/ >>> >> >> -- >> >> Regards, >> >> Reid Wahl, RHCA >> >> Senior Software Maintenance Engineer, Red Hat >> CEE - Platform Support Delivery - ClusterHA >> >> > > _______________________________________________ > Manage your subscription: > https://lists.clusterlabs.org/mailman/listinfo/users > > ClusterLabs home: https://www.clusterlabs.org/ > _______________________________________________ Manage your subscription: https://lists.clusterlabs.org/mailman/listinfo/users ClusterLabs home: https://www.clusterlabs.org/