Re: [ClusterLabs Developers] How to implement fencing agent with no associated hardware device with Pacemaker?

2020-08-04 Thread Digimer 
You'll need an agent that complies with the FenceAgentAPI. All pacemaker
cares about, fundamentally, is the agent's exit code (save for the
meta-data call it uses to know what is a valid config). It's entirely up
to you when you decide that a fence action has succeeded or failed.

Of course, reporting success without actually fencing the target could
end disastrously, so do your due diligence.

https://github.com/ClusterLabs/fence-agents/blob/master/doc/FenceAgentAPI.md

digimer

On 2020-08-04 4:57 p.m., Philippe M Stedman wrote:
> Hi Andrei,
> 
> What if we had no hardware device and all we wanted to do was run a
> single command when fencing off a failed node. Would that be possible?
> 
> Are fencing operations dependent on underlying hardware for Pacemaker?
> 
> Thanks,
> 
> *Phil Stedman*
> Db2 High Availability Development and Support
> Email: pmste...@us.ibm.com
> 
> Inactive hide details for Andrei Borzenkov ---07/30/2020 11:02:58
> PM---30.07.2020 22:17, Philippe M Stedman пишет: >Andrei Borzenkov
> ---07/30/2020 11:02:58 PM---30.07.2020 22:17, Philippe M Stedman пишет: >
> 
> From: Andrei Borzenkov 
> To: developers@clusterlabs.org, Philippe M Stedman
> , nw...@redhat.com
> Cc: Toby Haynes , Gerry R Sommerville
> , Alan Y Lee 
> Date: 07/30/2020 11:02 PM
> Subject: [EXTERNAL] Re: [ClusterLabs Developers] How to implement
> fencing agent with no associated hardware device with Pacemaker?
> 
> 
> 
> 
> 
> 30.07.2020 22:17, Philippe M Stedman пишет:
>>
>> Thanks Gerry.
>>
>> Hi Reid,
>>
>> The shared storage solution we are using has clustering capabilities
> of its
>> own built into it and is able to remotely fence off the lost node, all we
>> need to do is run the command to expel/fence the lost node as part of our
>> own custom fencing agent on the surviving node.
>>
>> FYI, the shared storage solution I am referring to here is IBM Spectrum
>> Scale.
>>
> 
> ...
> 
>>
>>   On Wed, Jul 29, 2020 at 8:34 PM Philippe M Stedman 
>>   wrote:
>>     Hi ClusterLabs developers,
>>
>>     I am looking into how to develop a fencing agent for Pacemaker that is
>>     not associated to any underlying hardware device. In our case we have
>>     two servers (we will expand to more in the future) which have
> access to
>>     shared storage. When one of the two nodes fails, we expect the
>>     surviving node to invoke our user-defined fencing agent and run a
>>     series of commands which will "expel" the lost host from accessing
>>     shared storage.
>>
>>     Do you have any advice on how to go about implementing such a
> solution?
>>     All the examples I can find online revolve around using some sort of
>>     underlying hardware device to implement fencing.
>>
> 
> In this case your fencing *is* associated with specific underlying
> hardware device. Just start with any of existing fencing or stonith
> agents and fill in commands to your underlying hardware to fence off
> node. You probably will need some sort of mapping from node name to
> whatever is required by your hardware to identify it.
> 
> 
> 
> 
> 
> ___
> Manage your subscription:
> https://lists.clusterlabs.org/mailman/listinfo/developers
> 
> ClusterLabs home: https://www.clusterlabs.org/
> 


-- 
Digimer
Papers and Projects: https://alteeve.com/w/
"I am, somehow, less interested in the weight and convolutions of
Einstein’s brain than in the near certainty that people of equal talent
have lived and died in cotton fields and sweatshops." - Stephen Jay Gould
___
Manage your subscription:
https://lists.clusterlabs.org/mailman/listinfo/developers

ClusterLabs home: https://www.clusterlabs.org/

Re: [ClusterLabs Developers] How to implement fencing agent with no associated hardware device with Pacemaker?

2020-08-04 Thread Philippe M Stedman 
Hi Andrei,

What if we had no hardware device and all we wanted to do was run a single
command when fencing off a failed node. Would that be possible?

Are fencing operations dependent on underlying hardware for Pacemaker?

Thanks,

Phil Stedman
Db2 High Availability Development and Support
Email: pmste...@us.ibm.com



From:   Andrei Borzenkov 
To: developers@clusterlabs.org, Philippe M Stedman
, nw...@redhat.com
Cc: Toby Haynes , Gerry R Sommerville
, Alan Y Lee 
Date:   07/30/2020 11:02 PM
Subject:[EXTERNAL] Re: [ClusterLabs Developers] How to implement
fencing agent with no associated hardware device with
Pacemaker?



30.07.2020 22:17, Philippe M Stedman пишет:
>
> Thanks Gerry.
>
> Hi Reid,
>
> The shared storage solution we are using has clustering capabilities of
its
> own built into it and is able to remotely fence off the lost node, all we
> need to do is run the command to expel/fence the lost node as part of our
> own custom fencing agent on the surviving node.
>
> FYI, the shared storage solution I am referring to here is IBM Spectrum
> Scale.
>

...

>
>   On Wed, Jul 29, 2020 at 8:34 PM Philippe M Stedman

>   wrote:
> Hi ClusterLabs developers,
>
> I am looking into how to develop a fencing agent for Pacemaker that
is
> not associated to any underlying hardware device. In our case we have
> two servers (we will expand to more in the future) which have access
to
> shared storage. When one of the two nodes fails, we expect the
> surviving node to invoke our user-defined fencing agent and run a
> series of commands which will "expel" the lost host from accessing
> shared storage.
>
> Do you have any advice on how to go about implementing such a
solution?
> All the examples I can find online revolve around using some sort of
> underlying hardware device to implement fencing.
>

In this case your fencing *is* associated with specific underlying
hardware device. Just start with any of existing fencing or stonith
agents and fill in commands to your underlying hardware to fence off
node. You probably will need some sort of mapping from node name to
whatever is required by your hardware to identify it.



___
Manage your subscription:
https://lists.clusterlabs.org/mailman/listinfo/developers

ClusterLabs home: https://www.clusterlabs.org/

Re: [ClusterLabs Developers] How to implement fencing agent with no associated hardware device with Pacemaker?

2020-07-31 Thread Andrei Borzenkov 
30.07.2020 22:17, Philippe M Stedman пишет:
> 
> Thanks Gerry.
> 
> Hi Reid,
> 
> The shared storage solution we are using has clustering capabilities of its
> own built into it and is able to remotely fence off the lost node, all we
> need to do is run the command to expel/fence the lost node as part of our
> own custom fencing agent on the surviving node.
> 
> FYI, the shared storage solution I am referring to here is IBM Spectrum
> Scale.
> 

...

> 
>   On Wed, Jul 29, 2020 at 8:34 PM Philippe M Stedman 
>   wrote:
> Hi ClusterLabs developers,
> 
> I am looking into how to develop a fencing agent for Pacemaker that is
> not associated to any underlying hardware device. In our case we have
> two servers (we will expand to more in the future) which have access to
> shared storage. When one of the two nodes fails, we expect the
> surviving node to invoke our user-defined fencing agent and run a
> series of commands which will "expel" the lost host from accessing
> shared storage.
> 
> Do you have any advice on how to go about implementing such a solution?
> All the examples I can find online revolve around using some sort of
> underlying hardware device to implement fencing.
> 

In this case your fencing *is* associated with specific underlying
hardware device. Just start with any of existing fencing or stonith
agents and fill in commands to your underlying hardware to fence off
node. You probably will need some sort of mapping from node name to
whatever is required by your hardware to identify it.

___
Manage your subscription:
https://lists.clusterlabs.org/mailman/listinfo/developers

ClusterLabs home: https://www.clusterlabs.org/

Re: [ClusterLabs Developers] How to implement fencing agent with no associated hardware device with Pacemaker?

2020-07-30 Thread Gerry R Sommerville 
Seems like we lost Phil in the first reply Adding him back. 
Gerry Sommerville
Db2 Development, pureScale Domain
E-mail: ge...@ca.ibm.com
 
 
- Original message -From: Reid Wahl To: developers@clusterlabs.orgCc: Toby Haynes , Gerry R Sommerville , Alan Y Lee Subject: [EXTERNAL] Re: [ClusterLabs Developers] How to implement fencing agent with no associated hardware device with Pacemaker?Date: Wed, Jul 29, 2020 11:42 PM 
I didn't see the phrase "how to develop" until after I sent the previous message. What is the reason for needing to develop a custom fencing agent? An already-built one might save you some work.
 
Basically, you need some reliable method to cut off an unhealthy node's access to shared storage, without depending on that node being responsive. So for example, anything that involves logging into the failed node is unreliable. 

On Wed, Jul 29, 2020 at 8:40 PM Reid Wahl  wrote:
If you have a hardware **watchdog timer**, then sbd is a good option. With shared storage, you can also implement fence_sbd.
 
KVM virtual machines also offer an emulated hardware watchdog. I'm not sure whether that would fit your criteria or not -- it depends on whether you're only excluding a management interface like an iLO/IMM, or whether you're also excluding a watchdog timer.
 
If you can't use sbd or conventional power fencing (e.g., fence_ipmilan), then you may be able to use fence_scsi or fence_mpath since you have shared storage.
 
What hardware or virtualization platform are you running on, and is there a particular reason you don't want to associate fencing with a hardware device? 

On Wed, Jul 29, 2020 at 8:34 PM Philippe M Stedman  wrote:
Hi ClusterLabs developers,I am looking into how to develop a fencing agent for Pacemaker that is not associated to any underlying hardware device. In our case we have two servers (we will expand to more in the future) which have access to shared storage. When one of the two nodes fails, we expect the surviving node to invoke our user-defined fencing agent and run a series of commands which will "expel" the lost host from accessing shared storage.Do you have any advice on how to go about implementing such a solution? All the examples I can find online revolve around using some sort of underlying hardware device to implement fencing.Help is greatly appreciated.Thanks,Phil StedmanDb2 High Availability Development and SupportEmail: pmste...@us.ibm.com___Manage your subscription:https://lists.clusterlabs.org/mailman/listinfo/developersClusterLabs home: https://www.clusterlabs.org/--
Regards, Reid Wahl, RHCA
Software Maintenance Engineer, Red HatCEE - Platform Support Delivery - ClusterHA--
Regards, Reid Wahl, RHCA
Software Maintenance Engineer, Red HatCEE - Platform Support Delivery - ClusterHA
 

___
Manage your subscription:
https://lists.clusterlabs.org/mailman/listinfo/developers

ClusterLabs home: https://www.clusterlabs.org/

Re: [ClusterLabs Developers] How to implement fencing agent with no associated hardware device with Pacemaker?

2020-07-29 Thread Reid Wahl 
I didn't see the phrase "how to develop" until after I sent the previous
message. What is the reason for needing to develop a custom fencing agent?
An already-built one might save you some work.

Basically, you need some reliable method to cut off an unhealthy node's
access to shared storage, without depending on that node being responsive.
So for example, anything that involves logging into the failed node is
unreliable.

On Wed, Jul 29, 2020 at 8:40 PM Reid Wahl  wrote:

> If you have a hardware **watchdog timer**, then sbd is a good option. With
> shared storage, you can also implement fence_sbd.
>
> KVM virtual machines also offer an emulated hardware watchdog. I'm not
> sure whether that would fit your criteria or not -- it depends on whether
> you're only excluding a management interface like an iLO/IMM, or whether
> you're also excluding a watchdog timer.
>
> If you can't use sbd or conventional power fencing (e.g., fence_ipmilan),
> then you may be able to use fence_scsi or fence_mpath since you have shared
> storage.
>
> What hardware or virtualization platform are you running on, and is there
> a particular reason you don't want to associate fencing with a hardware
> device?
>
> On Wed, Jul 29, 2020 at 8:34 PM Philippe M Stedman 
> wrote:
>
>> Hi ClusterLabs developers,
>>
>> I am looking into how to develop a fencing agent for Pacemaker that is
>> not associated to any underlying hardware device. In our case we have two
>> servers (we will expand to more in the future) which have access to shared
>> storage. When one of the two nodes fails, we expect the surviving node to
>> invoke our user-defined fencing agent and run a series of commands which
>> will "expel" the lost host from accessing shared storage.
>>
>> Do you have any advice on how to go about implementing such a solution?
>> All the examples I can find online revolve around using some sort of
>> underlying hardware device to implement fencing.
>>
>> Help is greatly appreciated.
>>
>> Thanks,
>>
>> *Phil Stedman*
>> Db2 High Availability Development and Support
>> Email: pmste...@us.ibm.com
>> ___
>> Manage your subscription:
>> https://lists.clusterlabs.org/mailman/listinfo/developers
>>
>> ClusterLabs home: https://www.clusterlabs.org/
>>
>
>
> --
> Regards,
>
> Reid Wahl, RHCA
> Software Maintenance Engineer, Red Hat
> CEE - Platform Support Delivery - ClusterHA
>


-- 
Regards,

Reid Wahl, RHCA
Software Maintenance Engineer, Red Hat
CEE - Platform Support Delivery - ClusterHA
___
Manage your subscription:
https://lists.clusterlabs.org/mailman/listinfo/developers

ClusterLabs home: https://www.clusterlabs.org/

Re: [ClusterLabs Developers] How to implement fencing agent with no associated hardware device with Pacemaker?

2020-07-29 Thread Reid Wahl 
If you have a hardware **watchdog timer**, then sbd is a good option. With
shared storage, you can also implement fence_sbd.

KVM virtual machines also offer an emulated hardware watchdog. I'm not sure
whether that would fit your criteria or not -- it depends on whether you're
only excluding a management interface like an iLO/IMM, or whether you're
also excluding a watchdog timer.

If you can't use sbd or conventional power fencing (e.g., fence_ipmilan),
then you may be able to use fence_scsi or fence_mpath since you have shared
storage.

What hardware or virtualization platform are you running on, and is there a
particular reason you don't want to associate fencing with a hardware
device?

On Wed, Jul 29, 2020 at 8:34 PM Philippe M Stedman 
wrote:

> Hi ClusterLabs developers,
>
> I am looking into how to develop a fencing agent for Pacemaker that is not
> associated to any underlying hardware device. In our case we have two
> servers (we will expand to more in the future) which have access to shared
> storage. When one of the two nodes fails, we expect the surviving node to
> invoke our user-defined fencing agent and run a series of commands which
> will "expel" the lost host from accessing shared storage.
>
> Do you have any advice on how to go about implementing such a solution?
> All the examples I can find online revolve around using some sort of
> underlying hardware device to implement fencing.
>
> Help is greatly appreciated.
>
> Thanks,
>
> *Phil Stedman*
> Db2 High Availability Development and Support
> Email: pmste...@us.ibm.com
> ___
> Manage your subscription:
> https://lists.clusterlabs.org/mailman/listinfo/developers
>
> ClusterLabs home: https://www.clusterlabs.org/
>


-- 
Regards,

Reid Wahl, RHCA
Software Maintenance Engineer, Red Hat
CEE - Platform Support Delivery - ClusterHA
___
Manage your subscription:
https://lists.clusterlabs.org/mailman/listinfo/developers

ClusterLabs home: https://www.clusterlabs.org/