Re: [ClusterLabs] Pacemaker in Azure

[Eric Robinson]

I agree completely. Are you offering to make those changes? Because they would 
expand the capability of resource angent and would be a welcome addition. Also, 
full disclosure, I need to have something in place by the weekend, lol.

From: Ken Gaillot 
Sent: Thursday, August 24, 2017 4:45:32 PM
To: Cluster Labs - All topics related to open-source clustering welcomed
Subject: Re: [ClusterLabs] Pacemaker in Azure

That would definitely be of wider interest.

I could see modifying the IPaddr2 RA to take some new arguments for
AWS/Azure parameters, and if those are configured, it would do the
appropriate API requests.

On Thu, 2017-08-24 at 23:27 +, Eric Robinson wrote:
> Leon -- I will pay you one trillion samolians for that resource agent!
> Any way we can get our hands on a copy?
>
>
>
> --
> Eric Robinson
>
>
>
> From: Leon Steffens [mailto:l...@steffensonline.com]
> Sent: Thursday, August 24, 2017 3:48 PM
> To: Cluster Labs - All topics related to open-source clustering
> welcomed 
> Subject: Re: [ClusterLabs] Pacemaker in Azure
>
>
>
> That's what we did in AWS.  The IPaddr2 resource agent does an arp
> broadcast after changing the local IP but this does not work in AWS
> (probably for the same reasons as Azure).
>
>
>
>
> We created our own OCF resource agent that uses the Amazon APIs to
> move the IP in AWS land and made that dependent on the IPaddr2
> resource, and it worked fine.
>
>
>
>
>
>
>
>
> Leon Steffens
>
>
>
>
> On Fri, Aug 25, 2017 at 8:34 AM, Eric Robinson
>  wrote:
>
> > Don't use Azure? ;)
>
> That would be my preference. But since I'm stuck with Azure
> (management decision) I need to come up with something. It
> appears there is an Azure API to make changes on-the-fly from
> a Linux box. Maybe I'll write a resource agent to change Azure
> and make IPaddr2 dependent on it. That might work?
>
> --
> Eric Robinson
>
>
> ___
> Users mailing list: Users@clusterlabs.org
> http://lists.clusterlabs.org/mailman/listinfo/users
>
> Project Home: http://www.clusterlabs.org
> Getting started:
> http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf
> Bugs: http://bugs.clusterlabs.org
>
>
>
>
> ___
> Users mailing list: Users@clusterlabs.org
> http://lists.clusterlabs.org/mailman/listinfo/users
>
> Project Home: http://www.clusterlabs.org
> Getting started: http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf
> Bugs: http://bugs.clusterlabs.org

--
Ken Gaillot 





___
Users mailing list: Users@clusterlabs.org
http://lists.clusterlabs.org/mailman/listinfo/users

Project Home: http://www.clusterlabs.org
Getting started: http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf
Bugs: http://bugs.clusterlabs.org
___
Users mailing list: Users@clusterlabs.org
http://lists.clusterlabs.org/mailman/listinfo/users

Project Home: http://www.clusterlabs.org
Getting started: http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf
Bugs: http://bugs.clusterlabs.org

Re: [ClusterLabs] Pacemaker in Azure

[Leon Steffens]

Unfortunately I can't post the full resource agent here.

In our search for solutions we did find a resource agent for managing AWS
Elastic IPs:
https://github.com/moomindani/aws-eip-resource-agent/blob/master/eip.  This
was not what we wanted, but it will give you an idea of how it can work.

Our script manages secondary private IPs by using:

aws ec2 assign-private-ip-addresses
aws ec2 unassign-private-ip-addresses
aws ec2 describe-network-interfaces


There are a few things to consider:
* The AWS call to assign IPs to an EC2 instance is asynchronous (or it was
the last time I checked), so you have to wait a bit (or poll AWS/Azure
until the IP is ready).
* The IP change is slower than a normal VIP change on the machine, so
expect a slightly longer outage.


Leon
___
Users mailing list: Users@clusterlabs.org
http://lists.clusterlabs.org/mailman/listinfo/users

Project Home: http://www.clusterlabs.org
Getting started: http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf
Bugs: http://bugs.clusterlabs.org

Re: [ClusterLabs] Pacemaker in Azure

[Ken Gaillot]

That would definitely be of wider interest.

I could see modifying the IPaddr2 RA to take some new arguments for
AWS/Azure parameters, and if those are configured, it would do the
appropriate API requests.

On Thu, 2017-08-24 at 23:27 +, Eric Robinson wrote:
> Leon -- I will pay you one trillion samolians for that resource agent!
> Any way we can get our hands on a copy? 
> 
>  
> 
> --
> Eric Robinson
> 
>  
> 
> From: Leon Steffens [mailto:l...@steffensonline.com] 
> Sent: Thursday, August 24, 2017 3:48 PM
> To: Cluster Labs - All topics related to open-source clustering
> welcomed 
> Subject: Re: [ClusterLabs] Pacemaker in Azure
> 
>  
> 
> That's what we did in AWS.  The IPaddr2 resource agent does an arp
> broadcast after changing the local IP but this does not work in AWS
> (probably for the same reasons as Azure). 
> 
>  
> 
> 
> We created our own OCF resource agent that uses the Amazon APIs to
> move the IP in AWS land and made that dependent on the IPaddr2
> resource, and it worked fine.
> 
> 
>  
> 
> 
>  
> 
> 
> Leon Steffens
> 
> 
>  
> 
> On Fri, Aug 25, 2017 at 8:34 AM, Eric Robinson
>  wrote:
> 
> > Don't use Azure? ;)
> 
> That would be my preference. But since I'm stuck with Azure
> (management decision) I need to come up with something. It
> appears there is an Azure API to make changes on-the-fly from
> a Linux box. Maybe I'll write a resource agent to change Azure
> and make IPaddr2 dependent on it. That might work?
> 
> --
> Eric Robinson
> 
> 
> ___
> Users mailing list: Users@clusterlabs.org
> http://lists.clusterlabs.org/mailman/listinfo/users
> 
> Project Home: http://www.clusterlabs.org
> Getting started:
> http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf
> Bugs: http://bugs.clusterlabs.org
> 
>  
> 
> 
> ___
> Users mailing list: Users@clusterlabs.org
> http://lists.clusterlabs.org/mailman/listinfo/users
> 
> Project Home: http://www.clusterlabs.org
> Getting started: http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf
> Bugs: http://bugs.clusterlabs.org

-- 
Ken Gaillot 





___
Users mailing list: Users@clusterlabs.org
http://lists.clusterlabs.org/mailman/listinfo/users

Project Home: http://www.clusterlabs.org
Getting started: http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf
Bugs: http://bugs.clusterlabs.org

Re: [ClusterLabs] Pacemaker in Azure

[Eric Robinson]

Leon -- I will pay you one trillion samolians for that resource agent! Any way 
we can get our hands on a copy?

--
Eric Robinson

From: Leon Steffens [mailto:l...@steffensonline.com]
Sent: Thursday, August 24, 2017 3:48 PM
To: Cluster Labs - All topics related to open-source clustering welcomed 

Subject: Re: [ClusterLabs] Pacemaker in Azure

That's what we did in AWS.  The IPaddr2 resource agent does an arp broadcast 
after changing the local IP but this does not work in AWS (probably for the 
same reasons as Azure).

We created our own OCF resource agent that uses the Amazon APIs to move the IP 
in AWS land and made that dependent on the IPaddr2 resource, and it worked fine.


Leon Steffens

On Fri, Aug 25, 2017 at 8:34 AM, Eric Robinson 
mailto:eric.robin...@psmnv.com>> wrote:
> Don't use Azure? ;)

That would be my preference. But since I'm stuck with Azure (management 
decision) I need to come up with something. It appears there is an Azure API to 
make changes on-the-fly from a Linux box. Maybe I'll write a resource agent to 
change Azure and make IPaddr2 dependent on it. That might work?

--
Eric Robinson


___
Users mailing list: Users@clusterlabs.org
http://lists.clusterlabs.org/mailman/listinfo/users

Project Home: http://www.clusterlabs.org
Getting started: http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf
Bugs: http://bugs.clusterlabs.org

___
Users mailing list: Users@clusterlabs.org
http://lists.clusterlabs.org/mailman/listinfo/users

Project Home: http://www.clusterlabs.org
Getting started: http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf
Bugs: http://bugs.clusterlabs.org

Re: [ClusterLabs] start one node only?

[Dimitri Maziuk]

On 08/24/2017 04:24 PM, Ken Gaillot wrote:

> How could it know that, from a cold boot? It doesn't know if the other
> node is down, or up but unreachable. wait_for_all is how to keep that
> fencing from happening at every cluster start, but the trade-off is you
> can't cold-boot a partial cluster.

OK, thanks.

-- 
Dimitri Maziuk
Programmer/sysadmin
BioMagResBank, UW-Madison -- http://www.bmrb.wisc.edu



signature.asc
Description: OpenPGP digital signature
___
Users mailing list: Users@clusterlabs.org
http://lists.clusterlabs.org/mailman/listinfo/users

Project Home: http://www.clusterlabs.org
Getting started: http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf
Bugs: http://bugs.clusterlabs.org

Re: [ClusterLabs] Pacemaker in Azure

[Leon Steffens]

That's what we did in AWS.  The IPaddr2 resource agent does an arp
broadcast after changing the local IP but this does not work in AWS
(probably for the same reasons as Azure).

We created our own OCF resource agent that uses the Amazon APIs to move the
IP in AWS land and made that dependent on the IPaddr2 resource, and it
worked fine.


Leon Steffens

On Fri, Aug 25, 2017 at 8:34 AM, Eric Robinson 
wrote:

> > Don't use Azure? ;)
>
> That would be my preference. But since I'm stuck with Azure (management
> decision) I need to come up with something. It appears there is an Azure
> API to make changes on-the-fly from a Linux box. Maybe I'll write a
> resource agent to change Azure and make IPaddr2 dependent on it. That might
> work?
>
> --
> Eric Robinson
>
>
> ___
> Users mailing list: Users@clusterlabs.org
> http://lists.clusterlabs.org/mailman/listinfo/users
>
> Project Home: http://www.clusterlabs.org
> Getting started: http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf
> Bugs: http://bugs.clusterlabs.org
>
___
Users mailing list: Users@clusterlabs.org
http://lists.clusterlabs.org/mailman/listinfo/users

Project Home: http://www.clusterlabs.org
Getting started: http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf
Bugs: http://bugs.clusterlabs.org

Re: [ClusterLabs] Pacemaker in Azure

[Eric Robinson]

> Don't use Azure? ;)

That would be my preference. But since I'm stuck with Azure (management 
decision) I need to come up with something. It appears there is an Azure API to 
make changes on-the-fly from a Linux box. Maybe I'll write a resource agent to 
change Azure and make IPaddr2 dependent on it. That might work?

--
Eric Robinson


___
Users mailing list: Users@clusterlabs.org
http://lists.clusterlabs.org/mailman/listinfo/users

Project Home: http://www.clusterlabs.org
Getting started: http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf
Bugs: http://bugs.clusterlabs.org

Re: [ClusterLabs] start one node only?

[Ken Gaillot]

On Thu, 2017-08-24 at 15:53 -0500, Dimitri Maziuk wrote:
> On 08/24/2017 03:40 PM, Ken Gaillot wrote:
> 
> > You could set wait_for_all to 0 in corosync.conf, then boot. The living
> > node should try to fence the other one, and proceed if fencing succeeds.
> 
> Didn't I just read a thread that says it won't: the other node is
> already down?

How could it know that, from a cold boot? It doesn't know if the other
node is down, or up but unreachable. wait_for_all is how to keep that
fencing from happening at every cluster start, but the trade-off is you
can't cold-boot a partial cluster.
-- 
Ken Gaillot 





___
Users mailing list: Users@clusterlabs.org
http://lists.clusterlabs.org/mailman/listinfo/users

Project Home: http://www.clusterlabs.org
Getting started: http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf
Bugs: http://bugs.clusterlabs.org

Re: [ClusterLabs] start one node only?

[Dimitri Maziuk]

On 08/24/2017 03:40 PM, Ken Gaillot wrote:

> You could set wait_for_all to 0 in corosync.conf, then boot. The living
> node should try to fence the other one, and proceed if fencing succeeds.

Didn't I just read a thread that says it won't: the other node is
already down?

-- 
Dimitri Maziuk
Programmer/sysadmin
BioMagResBank, UW-Madison -- http://www.bmrb.wisc.edu



signature.asc
Description: OpenPGP digital signature
___
Users mailing list: Users@clusterlabs.org
http://lists.clusterlabs.org/mailman/listinfo/users

Project Home: http://www.clusterlabs.org
Getting started: http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf
Bugs: http://bugs.clusterlabs.org

Re: [ClusterLabs] ClusterLabs.Org Documentation Problem?

[Ken Gaillot]

On Wed, 2017-08-23 at 23:33 +, Eric Robinson wrote:
> I have a BIG correction.
> 
> If you follow the instructions titled, "Pacemaker 1.1 for Corosync 2.x," and 
> NOT the ones entitled, "Pacemaker 1.1 for CMAN or Corosync 1.x," guess what? 
> It installs cman anyway, and you spend a couple of days wondering why none of 
> your changes to corosync.conf seem to be working.
> 
> --
> Eric Robinson

That's an unfortunate result of trying to use the corosync 2
instructions with CentOS 6, which only supports corosync 1 + CMAN.

The "Pacemaker Explained" document is independent of OS and toolset, but
"Clusters From Scratch" and the walk-through portions of "Pacemaker
Remote" have to pick one configuration to use for examples, and
currently they use CentOS 7.

At one point, we maintained dual versions of Clusters From Scratch for
CentOS and OpenSuSE, but it was too difficult to maintain. I believe
Debian maintains their own variant at a different location.

It would probably be worthwhile to add more info boxes to Clusters From
Scratch pointing out where other OSes might do things differently.

> -Original Message-
> From: Jan Friesse [mailto:jfrie...@redhat.com] 
> Sent: Tuesday, August 22, 2017 11:52 PM
> To: Cluster Labs - All topics related to open-source clustering welcomed 
> ; kgail...@redhat.com
> Subject: Re: [ClusterLabs] ClusterLabs.Org Documentation Problem?
> 
> > Thanks for the reply. Yes, it's a bit confusing. I did end up using the 
> > documentation for Corosync 2.X since that seemed newer, but it also assumed 
> > CentOS/RHEL7 and systemd-based commands. It also incorporates cman, pcsd, 
> > psmisc, and policycoreutils-pythonwhich, which are all new to me. If there 
> > is anything I can do to assist with getting the documentation cleaned up, 
> > I'd be more than glad to help.
> 
> Just a small correction.
> 
> Documentation shouldn't incorporate cman. Cman was used with corosync 1.x as 
> a configuration layer and (more important) quorum provider. With Corosync 2.x 
> quorum provider is already in corosync so no need for cman.
> 
> 
> 
> >
> > --
> > Eric Robinson
> >
> > -Original Message-
> > From: Ken Gaillot [mailto:kgail...@redhat.com]
> > Sent: Tuesday, August 22, 2017 2:08 PM
> > To: Cluster Labs - All topics related to open-source clustering 
> > welcomed 
> > Subject: Re: [ClusterLabs] ClusterLabs.Org Documentation Problem?
> >
> > On Tue, 2017-08-22 at 19:40 +, Eric Robinson wrote:
> >> The documentation located here…
> >>
> >>
> >>
> >> http://clusterlabs.org/doc/
> >>
> >>
> >>
> >> …is confusing because it offers two combinations:
> >>
> >>
> >>
> >> Pacemaker 1.0 for Corosync 1.x
> >>
> >> Pacemaker 1.1 for Corosync 2.x
> >>
> >>
> >>
> >> According to the documentation, if you use Corosync 1.x you need 
> >> Pacemaker 1.0, but if you use Corosync 2.x then you need Pacemaker 
> >> 1.1.
> >>
> >>
> >>
> >> However, on my Centos 6.9 system, when I do ‘yum install pacemaker 
> >> corosync” I get the following versions:
> >>
> >>
> >>
> >> pacemaker-1.1.15-5.el6.x86_64
> >>
> >> corosync-1.4.7-5.el6.x86_64
> >>
> >>
> >>
> >> What’s the correct answer? Does Pacemaker 1.1.15 work with Corosync 
> >> 1.4.7? If so, is the documentation at ClusterLabs misleading?
> >>
> >>
> >>
> >> --
> >> Eric Robinson
> >
> > The page actually offers a third option ... "Pacemaker 1.1 for CMAN or 
> > Corosync 1.x". That's the configuration used by CentOS 6.
> >
> > However, that's still a bit misleading; the documentation set for 
> > "Pacemaker 1.1 for Corosync 2.x" is the only one that is updated, and it's 
> > mostly independent of the underlying layer, so you should prefer that set.
> >
> > I plan to reorganize that page in the coming months, so I'll try to make it 
> > clearer.

-- 
Ken Gaillot 





___
Users mailing list: Users@clusterlabs.org
http://lists.clusterlabs.org/mailman/listinfo/users

Project Home: http://www.clusterlabs.org
Getting started: http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf
Bugs: http://bugs.clusterlabs.org

Re: [ClusterLabs] start one node only?

[Ken Gaillot]

On Thu, 2017-08-24 at 15:10 -0500, Dimitri Maziuk wrote:
> Hi everyone,
> 
> I seem to remember seeing theis once before, but my google-fu is
> failing: I've a 2-node active-passive cluster, when I power up one node
> only, resources remain stopped. Is there a way to boot a cluster on one
> node only?
> 
> -- Note that if I boot up the other node everything starts, and then I
> can shut one of them down and it'll keep running. But that doesn't seem
> to happen when starting cold.
> 
> What am I missing?
> 
> TIA

That's a fail-safe. You're probably using corosync's wait_for_all option
(most likely via the two_node option). See the votequorum(5) man page
for details.

You could set wait_for_all to 0 in corosync.conf, then boot. The living
node should try to fence the other one, and proceed if fencing succeeds.
You may want to set wait_for_all back to 1 once your cluster is back to
normal.
-- 
Ken Gaillot 





___
Users mailing list: Users@clusterlabs.org
http://lists.clusterlabs.org/mailman/listinfo/users

Project Home: http://www.clusterlabs.org
Getting started: http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf
Bugs: http://bugs.clusterlabs.org

Re: [ClusterLabs] start one node only?

[Dimitri Maziuk]

PS. centos 7.latest w/ the current pcs/corosync/pacemaker rpms as
distributed by centos, resources are stonith:fence_scsi, IPaddr2, and ZFS.

-- 
Dimitri Maziuk
Programmer/sysadmin
BioMagResBank, UW-Madison -- http://www.bmrb.wisc.edu



signature.asc
Description: OpenPGP digital signature
___
Users mailing list: Users@clusterlabs.org
http://lists.clusterlabs.org/mailman/listinfo/users

Project Home: http://www.clusterlabs.org
Getting started: http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf
Bugs: http://bugs.clusterlabs.org

[ClusterLabs] start one node only?

[Dimitri Maziuk]

Hi everyone,

I seem to remember seeing theis once before, but my google-fu is
failing: I've a 2-node active-passive cluster, when I power up one node
only, resources remain stopped. Is there a way to boot a cluster on one
node only?

-- Note that if I boot up the other node everything starts, and then I
can shut one of them down and it'll keep running. But that doesn't seem
to happen when starting cold.

What am I missing?

TIA
-- 
Dimitri Maziuk
Programmer/sysadmin
BioMagResBank, UW-Madison -- http://www.bmrb.wisc.edu



signature.asc
Description: OpenPGP digital signature
___
Users mailing list: Users@clusterlabs.org
http://lists.clusterlabs.org/mailman/listinfo/users

Project Home: http://www.clusterlabs.org
Getting started: http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf
Bugs: http://bugs.clusterlabs.org

Re: [ClusterLabs] Pacemaker in Azure

[Digimer]

On 2017-08-24 03:56 PM, Eric Robinson wrote:
> I deployed a couple of cluster nodes in Azure and found out right away
> that floating a virtual IP address between nodes does not work because
> Azure does not honor IP changes made from within the VMs. IP changes
> must be made to virtual NICs in the Azure portal itself. Anybody know of
> an easy way around this limitation?
> 
> --
> Eric Robinson

Don't use Azure? ;)




-- 
Digimer
Papers and Projects: https://alteeve.com/w/
"I am, somehow, less interested in the weight and convolutions of
Einstein’s brain than in the near certainty that people of equal talent
have lived and died in cotton fields and sweatshops." - Stephen Jay Gould

___
Users mailing list: Users@clusterlabs.org
http://lists.clusterlabs.org/mailman/listinfo/users

Project Home: http://www.clusterlabs.org
Getting started: http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf
Bugs: http://bugs.clusterlabs.org

[ClusterLabs] Pacemaker in Azure

[Eric Robinson]

I deployed a couple of cluster nodes in Azure and found out right away that 
floating a virtual IP address between nodes does not work because Azure does 
not honor IP changes made from within the VMs. IP changes must be made to 
virtual NICs in the Azure portal itself. Anybody know of an easy way around 
this limitation?

--
Eric Robinson

___
Users mailing list: Users@clusterlabs.org
http://lists.clusterlabs.org/mailman/listinfo/users

Project Home: http://www.clusterlabs.org
Getting started: http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf
Bugs: http://bugs.clusterlabs.org