Re: [ClusterLabs] Offtopic - role migration

[Ken Gaillot]

On Tue, 2023-04-18 at 19:50 +0200, Vladislav Bogdanov wrote:
> Btw, an interesting question. How much efforts would it take to
> support a migration of a Master role over the nodes? An use-case is
> drbd, configured for a multi-master mode internally, but with master-
> max=1 in the resource definition. Assuming that resource-agent
> supports that flow - 
> 1. Do nothing. 
> 2. Promote on a dest node. 
> 3. Demote on a source node.
> 
> Actually just wonder, because may be it could be some-how achievable
> to migrate VM which are on top of drbd which is not a multi-master in
> pacemaker. Fully theoretical case. Didn't verify the flow in-the-
> mind.
> 
> I believe that currently only the top-most resource is allowed to
> migrate, but may be there is some room for impovement?
> 
> Sorry for the off-topic.
> 
> Best
> Vlad

It would be worthwhile, but conceptually it's difficult to imagine a
solution.

If a resource must be colocated with the promoted role of another
resource, and only one instance can be promoted at a time, how would it
be possible to live-migrate? You couldn't promote the new instance
before demoting the old one, and you couldn't demote the old one
without stopping the dependent resource.

You would probably need some really complex new constraint types and/or
resource agent actions. Something like "colocate rsc1 with the promoted
role of rsc2-clone, unless it needs to migrate, in which case call this
special agent action to prepare it for running with a demoted instance,
then demote the instance, then migrate the resource, then promote the
new instance, then call this other agent action to return it to normal
operation".
-- 
Ken Gaillot 

___
Manage your subscription:
https://lists.clusterlabs.org/mailman/listinfo/users

ClusterLabs home: https://www.clusterlabs.org/

Re: [ClusterLabs] manner in which cluster migrates VirtualDomain - ?

[Ken Gaillot]

On Tue, 2023-04-18 at 19:36 +0200, lejeczek via Users wrote:
> 
> On 18/04/2023 18:22, Ken Gaillot wrote:
> > On Tue, 2023-04-18 at 14:58 +0200, lejeczek via Users wrote:
> > > Hi guys.
> > > 
> > > When it's done by the cluster itself, eg. a node goes 'standby' -
> > > how
> > > do clusters migrate VirtualDomain resources?
> > 1. Call resource agent migrate_to action on original node
> > 2. Call resource agent migrate_from action on new node
> > 3. Call resource agent stop action on original node
> > 
> > > Do users have any control over it and if so then how?
> > The allow-migrate resource meta-attribute (true/false)
> > 
> > > I'd imagine there must be some docs - I failed to find
> > It's sort of scattered throughout Pacemaker Explained -- the main
> > one
> > is:
> > 
> > https://clusterlabs.org/pacemaker/doc/2.1/Pacemaker_Explained/html/advanced-options.html#migrating-resources
> > 
> > > Especially in large deployments one obvious question would be -
> > > I'm
> > > guessing as my setup is rather SOHO - can VMs migrate in sequence
> > > or
> > > it is(always?) a kind of 'swarm' migration?
> > The migration-limit cluster property specifies how many live
> > migrations
> > may be initiated at once (the default of -1 means unlimited).
> But if this is cluster property - unless I got it wrong, 
> hopefully - then this govern any/all resources.
> If so, can such a limit be rounded down to RA type or 
> perhaps group of resources?
> 
> many thanks, L.

No, it's global
-- 
Ken Gaillot 

___
Manage your subscription:
https://lists.clusterlabs.org/mailman/listinfo/users

ClusterLabs home: https://www.clusterlabs.org/

[ClusterLabs] Offtopic - role migration

[Vladislav Bogdanov]

Btw, an interesting question. How much efforts would it take to support a 
migration of a Master role over the nodes? An use-case is drbd, configured 
for a multi-master mode internally, but with master-max=1 in the resource 
definition. Assuming that resource-agent supports that flow -

1. Do nothing.
2. Promote on a dest node.
3. Demote on a source node.

Actually just wonder, because may be it could be some-how achievable to 
migrate VM which are on top of drbd which is not a multi-master in 
pacemaker. Fully theoretical case. Didn't verify the flow in-the-mind.


I believe that currently only the top-most resource is allowed to migrate, 
but may be there is some room for impovement?


Sorry for the off-topic.

Best
Vlad

Ken Gaillot  18 апреля 2023 г. 18:23:00 написал:


On Tue, 2023-04-18 at 14:58 +0200, lejeczek via Users wrote:

Hi guys.

When it's done by the cluster itself, eg. a node goes 'standby' - how
do clusters migrate VirtualDomain resources?


1. Call resource agent migrate_to action on original node
2. Call resource agent migrate_from action on new node
3. Call resource agent stop action on original node


Do users have any control over it and if so then how?


The allow-migrate resource meta-attribute (true/false)


I'd imagine there must be some docs - I failed to find


It's sort of scattered throughout Pacemaker Explained -- the main one
is:

https://clusterlabs.org/pacemaker/doc/2.1/Pacemaker_Explained/html/advanced-options.html#migrating-resources


Especially in large deployments one obvious question would be - I'm
guessing as my setup is rather SOHO - can VMs migrate in sequence or
it is(always?) a kind of 'swarm' migration?


The migration-limit cluster property specifies how many live migrations
may be initiated at once (the default of -1 means unlimited).
--
Ken Gaillot 

___
Manage your subscription:
https://lists.clusterlabs.org/mailman/listinfo/users

ClusterLabs home: https://www.clusterlabs.org/


___
Manage your subscription:
https://lists.clusterlabs.org/mailman/listinfo/users

ClusterLabs home: https://www.clusterlabs.org/

Re: [ClusterLabs] manner in which cluster migrates VirtualDomain - ?

[lejeczek via Users]

On 18/04/2023 18:22, Ken Gaillot wrote:

On Tue, 2023-04-18 at 14:58 +0200, lejeczek via Users wrote:

Hi guys.

When it's done by the cluster itself, eg. a node goes 'standby' - how
do clusters migrate VirtualDomain resources?

1. Call resource agent migrate_to action on original node
2. Call resource agent migrate_from action on new node
3. Call resource agent stop action on original node


Do users have any control over it and if so then how?

The allow-migrate resource meta-attribute (true/false)


I'd imagine there must be some docs - I failed to find

It's sort of scattered throughout Pacemaker Explained -- the main one
is:

https://clusterlabs.org/pacemaker/doc/2.1/Pacemaker_Explained/html/advanced-options.html#migrating-resources


Especially in large deployments one obvious question would be - I'm
guessing as my setup is rather SOHO - can VMs migrate in sequence or
it is(always?) a kind of 'swarm' migration?

The migration-limit cluster property specifies how many live migrations
may be initiated at once (the default of -1 means unlimited).
But if this is cluster property - unless I got it wrong, 
hopefully - then this govern any/all resources.
If so, can such a limit be rounded down to RA type or 
perhaps group of resources?


many thanks, L.
___
Manage your subscription:
https://lists.clusterlabs.org/mailman/listinfo/users

ClusterLabs home: https://www.clusterlabs.org/

Re: [ClusterLabs] manner in which cluster migrates VirtualDomain - ?

[Ken Gaillot]

On Tue, 2023-04-18 at 14:58 +0200, lejeczek via Users wrote:
> Hi guys.
> 
> When it's done by the cluster itself, eg. a node goes 'standby' - how
> do clusters migrate VirtualDomain resources?

1. Call resource agent migrate_to action on original node
2. Call resource agent migrate_from action on new node
3. Call resource agent stop action on original node

> Do users have any control over it and if so then how?

The allow-migrate resource meta-attribute (true/false)

> I'd imagine there must be some docs - I failed to find

It's sort of scattered throughout Pacemaker Explained -- the main one
is:

https://clusterlabs.org/pacemaker/doc/2.1/Pacemaker_Explained/html/advanced-options.html#migrating-resources

> Especially in large deployments one obvious question would be - I'm
> guessing as my setup is rather SOHO - can VMs migrate in sequence or
> it is(always?) a kind of 'swarm' migration?

The migration-limit cluster property specifies how many live migrations
may be initiated at once (the default of -1 means unlimited).
-- 
Ken Gaillot 

___
Manage your subscription:
https://lists.clusterlabs.org/mailman/listinfo/users

ClusterLabs home: https://www.clusterlabs.org/

Re: [ClusterLabs] NFS mount fails to stop if NFS server is lost

[Miro Igov]

Force -f is improper in situations when NFS server is gone.

I have node intranet-test2 with nfs mount and nas-sync-test2 with NFS server 
and export resources.

I made following experiment:

1.  Paused the VM running nas-sync-test2 so cluster lost the member
2.  Executed umount -f /folder and it displayed error umount.nfs4: /folder: 
device is busy
3.  Executed umount -f -l /folder and all fine

 

Reference: 
https://askubuntu.com/questions/292043/how-to-unmount-nfs-when-server-is-gone

 

Is it possible to add the lazy -l option to the umount ?

 

From: Miro Igov  
Sent: 11 April 2023 21:47
To: Cluster Labs - All topics related to open-source clustering welcomed 

Subject: Re: [ClusterLabs] NFS mount fails to stop if NFS server is lost

 

What about umount -f -l ?

 

  _  

From: Users mailto:users-boun...@clusterlabs.org> > on behalf of Andrei Borzenkov 
mailto:arvidj...@gmail.com> >
Sent: Tuesday, April 11, 2023, 20:26
To: users@clusterlabs.org   
mailto:users@clusterlabs.org> >
Subject: Re: [ClusterLabs] NFS mount fails to stop if NFS server is lost


On 11.04.2023 17:35, Miro Igov wrote:
> Hello,
> 
> I have a node nas-sync-test1 with NFS server and NFS export running and
> another node intranet-test1 with data_1 fs mount:
> 
> 
> 
> primitive data_1 Filesystem \
> 
> params device="nas-sync-test1:/home/pharmya/NAS" fstype=nfs
> options=v4 directory="/data/synology/pharmya_office/NAS_Sync/NAS" \
> 
> op monitor interval=10s
> 
> 
> 
> Disconnecting nas-sync-test1 from the network shows it's state as UNCLEAN
> and pacemaker fences it.
> 
> Then it tries to stop data_1 but it shows timeout error.
> 
> 
> 
> I know unmounting of NFS mount when NFS server is gone requires force. Is
> there such option in Filesystem RA ?
> 

It does it unconditionally from the very beginning

# For networked filesystems, there's merit in trying -f:
case "$FSTYPE" in
nfs4|nfs|efs|cifs|smbfs) umount_force="-f" ;;
esac

But IIRC it is not enough (at least, not always) - so the trick is to 
setup the same IP address as server had. It could be an alias on a local 
client interface, does not matter. I suppose client gets stuck somewhere 
below application layer and having IP resets TCP connection allowing it 
to proceed.

I certainly had to do it in the past, and simple "umount -f" did not work.
___
Manage your subscription:
https://lists.clusterlabs.org/mailman/listinfo/users

ClusterLabs home: https://www.clusterlabs.org/




-- 

This message has been sent as a part of discussion between PHARMYA and the

addressee whose name is specified above. Should you receive this message by

mistake, we would be most grateful if you informed us that the message has

been sent to you. In this case, we also ask that you delete this message

from your mailbox, and do not forward it or any part of it to anyone else.

Thank you for your cooperation and understanding.



___
Manage your subscription:
https://lists.clusterlabs.org/mailman/listinfo/users

ClusterLabs home: https://www.clusterlabs.org/

[ClusterLabs] manner in which cluster migrates VirtualDomain - ?

[lejeczek via Users]

Hi guys.

When it's done by the cluster itself, eg. a node goes 
'standby' - how do clusters migrate VirtualDomain resources?

Do users have any control over it and if so then how?
I'd imagine there must be some docs - I failed to find
Especially in large deployments one obvious question would 
be - I'm guessing as my setup is rather SOHO - can VMs 
migrate in sequence or it is(always?) a kind of 'swarm' 
migration?


many thanks, L.___
Manage your subscription:
https://lists.clusterlabs.org/mailman/listinfo/users

ClusterLabs home: https://www.clusterlabs.org/