Re: [ClusterLabs] Offtopic - role migration
On Tue, Apr 18, 2023 at 9:09 PM Ken Gaillot wrote: > On Tue, 2023-04-18 at 19:50 +0200, Vladislav Bogdanov wrote: > > Btw, an interesting question. How much efforts would it take to > > support a migration of a Master role over the nodes? An use-case is > > drbd, configured for a multi-master mode internally, but with master- > > max=1 in the resource definition. Assuming that resource-agent > > supports that flow - > > 1. Do nothing. > > 2. Promote on a dest node. > > 3. Demote on a source node. > > > > Actually just wonder, because may be it could be some-how achievable > > to migrate VM which are on top of drbd which is not a multi-master in > > pacemaker. Fully theoretical case. Didn't verify the flow in-the- > > mind. > > > > I believe that currently only the top-most resource is allowed to > > migrate, but may be there is some room for impovement? > > > > Sorry for the off-topic. > > > > Best > > Vlad > > It would be worthwhile, but conceptually it's difficult to imagine a > solution. > > If a resource must be colocated with the promoted role of another > resource, and only one instance can be promoted at a time, how would it > be possible to live-migrate? You couldn't promote the new instance > before demoting the old one, and you couldn't demote the old one > without stopping the dependent resource. > > You would probably need some really complex new constraint types and/or > resource agent actions. Something like "colocate rsc1 with the promoted > role of rsc2-clone, unless it needs to migrate, in which case call this > special agent action to prepare it for running with a demoted instance, > then demote the instance, then migrate the resource, then promote the > new instance, then call this other agent action to return it to normal > operation". > Don't know if I got that correctly but wouldn't it rather have to be the other way round? - Migration source running on the promoted device - Start target in receiving mode but without automatically starting once state is complete (don't know it qemu supports that) and without access to block-devices - Start migration on the source side - Once state difference is small enough disable VM on source side + transfer the leftover state - Switch underlying filesystem/block-device to the destination - Make qemu pick up at the state is has memorized and transferred If we make the VM a promotable resource as well we could try to pull the promoted state of fliesystem & VM to the other side once migration is kicked off on the source-side. Source side VM would refuse demotion as long as the state is passed to the other side. We would need that measure of state difference is small enough qemu is using when it kicks of the activation of the target. At the end of demotion it would stop qemu and transfer the leftovers. Then pacemaker could demote the source filesystem and promote on the destination-side. That would trigger promoting the destination VM which makes it run on the transfered state. The above concept would probably work with cold migration to a state-image on the destination side. But that is probably not interesting. Don't know if qemu allows interception at the interesting points. Don't know if we need the resource-interface to be extended for this. Klaus -- > Ken Gaillot > > ___ > Manage your subscription: > https://lists.clusterlabs.org/mailman/listinfo/users > > ClusterLabs home: https://www.clusterlabs.org/ > > ___ Manage your subscription: https://lists.clusterlabs.org/mailman/listinfo/users ClusterLabs home: https://www.clusterlabs.org/
Re: [ClusterLabs] Offtopic - role migration
On Tue, 2023-04-18 at 19:50 +0200, Vladislav Bogdanov wrote: > Btw, an interesting question. How much efforts would it take to > support a migration of a Master role over the nodes? An use-case is > drbd, configured for a multi-master mode internally, but with master- > max=1 in the resource definition. Assuming that resource-agent > supports that flow - > 1. Do nothing. > 2. Promote on a dest node. > 3. Demote on a source node. > > Actually just wonder, because may be it could be some-how achievable > to migrate VM which are on top of drbd which is not a multi-master in > pacemaker. Fully theoretical case. Didn't verify the flow in-the- > mind. > > I believe that currently only the top-most resource is allowed to > migrate, but may be there is some room for impovement? > > Sorry for the off-topic. > > Best > Vlad It would be worthwhile, but conceptually it's difficult to imagine a solution. If a resource must be colocated with the promoted role of another resource, and only one instance can be promoted at a time, how would it be possible to live-migrate? You couldn't promote the new instance before demoting the old one, and you couldn't demote the old one without stopping the dependent resource. You would probably need some really complex new constraint types and/or resource agent actions. Something like "colocate rsc1 with the promoted role of rsc2-clone, unless it needs to migrate, in which case call this special agent action to prepare it for running with a demoted instance, then demote the instance, then migrate the resource, then promote the new instance, then call this other agent action to return it to normal operation". -- Ken Gaillot ___ Manage your subscription: https://lists.clusterlabs.org/mailman/listinfo/users ClusterLabs home: https://www.clusterlabs.org/
[ClusterLabs] Offtopic - role migration
Btw, an interesting question. How much efforts would it take to support a migration of a Master role over the nodes? An use-case is drbd, configured for a multi-master mode internally, but with master-max=1 in the resource definition. Assuming that resource-agent supports that flow - 1. Do nothing. 2. Promote on a dest node. 3. Demote on a source node. Actually just wonder, because may be it could be some-how achievable to migrate VM which are on top of drbd which is not a multi-master in pacemaker. Fully theoretical case. Didn't verify the flow in-the-mind. I believe that currently only the top-most resource is allowed to migrate, but may be there is some room for impovement? Sorry for the off-topic. Best Vlad Ken Gaillot 18 апреля 2023 г. 18:23:00 написал: On Tue, 2023-04-18 at 14:58 +0200, lejeczek via Users wrote: Hi guys. When it's done by the cluster itself, eg. a node goes 'standby' - how do clusters migrate VirtualDomain resources? 1. Call resource agent migrate_to action on original node 2. Call resource agent migrate_from action on new node 3. Call resource agent stop action on original node Do users have any control over it and if so then how? The allow-migrate resource meta-attribute (true/false) I'd imagine there must be some docs - I failed to find It's sort of scattered throughout Pacemaker Explained -- the main one is: https://clusterlabs.org/pacemaker/doc/2.1/Pacemaker_Explained/html/advanced-options.html#migrating-resources Especially in large deployments one obvious question would be - I'm guessing as my setup is rather SOHO - can VMs migrate in sequence or it is(always?) a kind of 'swarm' migration? The migration-limit cluster property specifies how many live migrations may be initiated at once (the default of -1 means unlimited). -- Ken Gaillot ___ Manage your subscription: https://lists.clusterlabs.org/mailman/listinfo/users ClusterLabs home: https://www.clusterlabs.org/ ___ Manage your subscription: https://lists.clusterlabs.org/mailman/listinfo/users ClusterLabs home: https://www.clusterlabs.org/
