Re: [openstack-dev] [nova] Server Groups - remove VM from group?
On 09/11/2014 05:01 PM, Jay Pipes wrote: > On 09/11/2014 04:51 PM, Matt Riedemann wrote: >> On 9/10/2014 6:00 PM, Russell Bryant wrote: >>> On 09/10/2014 06:46 PM, Joe Cropper wrote: Hmm, not sure I follow the concern, Russell. How is that any different from putting a VM into the group when it’s booted as is done today? This simply defers the ‘group insertion time’ to some time after initial the VM’s been spawned, so I’m not sure this creates anymore race conditions than what’s already there [1]. [1] Sure, the to-be-added VM could be in the midst of a migration or something, but that would be pretty simple to check make sure its task state is None or some such. >>> >>> The way this works at boot is already a nasty hack. It does policy >>> checking in the scheduler, and then has to re-do some policy checking at >>> launch time on the compute node. I'm afraid of making this any worse. >>> In any case, it's probably better to discuss this in the context of a >>> more detailed design proposal. >>> >> >> This [1] is the hack you're referring to right? >> >> [1] >> http://git.openstack.org/cgit/openstack/nova/tree/nova/compute/manager.py?id=2014.2.b3#n1297 >> > > That's the hack *I* had in the back of my mind. Yep. -- Russell Bryant ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [nova] Server Groups - remove VM from group?
On 09/11/2014 04:22 PM, Joe Cropper wrote: I would be a little wary about the DB level locking for stuff like that — it’s certainly doable, but also comes at the expense of things behaving ever-so-slightly different from DBMS to DBMS. Perhaps there are multiple “logical efforts” here—i.e., adding some APIs and cleaning up existing code. I think you could actually do it without locking. Pick a host as we do now, write it into the database, then check whether you hit a race and if so then clear that host from the database and go back to the beginning. Basically the same algorithm that we do now, but all contained within the scheduler code. Chris ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [nova] Server Groups - remove VM from group?
I would be a little wary about the DB level locking for stuff like that — it’s certainly doable, but also comes at the expense of things behaving ever-so-slightly different from DBMS to DBMS. Perhaps there are multiple “logical efforts” here—i.e., adding some APIs and cleaning up existing code. In any case, I’ve started a blueprint on this [1] and we can continue iterating in the nova-spec once kilo opens up. Thanks all for the good discussion on this thus far. [1] https://blueprints.launchpad.net/nova/+spec/dynamic-server-groups - Joe On Sep 11, 2014, at 5:04 PM, Chris Friesen wrote: > On 09/11/2014 03:01 PM, Jay Pipes wrote: >> On 09/11/2014 04:51 PM, Matt Riedemann wrote: >>> On 9/10/2014 6:00 PM, Russell Bryant wrote: On 09/10/2014 06:46 PM, Joe Cropper wrote: > Hmm, not sure I follow the concern, Russell. How is that any different > from putting a VM into the group when it’s booted as is done today? > This simply defers the ‘group insertion time’ to some time after > initial the VM’s been spawned, so I’m not sure this creates anymore > race > conditions than what’s already there [1]. > > [1] Sure, the to-be-added VM could be in the midst of a migration or > something, but that would be pretty simple to check make sure its task > state is None or some such. The way this works at boot is already a nasty hack. It does policy checking in the scheduler, and then has to re-do some policy checking at launch time on the compute node. I'm afraid of making this any worse. In any case, it's probably better to discuss this in the context of a more detailed design proposal. >>> >>> This [1] is the hack you're referring to right? >>> >>> [1] >>> http://git.openstack.org/cgit/openstack/nova/tree/nova/compute/manager.py?id=2014.2.b3#n1297 >>> >> >> That's the hack *I* had in the back of my mind. > > I think that's the only boot hack related to server groups. > > I was thinking that it should be possible to deal with the race more cleanly > by recording the selected compute node in the database at the time of > scheduling. As it stands, the host is implicitly encoded in the compute node > to which we send the boot request and nobody else knows about it. > > Chris > > > ___ > OpenStack-dev mailing list > OpenStack-dev@lists.openstack.org > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [nova] Server Groups - remove VM from group?
On 09/11/2014 03:01 PM, Jay Pipes wrote: On 09/11/2014 04:51 PM, Matt Riedemann wrote: On 9/10/2014 6:00 PM, Russell Bryant wrote: On 09/10/2014 06:46 PM, Joe Cropper wrote: Hmm, not sure I follow the concern, Russell. How is that any different from putting a VM into the group when it’s booted as is done today? This simply defers the ‘group insertion time’ to some time after initial the VM’s been spawned, so I’m not sure this creates anymore race conditions than what’s already there [1]. [1] Sure, the to-be-added VM could be in the midst of a migration or something, but that would be pretty simple to check make sure its task state is None or some such. The way this works at boot is already a nasty hack. It does policy checking in the scheduler, and then has to re-do some policy checking at launch time on the compute node. I'm afraid of making this any worse. In any case, it's probably better to discuss this in the context of a more detailed design proposal. This [1] is the hack you're referring to right? [1] http://git.openstack.org/cgit/openstack/nova/tree/nova/compute/manager.py?id=2014.2.b3#n1297 That's the hack *I* had in the back of my mind. I think that's the only boot hack related to server groups. I was thinking that it should be possible to deal with the race more cleanly by recording the selected compute node in the database at the time of scheduling. As it stands, the host is implicitly encoded in the compute node to which we send the boot request and nobody else knows about it. Chris ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [nova] Server Groups - remove VM from group?
On 09/11/2014 04:51 PM, Matt Riedemann wrote: On 9/10/2014 6:00 PM, Russell Bryant wrote: On 09/10/2014 06:46 PM, Joe Cropper wrote: Hmm, not sure I follow the concern, Russell. How is that any different from putting a VM into the group when it’s booted as is done today? This simply defers the ‘group insertion time’ to some time after initial the VM’s been spawned, so I’m not sure this creates anymore race conditions than what’s already there [1]. [1] Sure, the to-be-added VM could be in the midst of a migration or something, but that would be pretty simple to check make sure its task state is None or some such. The way this works at boot is already a nasty hack. It does policy checking in the scheduler, and then has to re-do some policy checking at launch time on the compute node. I'm afraid of making this any worse. In any case, it's probably better to discuss this in the context of a more detailed design proposal. This [1] is the hack you're referring to right? [1] http://git.openstack.org/cgit/openstack/nova/tree/nova/compute/manager.py?id=2014.2.b3#n1297 That's the hack *I* had in the back of my mind. -jay ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [nova] Server Groups - remove VM from group?
On 9/10/2014 6:00 PM, Russell Bryant wrote: On 09/10/2014 06:46 PM, Joe Cropper wrote: Hmm, not sure I follow the concern, Russell. How is that any different from putting a VM into the group when it’s booted as is done today? This simply defers the ‘group insertion time’ to some time after initial the VM’s been spawned, so I’m not sure this creates anymore race conditions than what’s already there [1]. [1] Sure, the to-be-added VM could be in the midst of a migration or something, but that would be pretty simple to check make sure its task state is None or some such. The way this works at boot is already a nasty hack. It does policy checking in the scheduler, and then has to re-do some policy checking at launch time on the compute node. I'm afraid of making this any worse. In any case, it's probably better to discuss this in the context of a more detailed design proposal. This [1] is the hack you're referring to right? [1] http://git.openstack.org/cgit/openstack/nova/tree/nova/compute/manager.py?id=2014.2.b3#n1297 -- Thanks, Matt Riedemann ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [nova] Server Groups - remove VM from group?
Great to hear. I started a blueprint for this [1]. More detail can be added once the kilo nova-specs directory is created… for now, I’ve tried to put some fairly detailed notes on the blueprint’s description. [1] https://blueprints.launchpad.net/nova/+spec/dynamic-server-groups - Joe On Sep 11, 2014, at 2:11 AM, Sylvain Bauza wrote: > > Le 11/09/2014 01:10, Joe Cropper a écrit : >> Agreed - I’ll draft up a formal proposal in the next week or two and we can >> focus the discussion there. Thanks for the feedback - this provides a good >> framework for implementation considerations. > > Count me on it, I'm interested in discussing the next stage. > > When preparing the scheduler split, I just discovered it was unnecessary to > keep the instance groups setup in the scheduler, because it was creating > dependencies to other Nova objects that the Scheduler doesn't necessarly need > to handle. > I proposed accordingly a patch for moving the logic to the conductor instead, > see the proposal here : > https://review.openstack.org/110043 > > Reviews are welcome of course. > > -Sylvain > > >> - Joe >> On Sep 10, 2014, at 6:00 PM, Russell Bryant wrote: >> >>> On 09/10/2014 06:46 PM, Joe Cropper wrote: Hmm, not sure I follow the concern, Russell. How is that any different from putting a VM into the group when it’s booted as is done today? This simply defers the ‘group insertion time’ to some time after initial the VM’s been spawned, so I’m not sure this creates anymore race conditions than what’s already there [1]. [1] Sure, the to-be-added VM could be in the midst of a migration or something, but that would be pretty simple to check make sure its task state is None or some such. >>> The way this works at boot is already a nasty hack. It does policy >>> checking in the scheduler, and then has to re-do some policy checking at >>> launch time on the compute node. I'm afraid of making this any worse. >>> In any case, it's probably better to discuss this in the context of a >>> more detailed design proposal. >>> >>> -- >>> Russell Bryant >>> >>> ___ >>> OpenStack-dev mailing list >>> OpenStack-dev@lists.openstack.org >>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev >> >> ___ >> OpenStack-dev mailing list >> OpenStack-dev@lists.openstack.org >> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > > > ___ > OpenStack-dev mailing list > OpenStack-dev@lists.openstack.org > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [nova] Server Groups - remove VM from group?
Le 11/09/2014 01:10, Joe Cropper a écrit : Agreed - I’ll draft up a formal proposal in the next week or two and we can focus the discussion there. Thanks for the feedback - this provides a good framework for implementation considerations. Count me on it, I'm interested in discussing the next stage. When preparing the scheduler split, I just discovered it was unnecessary to keep the instance groups setup in the scheduler, because it was creating dependencies to other Nova objects that the Scheduler doesn't necessarly need to handle. I proposed accordingly a patch for moving the logic to the conductor instead, see the proposal here : https://review.openstack.org/110043 Reviews are welcome of course. -Sylvain - Joe On Sep 10, 2014, at 6:00 PM, Russell Bryant wrote: On 09/10/2014 06:46 PM, Joe Cropper wrote: Hmm, not sure I follow the concern, Russell. How is that any different from putting a VM into the group when it’s booted as is done today? This simply defers the ‘group insertion time’ to some time after initial the VM’s been spawned, so I’m not sure this creates anymore race conditions than what’s already there [1]. [1] Sure, the to-be-added VM could be in the midst of a migration or something, but that would be pretty simple to check make sure its task state is None or some such. The way this works at boot is already a nasty hack. It does policy checking in the scheduler, and then has to re-do some policy checking at launch time on the compute node. I'm afraid of making this any worse. In any case, it's probably better to discuss this in the context of a more detailed design proposal. -- Russell Bryant ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [nova] Server Groups - remove VM from group?
Agreed - I’ll draft up a formal proposal in the next week or two and we can focus the discussion there. Thanks for the feedback - this provides a good framework for implementation considerations. - Joe On Sep 10, 2014, at 6:00 PM, Russell Bryant wrote: > On 09/10/2014 06:46 PM, Joe Cropper wrote: >> Hmm, not sure I follow the concern, Russell. How is that any different >> from putting a VM into the group when it’s booted as is done today? >> This simply defers the ‘group insertion time’ to some time after >> initial the VM’s been spawned, so I’m not sure this creates anymore race >> conditions than what’s already there [1]. >> >> [1] Sure, the to-be-added VM could be in the midst of a migration or >> something, but that would be pretty simple to check make sure its task >> state is None or some such. > > The way this works at boot is already a nasty hack. It does policy > checking in the scheduler, and then has to re-do some policy checking at > launch time on the compute node. I'm afraid of making this any worse. > In any case, it's probably better to discuss this in the context of a > more detailed design proposal. > > -- > Russell Bryant > > ___ > OpenStack-dev mailing list > OpenStack-dev@lists.openstack.org > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [nova] Server Groups - remove VM from group?
+1, Chris. I think the key thing here is that such race conditions can already happen if timed just right, unless there’s been some additional checks put in place in the compute API layer since I last scanned the code. We could even look at some x-process locking mechanisms as well if we think that’s necessary. Overall, this seems like a feasible, fairly minor enhancement--that wouldn’t make us any worse off than we are today (perhaps address some existing race conditions), all whilst providing a large improvement to the overall usability of the server groups. :-) - Joe On Sep 10, 2014, at 5:54 PM, Chris Friesen wrote: > On 09/10/2014 04:16 PM, Russell Bryant wrote: >> >> >>> On Sep 10, 2014, at 2:03 PM, Joe Cropper >>> wrote: >>> >>> I would like to craft up a blueprint proposal for Kilo to add two >>> simple extensions to the existing server group APIs that I believe >>> will make them infinitely more usable in any ‘real world’ scenario. >>> I’ll put more details in the proposal, but in a nutshell: >>> >>> 1. Adding a VM to a server group Only allow it to succeed if its >>> policy wouldn’t be violated by the addition of the VM >>> >> >> I'm not sure that determining this at the time of the API request is >> possible due to the parallel and async nature of the system. I'd love >> to hear ideas on how you think this might be done, but I'm really not >> optimistic and would rather just not go down this road. > > I can see a possible race against another instance booting into the group, or > another already-running instance being added to the group. I think the > solution is to do the update as an atomic database transaction. > > It seems like it should be possible to create a database operation that does > the following in a single transaction: > --look up the hosts for the instances in the group > --check that the scheduler policy would be satisfied (at least for the basic > affinity/anti-affinity policies) > --add the instance to the group > > Chris > > ___ > OpenStack-dev mailing list > OpenStack-dev@lists.openstack.org > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [nova] Server Groups - remove VM from group?
On 09/10/2014 06:46 PM, Joe Cropper wrote: > Hmm, not sure I follow the concern, Russell. How is that any different > from putting a VM into the group when it’s booted as is done today? > This simply defers the ‘group insertion time’ to some time after > initial the VM’s been spawned, so I’m not sure this creates anymore race > conditions than what’s already there [1]. > > [1] Sure, the to-be-added VM could be in the midst of a migration or > something, but that would be pretty simple to check make sure its task > state is None or some such. The way this works at boot is already a nasty hack. It does policy checking in the scheduler, and then has to re-do some policy checking at launch time on the compute node. I'm afraid of making this any worse. In any case, it's probably better to discuss this in the context of a more detailed design proposal. -- Russell Bryant ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [nova] Server Groups - remove VM from group?
On 09/10/2014 04:16 PM, Russell Bryant wrote: On Sep 10, 2014, at 2:03 PM, Joe Cropper wrote: I would like to craft up a blueprint proposal for Kilo to add two simple extensions to the existing server group APIs that I believe will make them infinitely more usable in any ‘real world’ scenario. I’ll put more details in the proposal, but in a nutshell: 1. Adding a VM to a server group Only allow it to succeed if its policy wouldn’t be violated by the addition of the VM I'm not sure that determining this at the time of the API request is possible due to the parallel and async nature of the system. I'd love to hear ideas on how you think this might be done, but I'm really not optimistic and would rather just not go down this road. I can see a possible race against another instance booting into the group, or another already-running instance being added to the group. I think the solution is to do the update as an atomic database transaction. It seems like it should be possible to create a database operation that does the following in a single transaction: --look up the hosts for the instances in the group --check that the scheduler policy would be satisfied (at least for the basic affinity/anti-affinity policies) --add the instance to the group Chris ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [nova] Server Groups - remove VM from group?
Hmm, not sure I follow the concern, Russell. How is that any different from putting a VM into the group when it’s booted as is done today? This simply defers the ‘group insertion time’ to some time after initial the VM’s been spawned, so I’m not sure this creates anymore race conditions than what’s already there [1]. [1] Sure, the to-be-added VM could be in the midst of a migration or something, but that would be pretty simple to check make sure its task state is None or some such. - Joe On Sep 10, 2014, at 5:16 PM, Russell Bryant wrote: > > >> On Sep 10, 2014, at 2:03 PM, Joe Cropper wrote: >> >> I agree, Chris. I think a number of folks put in a lot of really great work >> into the existing server groups and there has been a lot of interest on >> their usage, especially given that the scheduler already has some constructs >> in place to piggyback on them. >> >> I would like to craft up a blueprint proposal for Kilo to add two simple >> extensions to the existing server group APIs that I believe will make them >> infinitely more usable in any ‘real world’ scenario. I’ll put more details >> in the proposal, but in a nutshell: >> >> 1. Adding a VM to a server group >> Only allow it to succeed if its policy wouldn’t be violated by the addition >> of the VM >> > > I'm not sure that determining this at the time of the API request is possible > due to the parallel and async nature of the system. I'd love to hear ideas on > how you think this might be done, but I'm really not optimistic and would > rather just not go down this road. > >> 2. Removing a VM from a server group >> Just allow it >> >> I think this would round out the support that’s there and really allow us to >> capitalize on the hard work everyone’s already put into them. >> >> - Joe >> >>> On Aug 26, 2014, at 6:39 PM, Chris Friesen >>> wrote: >>> On 08/25/2014 11:25 AM, Joe Cropper wrote: I was thinking something simple such as only allowing the add operation to succeed IFF no policies are found to be in violation... and then nova wouldn't need to get into all the complexities you mention? >>> >>> Personally I would be in favour of this...nothing fancy, just add it if it >>> already meets all the criteria. This is basically just a database >>> operation so I would hope we could make it reliable in the face of >>> simultaneous things going on with the instance. >>> And remove would be fairly straightforward as well since no constraints would need to be checked. >>> >>> Agreed. >>> >>> Chris >>> >>> ___ >>> OpenStack-dev mailing list >>> OpenStack-dev@lists.openstack.org >>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev >> >> >> ___ >> OpenStack-dev mailing list >> OpenStack-dev@lists.openstack.org >> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > ___ > OpenStack-dev mailing list > OpenStack-dev@lists.openstack.org > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [nova] Server Groups - remove VM from group?
> On Sep 10, 2014, at 2:03 PM, Joe Cropper wrote: > > I agree, Chris. I think a number of folks put in a lot of really great work > into the existing server groups and there has been a lot of interest on their > usage, especially given that the scheduler already has some constructs in > place to piggyback on them. > > I would like to craft up a blueprint proposal for Kilo to add two simple > extensions to the existing server group APIs that I believe will make them > infinitely more usable in any ‘real world’ scenario. I’ll put more details > in the proposal, but in a nutshell: > > 1. Adding a VM to a server group > Only allow it to succeed if its policy wouldn’t be violated by the addition > of the VM > I'm not sure that determining this at the time of the API request is possible due to the parallel and async nature of the system. I'd love to hear ideas on how you think this might be done, but I'm really not optimistic and would rather just not go down this road. > 2. Removing a VM from a server group > Just allow it > > I think this would round out the support that’s there and really allow us to > capitalize on the hard work everyone’s already put into them. > > - Joe > >> On Aug 26, 2014, at 6:39 PM, Chris Friesen >> wrote: >> >>> On 08/25/2014 11:25 AM, Joe Cropper wrote: >>> I was thinking something simple such as only allowing the add >>> operation to succeed IFF no policies are found to be in violation... >>> and then nova wouldn't need to get into all the complexities you >>> mention? >> >> Personally I would be in favour of this...nothing fancy, just add it if it >> already meets all the criteria. This is basically just a database operation >> so I would hope we could make it reliable in the face of simultaneous things >> going on with the instance. >> >>> And remove would be fairly straightforward as well since no >>> constraints would need to be checked. >> >> Agreed. >> >> Chris >> >> ___ >> OpenStack-dev mailing list >> OpenStack-dev@lists.openstack.org >> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > > > ___ > OpenStack-dev mailing list > OpenStack-dev@lists.openstack.org > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [nova] Server Groups - remove VM from group?
I agree, Chris. I think a number of folks put in a lot of really great work into the existing server groups and there has been a lot of interest on their usage, especially given that the scheduler already has some constructs in place to piggyback on them. I would like to craft up a blueprint proposal for Kilo to add two simple extensions to the existing server group APIs that I believe will make them infinitely more usable in any ‘real world’ scenario. I’ll put more details in the proposal, but in a nutshell: 1. Adding a VM to a server group Only allow it to succeed if its policy wouldn’t be violated by the addition of the VM 2. Removing a VM from a server group Just allow it I think this would round out the support that’s there and really allow us to capitalize on the hard work everyone’s already put into them. - Joe On Aug 26, 2014, at 6:39 PM, Chris Friesen wrote: > On 08/25/2014 11:25 AM, Joe Cropper wrote: >> I was thinking something simple such as only allowing the add >> operation to succeed IFF no policies are found to be in violation... >> and then nova wouldn't need to get into all the complexities you >> mention? > > Personally I would be in favour of this...nothing fancy, just add it if it > already meets all the criteria. This is basically just a database operation > so I would hope we could make it reliable in the face of simultaneous things > going on with the instance. > >> And remove would be fairly straightforward as well since no >> constraints would need to be checked. > > Agreed. > > Chris > > ___ > OpenStack-dev mailing list > OpenStack-dev@lists.openstack.org > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [nova] Server Groups - remove VM from group?
On 08/25/2014 11:25 AM, Joe Cropper wrote: I was thinking something simple such as only allowing the add operation to succeed IFF no policies are found to be in violation... and then nova wouldn't need to get into all the complexities you mention? Personally I would be in favour of this...nothing fancy, just add it if it already meets all the criteria. This is basically just a database operation so I would hope we could make it reliable in the face of simultaneous things going on with the instance. And remove would be fairly straightforward as well since no constraints would need to be checked. Agreed. Chris ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [nova] Server Groups - remove VM from group?
> Even something like this is a lot more complicated than it sounds due to the fact that several operations can be happening in parallel. That's fair, but I was thinking that the 'add existing' VM is fairly close in behavior to 'add new' VM to the group, less of course any parallel operations happening on the VM itself. > I think we just need to draw a line for Nova that just doesn't include this functionality. If that's our general direction, no problem. I'm just thinking about this from a user's perspective; this would be very difficult for any administrator to use in its current form because you essentially can't make a mistake in the group management--any mistake results in you having to essentially delete the VM and start over, which is a pretty major usability issue IMO, at least in terms of most production environments. Don't get me wrong, I think server groups have a lot of interesting use cases (I actually would really like to use them) in their current form and I think as a starting point, this is great. But I think without some of these added flexibilities, I think it would be very challenging for any IT administrator to use them--hence why I'm exploring adding some additional functionality; I'm even happy to help implement this, if we can get any type of concurrence on the subject. :-) - Joe On Mon, Aug 25, 2014 at 12:58 PM, Russell Bryant wrote: > On 08/25/2014 01:25 PM, Joe Cropper wrote: >> I was thinking something simple such as only allowing the add operation to >> succeed IFF no policies are found to be in violation... and then nova >> wouldn't need to get into all the complexities you mention? > > Even something like this is a lot more complicated than it sounds due to > the fact that several operations can be happening in parallel. I think > we just need to draw a line for Nova that just doesn't include this > functionality. > >> And remove would be fairly straightforward as well since no constraints >> would need to be checked. > > Right, remove is straight forward, but seems a bit odd to have without > add. I'm not sure there's much value to it. > > -- > Russell Bryant > > ___ > OpenStack-dev mailing list > OpenStack-dev@lists.openstack.org > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [nova] Server Groups - remove VM from group?
On 08/25/2014 01:25 PM, Joe Cropper wrote: > I was thinking something simple such as only allowing the add operation to > succeed IFF no policies are found to be in violation... and then nova > wouldn't need to get into all the complexities you mention? Even something like this is a lot more complicated than it sounds due to the fact that several operations can be happening in parallel. I think we just need to draw a line for Nova that just doesn't include this functionality. > And remove would be fairly straightforward as well since no constraints would > need to be checked. Right, remove is straight forward, but seems a bit odd to have without add. I'm not sure there's much value to it. -- Russell Bryant ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [nova] Server Groups - remove VM from group?
I was thinking something simple such as only allowing the add operation to succeed IFF no policies are found to be in violation... and then nova wouldn't need to get into all the complexities you mention? And remove would be fairly straightforward as well since no constraints would need to be checked. Thoughts? Thanks, Joe > On Aug 25, 2014, at 12:10 PM, Russell Bryant wrote: > >> On 08/25/2014 12:56 PM, Joe Cropper wrote: >> That was indeed a rather long (and insightful) thread on the topic. >> It sounds like there are still some healthy discussions worth having >> on the subject -- either exploring your [potentially superseding] >> proposal, or minimally rounding out the existing server group API to >> support "add existing VM" [1] and "remove VM" -- I think these would >> make it a lot more usable (I'm thinking of the poor cloud >> administrator that makes a mistake when they boot an instance and >> either forgets to put it in a group or puts it in the wrong group -- >> it's square 1 for them)? >> >> Is this queued up as a discussion point for Paris? If so, count me in! > > Adding a VM is far from trivial and is why we ripped it out before > merging. That implies a potential reshuffling of a bunch of existing > VMs. Consider an affinity group of instances A and B and then you add > running instance C to that group. What do you expect to happen? Live > migrate C to the host running A and B? What if there isn't room? > Reschedule all 3 to find a host and live migrate all of them? This kind > of orchestration is a good bit outside of the scope of what's done > inside of Nova today. > > -- > Russell Bryant > > ___ > OpenStack-dev mailing list > OpenStack-dev@lists.openstack.org > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [nova] Server Groups - remove VM from group?
On 08/25/2014 12:56 PM, Joe Cropper wrote: > That was indeed a rather long (and insightful) thread on the topic. > It sounds like there are still some healthy discussions worth having > on the subject -- either exploring your [potentially superseding] > proposal, or minimally rounding out the existing server group API to > support "add existing VM" [1] and "remove VM" -- I think these would > make it a lot more usable (I'm thinking of the poor cloud > administrator that makes a mistake when they boot an instance and > either forgets to put it in a group or puts it in the wrong group -- > it's square 1 for them)? > > Is this queued up as a discussion point for Paris? If so, count me in! Adding a VM is far from trivial and is why we ripped it out before merging. That implies a potential reshuffling of a bunch of existing VMs. Consider an affinity group of instances A and B and then you add running instance C to that group. What do you expect to happen? Live migrate C to the host running A and B? What if there isn't room? Reschedule all 3 to find a host and live migrate all of them? This kind of orchestration is a good bit outside of the scope of what's done inside of Nova today. -- Russell Bryant ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [nova] Server Groups - remove VM from group?
That was indeed a rather long (and insightful) thread on the topic. It sounds like there are still some healthy discussions worth having on the subject -- either exploring your [potentially superseding] proposal, or minimally rounding out the existing server group API to support "add existing VM" [1] and "remove VM" -- I think these would make it a lot more usable (I'm thinking of the poor cloud administrator that makes a mistake when they boot an instance and either forgets to put it in a group or puts it in the wrong group -- it's square 1 for them)? Is this queued up as a discussion point for Paris? If so, count me in! Thanks, Joe On Mon, Aug 25, 2014 at 11:08 AM, Jay Pipes wrote: > On 08/25/2014 11:31 AM, Joe Cropper wrote: >> >> Thanks Jay. Those are the same types of questions I was pondering as >> well when debating how someone might use this. I think what we have >> is fine for a first pass, but that's what I was poking at... whether >> some of the abilities to add/remove members dynamically could exist >> (e.g., I no longer want this VM to have an anti-affinity policy >> relative to the others, etc.). > > > I guess what I was getting at is that I think the whole interface is flawed > and it's not worth putting in the effort to make it slightly less flawed. > > Best, > -jay > > >> - Joe >> >> On Mon, Aug 25, 2014 at 10:16 AM, Jay Pipes wrote: >>> >>> On 08/25/2014 11:10 AM, Joe Cropper wrote: Hello, Is our long-term vision to allow a VMs to be dynamically added/removed from a group? That is, unless I'm overlooking something, it appears that you can only add a VM to a server group at VM boot time and effectively remove it by deleting the VM? Just curious if this was a design point, or merely an approach at a staged implementation [that might welcome some additions]? :) >>> >>> >>> >>> See here: >>> >>> http://lists.openstack.org/pipermail/openstack-dev/2014-April/033746.html >>> >>> If I had my druthers, I would revert the whole extension. >>> >>> -jay >>> >>> ___ >>> OpenStack-dev mailing list >>> OpenStack-dev@lists.openstack.org >>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev >> >> >> ___ >> OpenStack-dev mailing list >> OpenStack-dev@lists.openstack.org >> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev >> > > > ___ > OpenStack-dev mailing list > OpenStack-dev@lists.openstack.org > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [nova] Server Groups - remove VM from group?
On 08/25/2014 11:31 AM, Joe Cropper wrote: Thanks Jay. Those are the same types of questions I was pondering as well when debating how someone might use this. I think what we have is fine for a first pass, but that's what I was poking at... whether some of the abilities to add/remove members dynamically could exist (e.g., I no longer want this VM to have an anti-affinity policy relative to the others, etc.). I guess what I was getting at is that I think the whole interface is flawed and it's not worth putting in the effort to make it slightly less flawed. Best, -jay - Joe On Mon, Aug 25, 2014 at 10:16 AM, Jay Pipes wrote: On 08/25/2014 11:10 AM, Joe Cropper wrote: Hello, Is our long-term vision to allow a VMs to be dynamically added/removed from a group? That is, unless I'm overlooking something, it appears that you can only add a VM to a server group at VM boot time and effectively remove it by deleting the VM? Just curious if this was a design point, or merely an approach at a staged implementation [that might welcome some additions]? :) See here: http://lists.openstack.org/pipermail/openstack-dev/2014-April/033746.html If I had my druthers, I would revert the whole extension. -jay ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [nova] Server Groups - remove VM from group?
Thanks Jay. Those are the same types of questions I was pondering as well when debating how someone might use this. I think what we have is fine for a first pass, but that's what I was poking at... whether some of the abilities to add/remove members dynamically could exist (e.g., I no longer want this VM to have an anti-affinity policy relative to the others, etc.). - Joe On Mon, Aug 25, 2014 at 10:16 AM, Jay Pipes wrote: > On 08/25/2014 11:10 AM, Joe Cropper wrote: >> >> Hello, >> >> Is our long-term vision to allow a VMs to be dynamically added/removed >> from a group? That is, unless I'm overlooking something, it appears >> that you can only add a VM to a server group at VM boot time and >> effectively remove it by deleting the VM? >> >> Just curious if this was a design point, or merely an approach at a >> staged implementation [that might welcome some additions]? :) > > > See here: > > http://lists.openstack.org/pipermail/openstack-dev/2014-April/033746.html > > If I had my druthers, I would revert the whole extension. > > -jay > > ___ > OpenStack-dev mailing list > OpenStack-dev@lists.openstack.org > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [nova] Server Groups - remove VM from group?
On 08/25/2014 11:10 AM, Joe Cropper wrote: Hello, Is our long-term vision to allow a VMs to be dynamically added/removed from a group? That is, unless I'm overlooking something, it appears that you can only add a VM to a server group at VM boot time and effectively remove it by deleting the VM? Just curious if this was a design point, or merely an approach at a staged implementation [that might welcome some additions]? :) See here: http://lists.openstack.org/pipermail/openstack-dev/2014-April/033746.html If I had my druthers, I would revert the whole extension. -jay ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] [nova] Server Groups - remove VM from group?
Hello, Is our long-term vision to allow a VMs to be dynamically added/removed from a group? That is, unless I'm overlooking something, it appears that you can only add a VM to a server group at VM boot time and effectively remove it by deleting the VM? Just curious if this was a design point, or merely an approach at a staged implementation [that might welcome some additions]? :) Thanks, Joe ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
