Re: [openstack-dev] [nova] RequestSpec questions about force_hosts/nodes and requested_destination
2017-08-30 15:43 GMT+02:00 Sylvain Bauza : > Still on PTO for one day, but I'll try to answer. Sorry for the delay, was > out traveling. > > > 2017-08-21 23:09 GMT+02:00 Matt Riedemann : > >> I don't dabble in the RequestSpec code much, but in trying to fix bug >> 1712008 [1] I'm venturing in there and have some questions. This is mostly >> an email to Sylvain for when he gets back from vacation but I wanted to >> dump it before moving forward. >> >> > Heh :-) > > >> Mainly, what is the difference between RequestSpec.force_hosts/force_nodes >> and RequestSpec.requested_destination? >> >> > force_[hosts/nodes] was a former key in the legacy filter_properties > dictionary that was passed to the scheduler. When I designed the > RequestSpec object, I just provided the same names for the fields that were > related to those keys. > requested_destination is a new field that wasn't previously in the legacy > dictionaries but was just added when I wrote in Newton a new feature about > verifying a target when moving. > > There is a main behavioural difference between those two RequestSpec > fields, that is because the behaviours are different between a boot request > and a move operation. > When you boot an instance, you can pass (weirdly and terribly from an API > point) a target for that instance by using the availability_zone flag. Some > request like nova boot inst1 --availability_zone nova:foo:bar will > actually call the scheduler to just verify if host "foo" (and node "bar") > is alive *without* verifying filters. > When you move that instance by passing a target (something like "nova > live-migrate inst1 foo2") will rather call the scheduler to verify only > that host (and not the others) *by running filters*. > > TBC, if you 'force' a boot, it won't be a real forced boot because it will > still call the scheduler but it will dumbly accept the proposed destination. > If you want to *propose* a destination for a move, it will ask the > scheduler to *really* verify that destination (or end up the migration if > not possible). You can still want to *force* a move, but in that case, it > just bypasses the scheduler. > > > >> When should one be used over the other? I take it that >> requested_destination is the newest and coolest thing and we should always >> use that first, and that's what the nova-api code is using, but I also see >> the scheduler code checking force_hosts/force_nodes. >> >> > MHO is that force_hosts is just a technical hack based on an unclear API > contract about guarantting a boot request to succeed against a specific > target. I'd rather propose a new API contract for providing a destination > exactly based on the same semantics that the ones we have for move > operations, ie. : > - if a destination is proposed, run scheduler filters only against that > destination and error out the boot request if the host can't sustain the > request. > - if a destination is proposed and a 'force' flag is used, then just pass > the RPC request to the target compute service and magically expect > everything will work. > > That said, there is a corner case now we have allocations made by the > scheduler. If we go straight to the compute service, we won't create the > allocation if a force operation is made. In that case, something (that has > to be discussed in the spec mentioning the API change) has to reconcile > allocations for the compute service. Maybe it's just a technical detail, > but that has to be written in the spec tho. > > Huh, haven't looked at the bug before writing the email. /me facepalms. Yeah, the bug you wrote is the exact problem I mentioned above... > Is that all legacy compatibility code? And if so, then why don't we handle >> requested_destination in RequestSpec routines like >> reset_forced_destinations() and to_legacy_filter_properties_dict(), i.e. >> for the latter, if it's a new style RequestSpec with requested_destination >> set, but we have to backport and call to_legacy_filter_properties_dict(), >> shouldn't requested_destination be used to set force_hosts/force_nodes on >> the old style filter properties? >> >> Since RequestSpec.requested_destination is the thing that restricts a >> move operation to a single cell, it seems pretty important to always be >> using that field when forcing where an instance is moving to. But I'm >> confused about whether or not both requested_destination *and* >> force_hosts/force_nodes should be set since the compat code doesn't seem to >> transform the former into the latter. >> >> > I agree, that sounds important to me to have consistent behaviours between > all nova operations that require a placement decision. I have that in my > pipe for a very long time (I mean, writing the spec for modifying the API > contract on instance creation) so I push it a bit more. > > >> If this is all transitional code, we should really document the hell out >> of this in the RequestSpec class itself for anyone trying to write new >> client side code with i
Re: [openstack-dev] [nova] RequestSpec questions about force_hosts/nodes and requested_destination
Still on PTO for one day, but I'll try to answer. Sorry for the delay, was out traveling. 2017-08-21 23:09 GMT+02:00 Matt Riedemann : > I don't dabble in the RequestSpec code much, but in trying to fix bug > 1712008 [1] I'm venturing in there and have some questions. This is mostly > an email to Sylvain for when he gets back from vacation but I wanted to > dump it before moving forward. > > Heh :-) > Mainly, what is the difference between RequestSpec.force_hosts/force_nodes > and RequestSpec.requested_destination? > > force_[hosts/nodes] was a former key in the legacy filter_properties dictionary that was passed to the scheduler. When I designed the RequestSpec object, I just provided the same names for the fields that were related to those keys. requested_destination is a new field that wasn't previously in the legacy dictionaries but was just added when I wrote in Newton a new feature about verifying a target when moving. There is a main behavioural difference between those two RequestSpec fields, that is because the behaviours are different between a boot request and a move operation. When you boot an instance, you can pass (weirdly and terribly from an API point) a target for that instance by using the availability_zone flag. Some request like nova boot inst1 --availability_zone nova:foo:bar will actually call the scheduler to just verify if host "foo" (and node "bar") is alive *without* verifying filters. When you move that instance by passing a target (something like "nova live-migrate inst1 foo2") will rather call the scheduler to verify only that host (and not the others) *by running filters*. TBC, if you 'force' a boot, it won't be a real forced boot because it will still call the scheduler but it will dumbly accept the proposed destination. If you want to *propose* a destination for a move, it will ask the scheduler to *really* verify that destination (or end up the migration if not possible). You can still want to *force* a move, but in that case, it just bypasses the scheduler. > When should one be used over the other? I take it that > requested_destination is the newest and coolest thing and we should always > use that first, and that's what the nova-api code is using, but I also see > the scheduler code checking force_hosts/force_nodes. > > MHO is that force_hosts is just a technical hack based on an unclear API contract about guarantting a boot request to succeed against a specific target. I'd rather propose a new API contract for providing a destination exactly based on the same semantics that the ones we have for move operations, ie. : - if a destination is proposed, run scheduler filters only against that destination and error out the boot request if the host can't sustain the request. - if a destination is proposed and a 'force' flag is used, then just pass the RPC request to the target compute service and magically expect everything will work. That said, there is a corner case now we have allocations made by the scheduler. If we go straight to the compute service, we won't create the allocation if a force operation is made. In that case, something (that has to be discussed in the spec mentioning the API change) has to reconcile allocations for the compute service. Maybe it's just a technical detail, but that has to be written in the spec tho. > Is that all legacy compatibility code? And if so, then why don't we handle > requested_destination in RequestSpec routines like > reset_forced_destinations() and to_legacy_filter_properties_dict(), i.e. > for the latter, if it's a new style RequestSpec with requested_destination > set, but we have to backport and call to_legacy_filter_properties_dict(), > shouldn't requested_destination be used to set force_hosts/force_nodes on > the old style filter properties? > > Since RequestSpec.requested_destination is the thing that restricts a > move operation to a single cell, it seems pretty important to always be > using that field when forcing where an instance is moving to. But I'm > confused about whether or not both requested_destination *and* > force_hosts/force_nodes should be set since the compat code doesn't seem to > transform the former into the latter. > > I agree, that sounds important to me to have consistent behaviours between all nova operations that require a placement decision. I have that in my pipe for a very long time (I mean, writing the spec for modifying the API contract on instance creation) so I push it a bit more. > If this is all transitional code, we should really document the hell out > of this in the RequestSpec class itself for anyone trying to write new > client side code with it, like me. > > That sounds a very good short win to me. -Sylvain > [1] https://bugs.launchpad.net/nova/+bug/1712008 > > -- > > Thanks, > > Matt > > __ > OpenStack Development Mailing List (not for usage questions) > Unsubscribe: openstack-dev-requ...@lists.
[openstack-dev] [nova] RequestSpec questions about force_hosts/nodes and requested_destination
I don't dabble in the RequestSpec code much, but in trying to fix bug 1712008 [1] I'm venturing in there and have some questions. This is mostly an email to Sylvain for when he gets back from vacation but I wanted to dump it before moving forward. Mainly, what is the difference between RequestSpec.force_hosts/force_nodes and RequestSpec.requested_destination? When should one be used over the other? I take it that requested_destination is the newest and coolest thing and we should always use that first, and that's what the nova-api code is using, but I also see the scheduler code checking force_hosts/force_nodes. Is that all legacy compatibility code? And if so, then why don't we handle requested_destination in RequestSpec routines like reset_forced_destinations() and to_legacy_filter_properties_dict(), i.e. for the latter, if it's a new style RequestSpec with requested_destination set, but we have to backport and call to_legacy_filter_properties_dict(), shouldn't requested_destination be used to set force_hosts/force_nodes on the old style filter properties? Since RequestSpec.requested_destination is the thing that restricts a move operation to a single cell, it seems pretty important to always be using that field when forcing where an instance is moving to. But I'm confused about whether or not both requested_destination *and* force_hosts/force_nodes should be set since the compat code doesn't seem to transform the former into the latter. If this is all transitional code, we should really document the hell out of this in the RequestSpec class itself for anyone trying to write new client side code with it, like me. [1] https://bugs.launchpad.net/nova/+bug/1712008 -- Thanks, Matt __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
