Le 03/06/2015 15:15, Nikola Đipanov a écrit :
On 06/02/2015 03:14 PM, Sylvain Bauza wrote:
Hi,
Currently working on implementing the RequestSpec object BP [1], I had
some cool comments on my change here :
https://review.openstack.org/#/c/145528/12/nova/objects/request_spec.py,cm
Since we didn't discussed on how to persist that RequestSpec object, I
think the comment is valuable.
For the moment, the only agreed spec for persisting the object that we
have is [2] but there is also a corollar here that means that we would
have to persist more than the current fields
https://review.openstack.org/#/c/169901/3/specs/liberty/approved/add-buildrequest-obj.rst,cm
So, there are 2 possibilities :
#1, we only persist the RequestSpec for the sole Scheduler and in that
case, we can leave as it is - only a few fields from Instance are stored
#2, we consider that RequestSpec can be used for more than just the
Scheduler, and then we need to make sure that we will have all the
instance fields then.
So these are 2 possibilities if we agree that we need to make progress
on the spec as is defined and merged now. What I was complaining
yesterday is that we don't seem to have done enough of high level
investigation into this stuff before embarking on writing a set of specs
that then due to their format obscure the problems we are actually
trying to solve.
Since Nova is big, it's pretty hard to take the big picture of all the
problems that we have and provide a spec which can be fine grained
enough for making sure it will take in account the overall problem.
At least, I'm seriously considering the RequestSpec object and how it is
persisted as a first attempt to version a Scheduler API and provide an
upgrade path for changing it.
Work around the scheduler touches on a lot of issues that have only
recently been noticed. While I am all for the incremental approach, it
seems silly to completely disregard the issues we already know about. We
should have a high level overview of the problems we know we want to
solve, and then come up with an incremental way of solving them, but not
without keeping an eye on the big picture at all times.
True story. That's why the biggest question I have is "do I actually add
more technical debt to the project by writing RequestSpec, or is it
something that helps reducing the debt ?"
So, that's why I'm very open to any comments explaining which kind of
problem the current proposal could raise, and how we could prevent that.
IIUC, your concern is about duplicating some information within the
RequestSpec object which would be persisted. My point is to consider
that if the usage is well defined (ie. only a contract between conductor
and scheduler, and not reused anywhere else) then that's a reasonable
trade-off.
An ad-hoc list of individual issues that we know about and should be
trying to solve (in no particular order) that all seem related to the
data model design problem we are trying to take a stab at here:
1/ RequestSpec is an unversioned dict even though it's the central piece
of a placement request for the scheduler
Targeted by
http://specs.openstack.org/openstack/nova-specs/specs/liberty/approved/request-spec-object.html
2/ There are scheduler_hints that are needed throughout the lifecycle of
an instance but are never persisted so are lost after boot
Targeted by
http://specs.openstack.org/openstack/nova-specs/specs/liberty/approved/persist-request-spec.html
3/ We have the Migration objects that are used both for resource
tracking for instances being migrated, and as an indication of an
instance being moved, but are not used in all the places we need this
kind of book keeping (live migration, rebuild)
http://specs.openstack.org/openstack/nova-specs/specs/liberty/approved/robustify_evacuate.html
is aiming to provide that evacuations, should we also propose to track
rebuilds and live migrations as well ?
4/ Evacuate (an orchestrated rebuild) is especially problematic because
it usually involves failure modes, which are difficult to identify and
handle properly without a consistently used data model.
Could you please describe more the problem ? Is it due to the fact that
we're not persisting the request spec so it's basically a wet finger for
knowing if that's good or not ?
5/ Some of the recently added constraints that influence resource
tracking (NUMA, CPU pinning) cannot simply be calculated from the flavor
on the fly when tracking resources, but need to be persisted after a
successful claim as they are dependent on the state of the host at that
very moment (see [1])
I just wonder if a scheduler API called 'check_destination(RequestSpec,
destination) could help ? I mean, when live-migrating, we know that we
try to check what we can, but since neither the spec is persisted, nor
the migration is verifying that the destination host can fulfill the
request, there are many bugs with that.
FYI, I tried to propose that https://review.openstack.org/#/c/179224/
which still needs to be approved. Comments welcome. As I said in the
spec proposal, it's half of the problem because the spec is not
persisted yet. Once we will have *both* a destination check and a
persisted spec, we should be able to verify those assertions, nope ?
6/ Related to the previous one - there is data related to the instance
in addition to the flavor that need to follow the '_old' and '_new'
pattern (needs the values related to both source and destination host
persisted during a migration/resize/live migration/)
Could you please describe more the problem again ? Do we have a problem
with the resource usage ? Do we have a bug which describes it ?
7/ The issues cells v2 folks are hitting (mentioned above) where they
don't want to have any Instances in the top level cell but still need to
persist stuff.
Agreed, that's why Andrew Laski proposed to amend
http://specs.openstack.org/openstack/nova-specs/specs/liberty/approved/persist-request-spec.html
to take in account the cells problem.
8/ Issues with having no access to individual instance UUIDs in the
scheduler, but a lot of data access for more complex filtering revolves
around it being present.
That's exactly why the objectification is necessary, because it will
stabilize what we provide to the Scheduler IMHO.
Most of the above have individual bugs that I can try to find and link
here too.
[1] https://bugs.launchpad.net/nova/+bug/1417667
The overall theme of all the above is (to paraphrase alaski from IRC)
how to organize the big blob of data that is an instance in all of it's
possible states, in such a way that it makes sense, nothing is missing,
there is as little duplication as possible, and access patterns of
different services that require different bits can work without massive
overhead.
+1 I think your concern is totally valid, I'm just trying to provide
some efforts that, once linked, will make Nova more stronger than it is now.
I'm not strongly opiniated on that, I maybe consider that #2 is probably
the best option but there is a tie in my mind. Help me figuring out
what's the best option.
If we want to keep things moving forward on this particular BP - I'd go
with adding the RequestSpec object and make sure the code that uses it
is migrated. I believe that spike alone will leave us with much better
idea about the problem.
In addition - writing a high level spec/wiki that we can refer back to
in individual BPs and see how they solve it would be massively helpful too.
Cool, do you want me to write this one ?
Thanks for all your insights, greatly appreciated.
-Sylvain
N.
-Sylvain
[1] :
http://specs.openstack.org/openstack/nova-specs/specs/liberty/approved/request-spec-object.html
[2] :
http://specs.openstack.org/openstack/nova-specs/specs/liberty/approved/persist-request-spec.html
__________________________________________________________________________
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
__________________________________________________________________________
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
__________________________________________________________________________
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
