On Aug 12, 2014, at 5:21 AM, Nikola Đipanov <ndipa...@redhat.com> wrote:

> Hey Nova-istas,
> While I was hacking on [1] I was considering how to approach the fact
> that we now need to track one more thing (NUMA node utilization) in our
> resources. I went with - "I'll add it to compute nodes table" thinking
> it's a fundamental enough property of a compute host that it deserves to
> be there, although I was considering  Extensible Resource Tracker at one
> point (ERT from now on - see [2]) but looking at the code - it did not
> seem to provide anything I desperately needed, so I went with keeping it
> simple.
> So fast-forward a few days, and I caught myself solving a problem that I
> kept thinking ERT should have solved - but apparently hasn't, and I
> think it is fundamentally a broken design without it - so I'd really
> like to see it re-visited.
> The problem can be described by the following lemma (if you take 'lemma'
> to mean 'a sentence I came up with just now' :)):
> """
> Due to the way scheduling works in Nova (roughly: pick a host based on
> stale(ish) data, rely on claims to trigger a re-schedule), _same exact_
> information that scheduling service used when making a placement
> decision, needs to be available to the compute service when testing the
> placement.
> “""


> This is not the case right now, and the ERT does not propose any way to
> solve it - (see how I hacked around needing to be able to get
> extra_specs when making claims in [3], without hammering the DB). The
> result will be that any resource that we add and needs user supplied
> info for scheduling an instance against it, will need a buggy
> re-implementation of gathering all the bits from the request that
> scheduler sees, to be able to work properly.
Agreed, ERT does not attempt to solve this problem of ensuring RT has an 
identical set of information for testing claims.  I don’t think it was intended 

ERT does solve the issue of bloat in the RT with adding just-one-more-thing to 
test usage-wise.  It gives a nice hook for inserting your claim logic for your 
specific use case.

> This is obviously a bigger concern when we want to allow users to pass
> data (through image or flavor) that can affect scheduling, but still a
> huge concern IMHO.
I think passing additional data through to compute just wasn’t a problem that 
ERT aimed to solve.  (Paul Murray?)  That being said, coordinating the passing 
of any extra data required to test a claim that is *not* sourced from the host 
itself would be a very nice addition.  You are working around it with some 
caching in your flavor db lookup use case, although one could of course cook up 
a cleaner patch to pass such data through on the “build this” request to the 

> As I see that there are already BPs proposing to use this IMHO broken
> ERT ([4] for example), which will surely add to the proliferation of
> code that hacks around these design shortcomings in what is already a
> messy, but also crucial (for perf as well as features) bit of Nova code.
> I propose to revert [2] ASAP since it is still fresh, and see how we can
> come up with a cleaner design.
I think the ERT is forward-progress here, but am willing to review 
patches/specs on improvements/replacements.  

> Would like to hear opinions on this, before I propose the patch tho!
> Thanks all,
> Nikola
> [1] https://blueprints.launchpad.net/nova/+spec/virt-driver-numa-placement
> [2] https://review.openstack.org/#/c/109643/
> [3] https://review.openstack.org/#/c/111782/
> [4] https://review.openstack.org/#/c/89893
> _______________________________________________
> OpenStack-dev mailing list
> OpenStack-dev@lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

OpenStack-dev mailing list

Reply via email to