On Aug 12, 2014, at 5:21 AM, Nikola Đipanov <ndipa...@redhat.com> wrote:
> Hey Nova-istas, > > While I was hacking on [1] I was considering how to approach the fact > that we now need to track one more thing (NUMA node utilization) in our > resources. I went with - "I'll add it to compute nodes table" thinking > it's a fundamental enough property of a compute host that it deserves to > be there, although I was considering Extensible Resource Tracker at one > point (ERT from now on - see [2]) but looking at the code - it did not > seem to provide anything I desperately needed, so I went with keeping it > simple. > > So fast-forward a few days, and I caught myself solving a problem that I > kept thinking ERT should have solved - but apparently hasn't, and I > think it is fundamentally a broken design without it - so I'd really > like to see it re-visited. > > The problem can be described by the following lemma (if you take 'lemma' > to mean 'a sentence I came up with just now' :)): > > """ > Due to the way scheduling works in Nova (roughly: pick a host based on > stale(ish) data, rely on claims to trigger a re-schedule), _same exact_ > information that scheduling service used when making a placement > decision, needs to be available to the compute service when testing the > placement. > “"" Correct > > This is not the case right now, and the ERT does not propose any way to > solve it - (see how I hacked around needing to be able to get > extra_specs when making claims in [3], without hammering the DB). The > result will be that any resource that we add and needs user supplied > info for scheduling an instance against it, will need a buggy > re-implementation of gathering all the bits from the request that > scheduler sees, to be able to work properly. Agreed, ERT does not attempt to solve this problem of ensuring RT has an identical set of information for testing claims. I don’t think it was intended to. ERT does solve the issue of bloat in the RT with adding just-one-more-thing to test usage-wise. It gives a nice hook for inserting your claim logic for your specific use case. > > This is obviously a bigger concern when we want to allow users to pass > data (through image or flavor) that can affect scheduling, but still a > huge concern IMHO. I think passing additional data through to compute just wasn’t a problem that ERT aimed to solve. (Paul Murray?) That being said, coordinating the passing of any extra data required to test a claim that is *not* sourced from the host itself would be a very nice addition. You are working around it with some caching in your flavor db lookup use case, although one could of course cook up a cleaner patch to pass such data through on the “build this” request to the compute. > > As I see that there are already BPs proposing to use this IMHO broken > ERT ([4] for example), which will surely add to the proliferation of > code that hacks around these design shortcomings in what is already a > messy, but also crucial (for perf as well as features) bit of Nova code. > > I propose to revert [2] ASAP since it is still fresh, and see how we can > come up with a cleaner design. > I think the ERT is forward-progress here, but am willing to review patches/specs on improvements/replacements. > Would like to hear opinions on this, before I propose the patch tho! > > Thanks all, > > Nikola > > [1] https://blueprints.launchpad.net/nova/+spec/virt-driver-numa-placement > [2] https://review.openstack.org/#/c/109643/ > [3] https://review.openstack.org/#/c/111782/ > [4] https://review.openstack.org/#/c/89893 > > _______________________________________________ > OpenStack-dev mailing list > OpenStack-dev@lists.openstack.org > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev _______________________________________________ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev