Re: [openstack-dev] [Nova] [Gantt] Scheduler split status (updated)
On 16 July 2014 03:45, Debojyoti Dutta ddu...@gmail.com wrote: https://etherpad.openstack.org/p/SchedulerUseCases [08:43:35] n0ano #action all update the use case etherpad athttps://etherpad.openstack.org/p/SchedulerUseCases Please update your use cases here .. Added. -Rob -- Robert Collins rbtcoll...@hp.com Distinguished Technologist HP Converged Cloud ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Nova] [Gantt] Scheduler split status (updated)
Le 17/07/2014 01:24, Robert Collins a écrit : On 15 July 2014 06:10, Jay Pipes jaypi...@gmail.com wrote: Frankly, I don't think a lot of the NFV use cases are well-defined. Even more frankly, I don't see any benefit to a split-out scheduler to a single NFV use case. Don't you see each Summit the lots of talks (and people attending them) talking about how OpenStack should look at Pets vs. Cattle and saying that the scheduler should be out of Nova ? There's been no concrete benefits discussed to having the scheduler outside of Nova. I don't really care how many people say that the scheduler should be out of Nova unless those same people come to the table with concrete reasons why. Just saying something is a benefit does not make it a benefit, and I think I've outlined some of the very real dangers -- in terms of code and payload complexity -- of breaking the scheduler out of Nova until the interfaces are cleaned up and the scheduler actually owns the resources upon which it exercises placement decisions. I agree with the risks if we get it wrong. In terms of benefits, I want to do cross-domain scheduling: 'Give me five Galera servers with no shared non-HA infrastructure and resiliency to no less than 2 separate failures'. By far the largest push back I get is 'how do I make Ironic pick the servers I want it to' when talking to ops folk about using Ironic. And when you dig into that, it falls into two buckets: - role based mappings (e.g. storage optimised vs cpu optimised) - which Ironic can trivially do - failure domain and performance domain optimisation - which Nova cannot do at all today. I want this very very very badly, and I would love to be pushing directly on it, but its just under a few other key features like 'working HA' and 'efficient updates' that sadly matter more in the short term. I share your views on what should be the scheduler, once pushed out of Nova. As I said, there are various concerns and asked features for the scheduler that are missing here and now, and which are too big to be fit in Nova. In order to make sure we're going into the right direction, we decided during last Gantt meeting to provide usecases where an external Scheduler could be interesting. Don't hesitate to add your baremetal usecases (for deployment with TripleO or others) in there : https://etherpad.openstack.org/p/SchedulerUseCases Take it as a first attempt to identify what would be the mission statement for Gantt, if you wish. Sorry, I'm not following you. Who is saying to Gantt I want to store this data? All I am saying is that the thing that places a resource on some provider of that resource should be the thing that owns the process of a requester *claiming* the resources on that provider, and in order to properly track resources in a race-free way in such a system, then the system needs to contain the resource tracker. Trying to translate that: - the scheduler (thing that places a resource) - should own the act of claiming a resource - to avoid races the scheduler should own the tracker So I think we need to aknowledge that Right Now we have massive races. We can choose where we put our efforts - we can try to fix them in the current architecture, we can try to fix them by changing the architecture. I think you agree that the current architecture is wrong; and that from a risk perspective the gantt extraction should not change the architecture - as part of making it graceful and cinder-like with immediate use by Nova. But once extracted the architecture can change - versioned APIs FTW. To my mind the key question is not whether the thing will be *better* with gantt extracted, it is whether it will be *no worse*, while simultaneously enabling a bunch of pent up demand in another part of the community. That seems hard to answer categorically, but it seems to me the key risk is whether changing the architecture will be too hard / unsafe post extraction. However in Nova it takes months and months to land things (and I'm not poking here - TripleO has the same issue at the moment) - I think there is a very real possibility that gantt can improve much faster and efficiently as a new project, once forklifted out. Patches to Nova to move to newer APIs can be posted and worked with while folk work on other bits of key plumbing like performance (e.g. not loading every host in the entire cloud into ram on every scheduling request), scalability (e.g. elegantly solving the current racy behaviour between different scheduler instances) and begin the work to expose the scheduler to neutron and cinder. Unless I misunderstood (and that happens, I'm badly human - and French -), I'm giving a +2 to your statement : yes, there are race conditions in the scheduler (and I saw the bug you filed and I'm hesitating to handle it now), yes the scheduler is not that perfect now, yes we should ensure that claiming a resource should be 'ACID'
Re: [openstack-dev] [Nova] [Gantt] Scheduler split status (updated)
Le 14/07/2014 20:10, Jay Pipes a écrit : On 07/14/2014 10:16 AM, Sylvain Bauza wrote: Le 12/07/2014 06:07, Jay Pipes a écrit : On 07/11/2014 07:14 AM, John Garbutt wrote: On 10 July 2014 16:59, Sylvain Bauza sba...@redhat.com wrote: Le 10/07/2014 15:47, Russell Bryant a écrit : On 07/10/2014 05:06 AM, Sylvain Bauza wrote: Hi all, === tl;dr: Now that we agree on waiting for the split prereqs to be done, we debate on if ResourceTracker should be part of the scheduler code and consequently Scheduler should expose ResourceTracker APIs so that Nova wouldn't own compute nodes resources. I'm proposing to first come with RT as Nova resource in Juno and move ResourceTracker in Scheduler for K, so we at least merge some patches by Juno. === Some debates occured recently about the scheduler split, so I think it's important to loop back with you all to see where we are and what are the discussions. Again, feel free to express your opinions, they are welcome. Where did this resource tracker discussion come up? Do you have any references that I can read to catch up on it? I would like to see more detail on the proposal for what should stay in Nova vs. be moved. What is the interface between Nova and the scheduler here? Oh, missed the most important question you asked. So, about the interface in between scheduler and Nova, the original agreed proposal is in the spec https://review.openstack.org/82133 (approved) where the Scheduler exposes : - select_destinations() : for querying the scheduler to provide candidates - update_resource_stats() : for updating the scheduler internal state (ie. HostState) Here, update_resource_stats() is called by the ResourceTracker, see the implementations (in review) https://review.openstack.org/82778 and https://review.openstack.org/104556. The alternative that has just been raised this week is to provide a new interface where ComputeNode claims for resources and frees these resources, so that all the resources are fully owned by the Scheduler. An initial PoC has been raised here https://review.openstack.org/103598 but I tried to see what would be a ResourceTracker proxified by a Scheduler client here : https://review.openstack.org/105747. As the spec hasn't been written, the names of the interfaces are not properly defined but I made a proposal as : - select_destinations() : same as above - usage_claim() : claim a resource amount - usage_update() : update a resource amount - usage_drop(): frees the resource amount Again, this is a dummy proposal, a spec has to written if we consider moving the RT. While I am not against moving the resource tracker, I feel we could move this to Gantt after the core scheduling has been moved. Big -1 from me on this, John. Frankly, I see no urgency whatsoever -- and actually very little benefit -- to moving the scheduler out of Nova. The Gantt project I think is getting ahead of itself by focusing on a split instead of focusing on cleaning up the interfaces between nova-conductor, nova-scheduler, and nova-compute. -1 on saying there is no urgency. Don't you see the NFV group saying each meeting what is the status of the scheduler split ? Frankly, I don't think a lot of the NFV use cases are well-defined. Even more frankly, I don't see any benefit to a split-out scheduler to a single NFV use case. I don't know if you can, but if you're interested in giving feedback to the NFV team, we do run weekly meeting on #openstack-meeting-alt every Wednesday 2pm UTC. You can find a list of all the associated blueprints here https://wiki.openstack.org/wiki/Teams/NFV#Active_Blueprints whose list is processed hourly by a backend script so it generates a Gerrit dashboard accessible here : http://nfv.russellbryant.net By saying that, you can find https://blueprints.launchpad.net/nova/+spec/solver-scheduler as a possible use-case for NFV. As Paul and Yathi said, there is a need for a global resource placement engine able to cope with both network and compute resources if we need to provide NFV use-cases, that appears to me quite clearly and that's why I joined the NFV team. Don't you see each Summit the lots of talks (and people attending them) talking about how OpenStack should look at Pets vs. Cattle and saying that the scheduler should be out of Nova ? There's been no concrete benefits discussed to having the scheduler outside of Nova. I don't really care how many people say that the scheduler should be out of Nova unless those same people come to the table with concrete reasons why. Just saying something is a benefit does not make it a benefit, and I think I've outlined some of the very real dangers -- in terms of code and payload complexity -- of breaking the scheduler out of Nova until the interfaces are cleaned up and the scheduler actually owns the resources upon which it exercises placement decisions. From an operator perspective, people waited so long for having a
Re: [openstack-dev] [Nova] [Gantt] Scheduler split status (updated)
https://etherpad.openstack.org/p/SchedulerUseCases [08:43:35] n0ano #action all update the use case etherpad athttps://etherpad.openstack.org/p/SchedulerUseCases Please update your use cases here .. thx debo On Tue, Jul 15, 2014 at 2:50 AM, Sylvain Bauza sba...@redhat.com wrote: Le 14/07/2014 20:10, Jay Pipes a écrit : On 07/14/2014 10:16 AM, Sylvain Bauza wrote: Le 12/07/2014 06:07, Jay Pipes a écrit : On 07/11/2014 07:14 AM, John Garbutt wrote: On 10 July 2014 16:59, Sylvain Bauza sba...@redhat.com wrote: Le 10/07/2014 15:47, Russell Bryant a écrit : On 07/10/2014 05:06 AM, Sylvain Bauza wrote: Hi all, === tl;dr: Now that we agree on waiting for the split prereqs to be done, we debate on if ResourceTracker should be part of the scheduler code and consequently Scheduler should expose ResourceTracker APIs so that Nova wouldn't own compute nodes resources. I'm proposing to first come with RT as Nova resource in Juno and move ResourceTracker in Scheduler for K, so we at least merge some patches by Juno. === Some debates occured recently about the scheduler split, so I think it's important to loop back with you all to see where we are and what are the discussions. Again, feel free to express your opinions, they are welcome. Where did this resource tracker discussion come up? Do you have any references that I can read to catch up on it? I would like to see more detail on the proposal for what should stay in Nova vs. be moved. What is the interface between Nova and the scheduler here? Oh, missed the most important question you asked. So, about the interface in between scheduler and Nova, the original agreed proposal is in the spec https://review.openstack.org/82133 (approved) where the Scheduler exposes : - select_destinations() : for querying the scheduler to provide candidates - update_resource_stats() : for updating the scheduler internal state (ie. HostState) Here, update_resource_stats() is called by the ResourceTracker, see the implementations (in review) https://review.openstack.org/82778 and https://review.openstack.org/104556. The alternative that has just been raised this week is to provide a new interface where ComputeNode claims for resources and frees these resources, so that all the resources are fully owned by the Scheduler. An initial PoC has been raised here https://review.openstack.org/103598 but I tried to see what would be a ResourceTracker proxified by a Scheduler client here : https://review.openstack.org/105747. As the spec hasn't been written, the names of the interfaces are not properly defined but I made a proposal as : - select_destinations() : same as above - usage_claim() : claim a resource amount - usage_update() : update a resource amount - usage_drop(): frees the resource amount Again, this is a dummy proposal, a spec has to written if we consider moving the RT. While I am not against moving the resource tracker, I feel we could move this to Gantt after the core scheduling has been moved. Big -1 from me on this, John. Frankly, I see no urgency whatsoever -- and actually very little benefit -- to moving the scheduler out of Nova. The Gantt project I think is getting ahead of itself by focusing on a split instead of focusing on cleaning up the interfaces between nova-conductor, nova-scheduler, and nova-compute. -1 on saying there is no urgency. Don't you see the NFV group saying each meeting what is the status of the scheduler split ? Frankly, I don't think a lot of the NFV use cases are well-defined. Even more frankly, I don't see any benefit to a split-out scheduler to a single NFV use case. I don't know if you can, but if you're interested in giving feedback to the NFV team, we do run weekly meeting on #openstack-meeting-alt every Wednesday 2pm UTC. You can find a list of all the associated blueprints here https://wiki.openstack.org/wiki/Teams/NFV#Active_Blueprints whose list is processed hourly by a backend script so it generates a Gerrit dashboard accessible here : http://nfv.russellbryant.net By saying that, you can find https://blueprints.launchpad.net/nova/+spec/solver-scheduler as a possible use-case for NFV. As Paul and Yathi said, there is a need for a global resource placement engine able to cope with both network and compute resources if we need to provide NFV use-cases, that appears to me quite clearly and that's why I joined the NFV team. Don't you see each Summit the lots of talks (and people attending them) talking about how OpenStack should look at Pets vs. Cattle and saying that the scheduler should be out of Nova ? There's been no concrete benefits discussed to having the scheduler outside of Nova. I don't really care how many people say that the scheduler should be out of Nova unless those same people come to the table with concrete reasons why. Just saying something is a benefit does not make it a benefit, and I think I've outlined some of the
Re: [openstack-dev] [Nova] [Gantt] Scheduler split status (updated)
Unfortunately, much as I agree with your sentiment (Death to MS Outlook) my IT overlords have pretty much forced me into using it. I still top post but try and use some copied context (typically by adding an `in re:' to be explicit) so you know what part of the long email I'm referring to. -- Don Dugger Censeo Toto nos in Kansa esse decisse. - D. Gale Ph: 303/443-3786 -Original Message- From: Jay Pipes [mailto:jaypi...@gmail.com] Sent: Tuesday, July 15, 2014 9:05 AM To: openstack-dev@lists.openstack.org Subject: Re: [openstack-dev] [Nova] [Gantt] Scheduler split status (updated) Hi Paul, thanks for your reply. Comments inline. BTW, is there any way to reply inline instead of top-posting? On these longer emails, it gets hard sometimes to follow your reply to specific things I mentioned (vs. what John G mentioned). Death to MS Outlook. On 07/14/2014 04:40 PM, Murray, Paul (HP Cloud) wrote: On extensible resource tracking Jay, I am surprised to hear you say no one has explained to you why there is an extensible resource tracking blueprint. It's simple, there was a succession of blueprints wanting to add data about this and that to the resource tracker and the scheduler and the database tables used to communicate. These included capabilities, all the stuff in the stats, rxtx_factor, the equivalent for cpu (which only works on one hypervisor I think), pci_stats and more were coming including, https://blueprints.launchpad.net/nova/+spec/network-bandwidth-entitleme nt https://blueprints.launchpad.net/nova/+spec/cpu-entitlement So, in short, your claim that there are no operators asking for additional stuff is simply not true. A few things about the above blueprints 1) Neither above blueprint is approved. 2) Neither above blueprint nor the extensible resource tracker blueprint contains a single *use case*. The blueprints are full of statements like We want to extend this model to add a measure of XXX and We propose a unified API to support YYY, however none of them actually contains a real use case. A use case is in the form of As a XXX user, I want to be able to YYY so that my ZZZ can do AAA. Blueprints without use cases are not necessarily things to be disregarded, but when the blueprint proposes a significant change in behaviour/design or a new feature, without specifying one or more use cases that are satisfied by the proposed spec, the blueprint is suspicious, in my mind. 3) The majority of the feature requests in the CPUEntitlement are enabled with the use of existing host aggregates and their cpu_allocation_ratios and Dan Berrange's work on adding NUMA topology aspects to the compute node and flavours. 4) In my previous emails, I was pretty specific that I had never met a single operator or admin that was sitting there tinkering with weight multipliers trying to control the placement of VMs in their cloud. When I talk about the *needless complexity* in the current scheduler design, I am talking specifically about the weigher multipliers. I can guarantee you that there isn't a single person out there sitting behind the scenes going Oooh, let me change my ram weigher multiplier from 1.0 to .675 and see what happens. It's just not something that is done -- that is way too low of a level for the Compute scheduler to be thinking at. The Nova scheduler *is not a process or thread scheduler*. Folks who think that the Nova scheduler should emulate the Linux kernel scheduling policies and strategies are thinking on *completely* the wrong level, IMO. We should be focusing on making the scheduler *simpler*, with admin users *easily* able to figure out how to control placement decisions fo r their host aggregates and, more importantly, allow *tenant-by-tenant sorting policies* [1] so that scheduling decisions for different classes of tenants can be controlled distinctly. Around about the Icehouse summit (I think) it was suggested that we should stop the obvious trend and add a way to make resource tracking extensible, similar to metrics, which had just been added as an extensible way of collecting on going usage data (because that was also wanted). OK, I do understand that. I actually think it would have been more appropriate to define real models for these new resource types instead of making it a free-for-all with too much ability to support out-of-tree custom, non-standard resource types, but I understand the idea behind it. The json blob you refer to was down to the bad experience of the compute_node_stats table implemented for stats - which had a particular performance hit because it required an expensive join. This was dealt with by removing the table and adding a string field to contain the data as a json blob. A pure performance optimization. Interesting. This is good to know (and would have been good to note on the ERT blueprint). The problem I have with this is that we are muddying the code and the DB schema
Re: [openstack-dev] [Nova] [Gantt] Scheduler split status (updated)
On 07/14/2014 12:10 PM, Jay Pipes wrote: On 07/14/2014 10:16 AM, Sylvain Bauza wrote: From an operator perspective, people waited so long for having a scheduler doing scheduling and not only resource placement. Could you elaborate a bit here? What operators are begging for the scheduler to do more than resource placement? And if they are begging for this, what use cases are they trying to address? I'm curious about this as well, what more than resource placement *should* the scheduler handle? On the other hand, I *do* see a usecase for a more holistic scheduler that can take into account a whole group of resources at once (multiple instances, volumes, networks, etc. with various constraints on them, combined with things like server groups and host aggregates). In a simple scenario, suppose a compute node fails. I want to evacuate all the instances that had been on that compute node, but ideally I'd like the scheduler to look at all the instances simultaneously to try to place them...if it does them one at a time it might make a decision that doesn't use resources in an optimal way, possibly even resulting in the failure of one or more evacuations even though there is technically sufficient room in the cluster for all instances. Alternately, suppose I have a group of instances that really want to be placed on the same physical network segment. (For low latency, maybe.) Lastly, I think that any information that fed into the original scheduler decision should be preserved for use by subsequent scheduling operations (migration, evacuation, etc.) This includes stuff like image/flavor metadata, boot-time scheduler hints, etc. Chris ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Nova] [Gantt] Scheduler split status (updated)
Hi Don, comments inline... On 07/14/2014 12:18 AM, Dugger, Donald D wrote: My understanding is the main goal is to get a fully functional gantt working before we do the split. This means we have to clean up the nova interfaces so that all of the current scheduler functionality, including things like aggregates and resource tracking, so that we can make the split and create the gantt tree which will be the default scheduler. +1. Clean before cleave. This means I see 3 main tasks that need to be done before we do the split: 1) Create the scheduler client library 2) Complete the isolate scheduler DB accesses 3) Move the resource tracker out of Nova and into the scheduler If we can focus on those 3 tasks we should be able to actually split the code out into a fully functional scheduler. While I have little disagreement on the above tasks, I feel that actually the order should be: 3), then 2), then 1). My reasoning for this is that the client interface would be dramatically changed by 3), and the number of DB accesses would also be increased by 3), therefore it is more important to fix the current lack of claim-based resource tracking in the scheduler before we move to either 1) or 2). Best, -jay -- Don Dugger Censeo Toto nos in Kansa esse decisse. - D. Gale Ph: 303/443-3786 -Original Message- From: Sylvain Bauza [mailto:sba...@redhat.com] Sent: Friday, July 11, 2014 8:38 AM To: John Garbutt Cc: OpenStack Development Mailing List (not for usage questions) Subject: Re: [openstack-dev] [Nova] [Gantt] Scheduler split status (updated) Le 11/07/2014 13:14, John Garbutt a écrit : On 10 July 2014 16:59, Sylvain Bauza sba...@redhat.com wrote: Le 10/07/2014 15:47, Russell Bryant a écrit : On 07/10/2014 05:06 AM, Sylvain Bauza wrote: Hi all, === tl;dr: Now that we agree on waiting for the split prereqs to be done, we debate on if ResourceTracker should be part of the scheduler code and consequently Scheduler should expose ResourceTracker APIs so that Nova wouldn't own compute nodes resources. I'm proposing to first come with RT as Nova resource in Juno and move ResourceTracker in Scheduler for K, so we at least merge some patches by Juno. === Some debates occured recently about the scheduler split, so I think it's important to loop back with you all to see where we are and what are the discussions. Again, feel free to express your opinions, they are welcome. Where did this resource tracker discussion come up? Do you have any references that I can read to catch up on it? I would like to see more detail on the proposal for what should stay in Nova vs. be moved. What is the interface between Nova and the scheduler here? Oh, missed the most important question you asked. So, about the interface in between scheduler and Nova, the original agreed proposal is in the spec https://review.openstack.org/82133 (approved) where the Scheduler exposes : - select_destinations() : for querying the scheduler to provide candidates - update_resource_stats() : for updating the scheduler internal state (ie. HostState) Here, update_resource_stats() is called by the ResourceTracker, see the implementations (in review) https://review.openstack.org/82778 and https://review.openstack.org/104556. The alternative that has just been raised this week is to provide a new interface where ComputeNode claims for resources and frees these resources, so that all the resources are fully owned by the Scheduler. An initial PoC has been raised here https://review.openstack.org/103598 but I tried to see what would be a ResourceTracker proxified by a Scheduler client here : https://review.openstack.org/105747. As the spec hasn't been written, the names of the interfaces are not properly defined but I made a proposal as : - select_destinations() : same as above - usage_claim() : claim a resource amount - usage_update() : update a resource amount - usage_drop(): frees the resource amount Again, this is a dummy proposal, a spec has to written if we consider moving the RT. While I am not against moving the resource tracker, I feel we could move this to Gantt after the core scheduling has been moved. I was imagining the extensible resource tracker to become (sort of) equivalent to cinder volume drivers. Also the persistent resource claims will give us another plugin point for gantt. That might not be enough, but I think it easier to see once the other elements have moved. But the key point thing I like, is how the current approach amounts to refactoring, similar to the cinder move. I feel we should stick to that if possible. John Thanks John for your feedback. I'm +1 with you, we need to go on the way we defined with all the community, create Gantt once the prereqs are done (see my above and first mail for these) and see after if the line is needed to move. I think this discussion should also be interesting if we also take in account the current Cinder and Neutron scheduling needs, so we would say if it's
Re: [openstack-dev] [Nova] [Gantt] Scheduler split status (updated)
On 12 July 2014 05:07, Jay Pipes jaypi...@gmail.com wrote: On 07/11/2014 07:14 AM, John Garbutt wrote: While I am not against moving the resource tracker, I feel we could move this to Gantt after the core scheduling has been moved. Big -1 from me on this, John. Frankly, I see no urgency whatsoever -- and actually very little benefit -- to moving the scheduler out of Nova. The Gantt project I think is getting ahead of itself by focusing on a split instead of focusing on cleaning up the interfaces between nova-conductor, nova-scheduler, and nova-compute. I see little to no long-term benefit in splitting the scheduler -- especially with its current design -- out from Nova. It's not like Neutron or Cinder, where the split-out service is providing management of a particular kind of resource (network, block storage). The Gantt project isn't providing any resource itself. Instead, it would be acting as a proxy for the placement other services' resources, which, IMO, in and of itself, is not a reason to go through the trouble of splitting the scheduler out of Nova. I am thinking about scaling out the Nova community here. I like the idea of a smaller sub-community, outside of the Nova review queue, that focus its efforts of moving the scheduler forward. I was hoping we get Gantt created by the end of Juno, with nova-scheduler deprecated at the end of Juno, as is. Then the Gantt community can start evolving the current system. We have blocked lots of ideas, saying please wait for Gantt. The people wanting to have scheduling that is aware of both nova and cinder resources, and similar arguments for networking locality aware compute scheduling, and the ones that come to mind. Clearly there is a cost of evolving the interface, once the scheduler is split out. Maybe in this case we want to wait, and I am cool with that, but I wanted to make sure we have our eyes open to what we are delaying. I was imagining the extensible resource tracker to become (sort of) equivalent to cinder volume drivers. The problem with the extensible resource tracker design is that it, again, just shoves a bunch of stuff into a JSON field and both the existing resource tracker code (in nova-compute) as well as the scheduler code (in nova-scheduler) then need to use and abuse this BLOB of random data. I tried to make my point on the extensible resource tracker blueprint about my objections to the design. My first, and main, objection is that there was never demonstrated ANY clear use case for the extensibility of resources that was not already covered by the existing resource tracker and scheduler. The only use case was a vague we have out of tree custom plugins that depend on divergent behaviour and therefore we need a plugin point to change the way the scheduler thinks of a particular resource. And that is not a use case. It's simply a request to break compatibility between clouds with out-of-tree source code. ... please somebody explain a clear use case for these things that does not involve we want to run our divergent code. To be clear, I have no interest in making life easier for out of tree extensions. One reason I like it, is because the code is becoming unwieldy as we keep extending what people would like to report, and what was proposed seemed very like a nice version of the Open Closed principle. Another use case I want to see is to allow deployers to reduce the data each compute node is reporting to the scheduler down the bare minimum for what is required in the configured scheduler filters and weights. This, coupled with the work of only sending deltas rather than all the data all the time, sound really help reduce the DB load generated by the host reporting. We also urgently need to version the data, and the above refactoring, assuming it was quick, I hoped would help us more clearly see where to add in the versioning. In the end this has now just delayed that effort, and thats a probably the biggest looser in this whole debate. Agreed, these are all quite short term goals, and maybe there are better ways of achieving those goals. I intensely dislike the current complexly of the configuration, but I have not had any time to suggest a viable alternative yet. The only thing that popped into my head, is that moving the resource tracker to the scheduler would really help, because you end up getting a single model with related filter vs weight and reporter pairs, or something like that. With the idea of combining filters and weights into a single system, where you can weight and/or filter on based on some property reported form the Resource Tracker. Also the persistent resource claims will give us another plugin point for gantt. That might not be enough, but I think it easier to see once the other elements have moved. Is there some code you can provide me a link to for these persistent resource claims? Not sure I've seen that code yet... or a blueprint?
Re: [openstack-dev] [Nova] [Gantt] Scheduler split status (updated)
On 07/14/2014 10:16 AM, Sylvain Bauza wrote: Le 12/07/2014 06:07, Jay Pipes a écrit : On 07/11/2014 07:14 AM, John Garbutt wrote: On 10 July 2014 16:59, Sylvain Bauza sba...@redhat.com wrote: Le 10/07/2014 15:47, Russell Bryant a écrit : On 07/10/2014 05:06 AM, Sylvain Bauza wrote: Hi all, === tl;dr: Now that we agree on waiting for the split prereqs to be done, we debate on if ResourceTracker should be part of the scheduler code and consequently Scheduler should expose ResourceTracker APIs so that Nova wouldn't own compute nodes resources. I'm proposing to first come with RT as Nova resource in Juno and move ResourceTracker in Scheduler for K, so we at least merge some patches by Juno. === Some debates occured recently about the scheduler split, so I think it's important to loop back with you all to see where we are and what are the discussions. Again, feel free to express your opinions, they are welcome. Where did this resource tracker discussion come up? Do you have any references that I can read to catch up on it? I would like to see more detail on the proposal for what should stay in Nova vs. be moved. What is the interface between Nova and the scheduler here? Oh, missed the most important question you asked. So, about the interface in between scheduler and Nova, the original agreed proposal is in the spec https://review.openstack.org/82133 (approved) where the Scheduler exposes : - select_destinations() : for querying the scheduler to provide candidates - update_resource_stats() : for updating the scheduler internal state (ie. HostState) Here, update_resource_stats() is called by the ResourceTracker, see the implementations (in review) https://review.openstack.org/82778 and https://review.openstack.org/104556. The alternative that has just been raised this week is to provide a new interface where ComputeNode claims for resources and frees these resources, so that all the resources are fully owned by the Scheduler. An initial PoC has been raised here https://review.openstack.org/103598 but I tried to see what would be a ResourceTracker proxified by a Scheduler client here : https://review.openstack.org/105747. As the spec hasn't been written, the names of the interfaces are not properly defined but I made a proposal as : - select_destinations() : same as above - usage_claim() : claim a resource amount - usage_update() : update a resource amount - usage_drop(): frees the resource amount Again, this is a dummy proposal, a spec has to written if we consider moving the RT. While I am not against moving the resource tracker, I feel we could move this to Gantt after the core scheduling has been moved. Big -1 from me on this, John. Frankly, I see no urgency whatsoever -- and actually very little benefit -- to moving the scheduler out of Nova. The Gantt project I think is getting ahead of itself by focusing on a split instead of focusing on cleaning up the interfaces between nova-conductor, nova-scheduler, and nova-compute. -1 on saying there is no urgency. Don't you see the NFV group saying each meeting what is the status of the scheduler split ? Frankly, I don't think a lot of the NFV use cases are well-defined. Even more frankly, I don't see any benefit to a split-out scheduler to a single NFV use case. Don't you see each Summit the lots of talks (and people attending them) talking about how OpenStack should look at Pets vs. Cattle and saying that the scheduler should be out of Nova ? There's been no concrete benefits discussed to having the scheduler outside of Nova. I don't really care how many people say that the scheduler should be out of Nova unless those same people come to the table with concrete reasons why. Just saying something is a benefit does not make it a benefit, and I think I've outlined some of the very real dangers -- in terms of code and payload complexity -- of breaking the scheduler out of Nova until the interfaces are cleaned up and the scheduler actually owns the resources upon which it exercises placement decisions. From an operator perspective, people waited so long for having a scheduler doing scheduling and not only resource placement. Could you elaborate a bit here? What operators are begging for the scheduler to do more than resource placement? And if they are begging for this, what use cases are they trying to address? I'm genuinely curious, so looking forward to your reply here! :) snip... As for the idea that things will get *easier* once scheduler code is broken out of Nova, I go back to my original statement that I don't really see the benefit of the split at this point, and I would just bring up the fact that Neutron/nova-network is a shining example of how things can easily backfire when splitting of code is done too early before interfaces are cleaned up and responsibilities between internal components are not clearly agreed upon. Please, please, don't mix the rationale for extensible Resource Tracker and the
[openstack-dev] [Nova] [Gantt] Scheduler split status (updated)
Hi All, I'm sorry I am so late to this lively discussion - it looks a good one! Jay has been driving the debate a bit so most of this is in response to his comments. But please, anyone should chip in. On extensible resource tracking Jay, I am surprised to hear you say no one has explained to you why there is an extensible resource tracking blueprint. It's simple, there was a succession of blueprints wanting to add data about this and that to the resource tracker and the scheduler and the database tables used to communicate. These included capabilities, all the stuff in the stats, rxtx_factor, the equivalent for cpu (which only works on one hypervisor I think), pci_stats and more were coming including, https://blueprints.launchpad.net/nova/+spec/network-bandwidth-entitlement https://blueprints.launchpad.net/nova/+spec/cpu-entitlement So, in short, your claim that there are no operators asking for additional stuff is simply not true. Around about the Icehouse summit (I think) it was suggested that we should stop the obvious trend and add a way to make resource tracking extensible, similar to metrics, which had just been added as an extensible way of collecting on going usage data (because that was also wanted). The json blob you refer to was down to the bad experience of the compute_node_stats table implemented for stats - which had a particular performance hit because it required an expensive join. This was dealt with by removing the table and adding a string field to contain the data as a json blob. A pure performance optimization. Clearly there is no need to store things in this way and with Nova objects being introduced there is a means to provide strict type checking on the data even if it is stored as json blobs in the database. On scheduler split I have no particular position on splitting the scheduler. However, there was an interesting reaction to the network bandwidth entitlement blueprint listed above. The nova community felt it was a network thing and so nova should not provide it - neutron should. Of course, in nova, the nova scheduler makes placement decisions... can you see where this is going...? Nova needs to coordinate its placement decision with neutron to decide if a host has sufficient bandwidth available. Similar points are made about cinder - nova has no idea about cinder, but in some environments the location of a volume matters when you come to place an instance. I should re-iterate that I have no position on splitting out the scheduler, but some way to deal with information from outside nova is certainly desirable. Maybe other services have the same dilemma. On global resource tracker I have to say I am inclined to be against the idea of turning the scheduler into a global resource tracker. I do see the benefit of obtaining a resource claim up front, we have all seen that the scheduler can make incorrect choices because of the delay in reflecting resource allocation to the database and so to the scheduler - it operates on imperfect information. However, it is best to avoid a global service relying on synchronous interaction with compute nodes during the process of servicing a request. I have looked at your example code for the scheduler (global resource tracker) and it seems to make a choice from local information and then interact with the chosen compute node to obtain a claim and then try again if the claim fails. I get it - I see that it deals with the same list of hosts on the retry. I also see it has no better chance of getting it right. Your desire to have a claim is borne out by the persistent claims spec (I love the spec, I really I don't see why they have to be persistent). I think that is a great idea. Why not let the scheduler make placement suggestions (as a global service) and then allow conductors to obtain the claim and retry if the claim fails? Similar process to your code, but the scheduler only does its part and the conductors scale out the process by acting more locally and with more parallelism. (Of course, you could also be optimistic and allow the compute node to do the claim as part of the create as the degenerate case). To emphasize the point further, what would a cells scheduler do? Would that also make a synchronous operation to obtain the claim? My reaction to the global resource tracker idea has been quite negative. I want to like the idea because I like the thought of knowing I have the resources when I get my answer. Its just that I think the persistent claims (without the persistent part :) ) gives us a lot of what we need. But I am still open to be convinced. Paul On 07/14/2014 10:16 AM, Sylvain Bauza wrote: Le 12/07/2014 06:07, Jay Pipes a écrit : On 07/11/2014 07:14 AM, John Garbutt wrote: On 10 July 2014 16:59, Sylvain Bauza sbauza at redhat.com wrote: Le 10/07/2014 15:47, Russell Bryant a écrit : On 07/10/2014 05:06 AM, Sylvain Bauza wrote: Hi all, === tl;dr: Now that
Re: [openstack-dev] [Nova] [Gantt] Scheduler split status (updated)
My understanding is the main goal is to get a fully functional gantt working before we do the split. This means we have to clean up the nova interfaces so that all of the current scheduler functionality, including things like aggregates and resource tracking, so that we can make the split and create the gantt tree which will be the default scheduler. This means I see 3 main tasks that need to be done before we do the split: 1) Create the scheduler client library 2) Complete the isolate scheduler DB accesses 3) Move the resource tracker out of Nova and into the scheduler If we can focus on those 3 tasks we should be able to actually split the code out into a fully functional scheduler. -- Don Dugger Censeo Toto nos in Kansa esse decisse. - D. Gale Ph: 303/443-3786 -Original Message- From: Sylvain Bauza [mailto:sba...@redhat.com] Sent: Friday, July 11, 2014 8:38 AM To: John Garbutt Cc: OpenStack Development Mailing List (not for usage questions) Subject: Re: [openstack-dev] [Nova] [Gantt] Scheduler split status (updated) Le 11/07/2014 13:14, John Garbutt a écrit : On 10 July 2014 16:59, Sylvain Bauza sba...@redhat.com wrote: Le 10/07/2014 15:47, Russell Bryant a écrit : On 07/10/2014 05:06 AM, Sylvain Bauza wrote: Hi all, === tl;dr: Now that we agree on waiting for the split prereqs to be done, we debate on if ResourceTracker should be part of the scheduler code and consequently Scheduler should expose ResourceTracker APIs so that Nova wouldn't own compute nodes resources. I'm proposing to first come with RT as Nova resource in Juno and move ResourceTracker in Scheduler for K, so we at least merge some patches by Juno. === Some debates occured recently about the scheduler split, so I think it's important to loop back with you all to see where we are and what are the discussions. Again, feel free to express your opinions, they are welcome. Where did this resource tracker discussion come up? Do you have any references that I can read to catch up on it? I would like to see more detail on the proposal for what should stay in Nova vs. be moved. What is the interface between Nova and the scheduler here? Oh, missed the most important question you asked. So, about the interface in between scheduler and Nova, the original agreed proposal is in the spec https://review.openstack.org/82133 (approved) where the Scheduler exposes : - select_destinations() : for querying the scheduler to provide candidates - update_resource_stats() : for updating the scheduler internal state (ie. HostState) Here, update_resource_stats() is called by the ResourceTracker, see the implementations (in review) https://review.openstack.org/82778 and https://review.openstack.org/104556. The alternative that has just been raised this week is to provide a new interface where ComputeNode claims for resources and frees these resources, so that all the resources are fully owned by the Scheduler. An initial PoC has been raised here https://review.openstack.org/103598 but I tried to see what would be a ResourceTracker proxified by a Scheduler client here : https://review.openstack.org/105747. As the spec hasn't been written, the names of the interfaces are not properly defined but I made a proposal as : - select_destinations() : same as above - usage_claim() : claim a resource amount - usage_update() : update a resource amount - usage_drop(): frees the resource amount Again, this is a dummy proposal, a spec has to written if we consider moving the RT. While I am not against moving the resource tracker, I feel we could move this to Gantt after the core scheduling has been moved. I was imagining the extensible resource tracker to become (sort of) equivalent to cinder volume drivers. Also the persistent resource claims will give us another plugin point for gantt. That might not be enough, but I think it easier to see once the other elements have moved. But the key point thing I like, is how the current approach amounts to refactoring, similar to the cinder move. I feel we should stick to that if possible. John Thanks John for your feedback. I'm +1 with you, we need to go on the way we defined with all the community, create Gantt once the prereqs are done (see my above and first mail for these) and see after if the line is needed to move. I think this discussion should also be interesting if we also take in account the current Cinder and Neutron scheduling needs, so we would say if it's the good direction. Others ? Note: The spec https://review.openstack.org/89893 is not yet approved today, as the Spec approval freeze happened, I would like to discuss with the team if we can have an exception on it so the work could happen by Juno. Thanks, -Sylvain ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo
Re: [openstack-dev] [Nova] [Gantt] Scheduler split status (updated)
On 10 July 2014 16:59, Sylvain Bauza sba...@redhat.com wrote: Le 10/07/2014 15:47, Russell Bryant a écrit : On 07/10/2014 05:06 AM, Sylvain Bauza wrote: Hi all, === tl;dr: Now that we agree on waiting for the split prereqs to be done, we debate on if ResourceTracker should be part of the scheduler code and consequently Scheduler should expose ResourceTracker APIs so that Nova wouldn't own compute nodes resources. I'm proposing to first come with RT as Nova resource in Juno and move ResourceTracker in Scheduler for K, so we at least merge some patches by Juno. === Some debates occured recently about the scheduler split, so I think it's important to loop back with you all to see where we are and what are the discussions. Again, feel free to express your opinions, they are welcome. Where did this resource tracker discussion come up? Do you have any references that I can read to catch up on it? I would like to see more detail on the proposal for what should stay in Nova vs. be moved. What is the interface between Nova and the scheduler here? Oh, missed the most important question you asked. So, about the interface in between scheduler and Nova, the original agreed proposal is in the spec https://review.openstack.org/82133 (approved) where the Scheduler exposes : - select_destinations() : for querying the scheduler to provide candidates - update_resource_stats() : for updating the scheduler internal state (ie. HostState) Here, update_resource_stats() is called by the ResourceTracker, see the implementations (in review) https://review.openstack.org/82778 and https://review.openstack.org/104556. The alternative that has just been raised this week is to provide a new interface where ComputeNode claims for resources and frees these resources, so that all the resources are fully owned by the Scheduler. An initial PoC has been raised here https://review.openstack.org/103598 but I tried to see what would be a ResourceTracker proxified by a Scheduler client here : https://review.openstack.org/105747. As the spec hasn't been written, the names of the interfaces are not properly defined but I made a proposal as : - select_destinations() : same as above - usage_claim() : claim a resource amount - usage_update() : update a resource amount - usage_drop(): frees the resource amount Again, this is a dummy proposal, a spec has to written if we consider moving the RT. While I am not against moving the resource tracker, I feel we could move this to Gantt after the core scheduling has been moved. I was imagining the extensible resource tracker to become (sort of) equivalent to cinder volume drivers. Also the persistent resource claims will give us another plugin point for gantt. That might not be enough, but I think it easier to see once the other elements have moved. But the key point thing I like, is how the current approach amounts to refactoring, similar to the cinder move. I feel we should stick to that if possible. John ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Nova] [Gantt] Scheduler split status (updated)
Le 11/07/2014 13:14, John Garbutt a écrit : On 10 July 2014 16:59, Sylvain Bauza sba...@redhat.com wrote: Le 10/07/2014 15:47, Russell Bryant a écrit : On 07/10/2014 05:06 AM, Sylvain Bauza wrote: Hi all, === tl;dr: Now that we agree on waiting for the split prereqs to be done, we debate on if ResourceTracker should be part of the scheduler code and consequently Scheduler should expose ResourceTracker APIs so that Nova wouldn't own compute nodes resources. I'm proposing to first come with RT as Nova resource in Juno and move ResourceTracker in Scheduler for K, so we at least merge some patches by Juno. === Some debates occured recently about the scheduler split, so I think it's important to loop back with you all to see where we are and what are the discussions. Again, feel free to express your opinions, they are welcome. Where did this resource tracker discussion come up? Do you have any references that I can read to catch up on it? I would like to see more detail on the proposal for what should stay in Nova vs. be moved. What is the interface between Nova and the scheduler here? Oh, missed the most important question you asked. So, about the interface in between scheduler and Nova, the original agreed proposal is in the spec https://review.openstack.org/82133 (approved) where the Scheduler exposes : - select_destinations() : for querying the scheduler to provide candidates - update_resource_stats() : for updating the scheduler internal state (ie. HostState) Here, update_resource_stats() is called by the ResourceTracker, see the implementations (in review) https://review.openstack.org/82778 and https://review.openstack.org/104556. The alternative that has just been raised this week is to provide a new interface where ComputeNode claims for resources and frees these resources, so that all the resources are fully owned by the Scheduler. An initial PoC has been raised here https://review.openstack.org/103598 but I tried to see what would be a ResourceTracker proxified by a Scheduler client here : https://review.openstack.org/105747. As the spec hasn't been written, the names of the interfaces are not properly defined but I made a proposal as : - select_destinations() : same as above - usage_claim() : claim a resource amount - usage_update() : update a resource amount - usage_drop(): frees the resource amount Again, this is a dummy proposal, a spec has to written if we consider moving the RT. While I am not against moving the resource tracker, I feel we could move this to Gantt after the core scheduling has been moved. I was imagining the extensible resource tracker to become (sort of) equivalent to cinder volume drivers. Also the persistent resource claims will give us another plugin point for gantt. That might not be enough, but I think it easier to see once the other elements have moved. But the key point thing I like, is how the current approach amounts to refactoring, similar to the cinder move. I feel we should stick to that if possible. John Thanks John for your feedback. I'm +1 with you, we need to go on the way we defined with all the community, create Gantt once the prereqs are done (see my above and first mail for these) and see after if the line is needed to move. I think this discussion should also be interesting if we also take in account the current Cinder and Neutron scheduling needs, so we would say if it's the good direction. Others ? Note: The spec https://review.openstack.org/89893 is not yet approved today, as the Spec approval freeze happened, I would like to discuss with the team if we can have an exception on it so the work could happen by Juno. Thanks, -Sylvain ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Nova] [Gantt] Scheduler split status (updated)
On 07/11/2014 07:14 AM, John Garbutt wrote: On 10 July 2014 16:59, Sylvain Bauza sba...@redhat.com wrote: Le 10/07/2014 15:47, Russell Bryant a écrit : On 07/10/2014 05:06 AM, Sylvain Bauza wrote: Hi all, === tl;dr: Now that we agree on waiting for the split prereqs to be done, we debate on if ResourceTracker should be part of the scheduler code and consequently Scheduler should expose ResourceTracker APIs so that Nova wouldn't own compute nodes resources. I'm proposing to first come with RT as Nova resource in Juno and move ResourceTracker in Scheduler for K, so we at least merge some patches by Juno. === Some debates occured recently about the scheduler split, so I think it's important to loop back with you all to see where we are and what are the discussions. Again, feel free to express your opinions, they are welcome. Where did this resource tracker discussion come up? Do you have any references that I can read to catch up on it? I would like to see more detail on the proposal for what should stay in Nova vs. be moved. What is the interface between Nova and the scheduler here? Oh, missed the most important question you asked. So, about the interface in between scheduler and Nova, the original agreed proposal is in the spec https://review.openstack.org/82133 (approved) where the Scheduler exposes : - select_destinations() : for querying the scheduler to provide candidates - update_resource_stats() : for updating the scheduler internal state (ie. HostState) Here, update_resource_stats() is called by the ResourceTracker, see the implementations (in review) https://review.openstack.org/82778 and https://review.openstack.org/104556. The alternative that has just been raised this week is to provide a new interface where ComputeNode claims for resources and frees these resources, so that all the resources are fully owned by the Scheduler. An initial PoC has been raised here https://review.openstack.org/103598 but I tried to see what would be a ResourceTracker proxified by a Scheduler client here : https://review.openstack.org/105747. As the spec hasn't been written, the names of the interfaces are not properly defined but I made a proposal as : - select_destinations() : same as above - usage_claim() : claim a resource amount - usage_update() : update a resource amount - usage_drop(): frees the resource amount Again, this is a dummy proposal, a spec has to written if we consider moving the RT. While I am not against moving the resource tracker, I feel we could move this to Gantt after the core scheduling has been moved. Big -1 from me on this, John. Frankly, I see no urgency whatsoever -- and actually very little benefit -- to moving the scheduler out of Nova. The Gantt project I think is getting ahead of itself by focusing on a split instead of focusing on cleaning up the interfaces between nova-conductor, nova-scheduler, and nova-compute. I see little to no long-term benefit in splitting the scheduler -- especially with its current design -- out from Nova. It's not like Neutron or Cinder, where the split-out service is providing management of a particular kind of resource (network, block storage). The Gantt project isn't providing any resource itself. Instead, it would be acting as a proxy for the placement other services' resources, which, IMO, in and of itself, is not a reason to go through the trouble of splitting the scheduler out of Nova. I was imagining the extensible resource tracker to become (sort of) equivalent to cinder volume drivers. The problem with the extensible resource tracker design is that it, again, just shoves a bunch of stuff into a JSON field and both the existing resource tracker code (in nova-compute) as well as the scheduler code (in nova-scheduler) then need to use and abuse this BLOB of random data. I tried to make my point on the extensible resource tracker blueprint about my objections to the design. My first, and main, objection is that there was never demonstrated ANY clear use case for the extensibility of resources that was not already covered by the existing resource tracker and scheduler. The only use case was a vague we have out of tree custom plugins that depend on divergent behaviour and therefore we need a plugin point to change the way the scheduler thinks of a particular resource. And that is not a use case. It's simply a request to break compatibility between clouds with out-of-tree source code. My second objection was that the proposed implementation added yet more completely unnecessary complication to the scheduler, with the introduction of scheduler consumers, one for each extensible resource added as plugins to the resource tracker. The existing scheduler is already displaying a silly amount of needless complexity, and current (and approved) blueprints like this one merely add to the endless array of configuration options and ostensibly flexible behaviour. The problem with ALL of these approved and in-review specs, IMO, is
Re: [openstack-dev] [Nova] [Gantt] Scheduler split status (updated)
Hi all, === tl;dr: Now that we agree on waiting for the split prereqs to be done, we debate on if ResourceTracker should be part of the scheduler code and consequently Scheduler should expose ResourceTracker APIs so that Nova wouldn't own compute nodes resources. I'm proposing to first come with RT as Nova resource in Juno and move ResourceTracker in Scheduler for K, so we at least merge some patches by Juno. === Some debates occured recently about the scheduler split, so I think it's important to loop back with you all to see where we are and what are the discussions. Again, feel free to express your opinions, they are welcome. Le 03/07/2014 19:53, Sylvain Bauza a écrit : Hi, == tl; dr: A decision has been made to split out the scheduler to a separate project not on a feature parity basis with nova-scheduler, your comments are welcome. == snip/ So, based on the feedback consensus, we agreed to wait for all the preparation work to be made before doing the split. That said, now that we're focusing back on the split prereqs, some good talks are coming in about what the Scheduler should express as APIs and what would be consequently owned by the scheduler (and later Gantt). During the Summit, we expressed the boundary in between Scheduler and Nova as the current Scheduler namespace in Nova and where the ResourceTracker would take use of a Scheduler client for updating stats to the Scheduler [1]. In this model, compute nodes would stay as a Nova resource and Scheduler (and later Gantt) would have its own representation of the resources it has to manage. The implementation of this model is https://review.openstack.org/82778 still under review. Note also the spec has been validated 3 months ago [2], so this was planned to delivered for Juno. Based on a review we got on the implementation patch [3], there is now an interesting discussion about if Nova should still keep its own representation of the resources (and consequently should manage all claims for resource usage). The idea of it would be to move the ResourceTracker to the Scheduler namespace and consequently the Scheduler should expose Tracking APIs for claiming/unclaiming usage, so the Compute Manager (and no longer the ResourceTracker) would make use of a Scheduler client. A PoC for seeing the ResourceTracker proxified by a Scheduler client can be found here https://review.openstack.org/105747 As we're now beyond the Nova Specs proposal deadline, that would mean that any spec which would be created would see its implementation merged by K at least. Don't blame me if I'm trying to be pragmatic : at a first glance, I appreciate the idea of having the Tracker moved to the Scheduler, but I'm seeing this as another possible step for the split, not the first one. So my proposal is to say : 1. Move on the existing validated spec and review the existing implementation https://review.openstack.org/82778 2. Create a spec targeted for K to see what would be a world where the tracking of resources is done by the Scheduler 3. If that above spec gets validated, do the necessary work in K to move the line 3(bis). If that above spec gets refused, we are still on the good path for splitting the scheduler Thoughts ? -Sylvain [1] https://etherpad.openstack.org/p/juno-nova-gantt-apis [2] https://review.openstack.org/82133 [3] https://review.openstack.org/#/c/82778/42 ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Nova] [Gantt] Scheduler split status (updated)
On 07/10/2014 05:06 AM, Sylvain Bauza wrote: Hi all, === tl;dr: Now that we agree on waiting for the split prereqs to be done, we debate on if ResourceTracker should be part of the scheduler code and consequently Scheduler should expose ResourceTracker APIs so that Nova wouldn't own compute nodes resources. I'm proposing to first come with RT as Nova resource in Juno and move ResourceTracker in Scheduler for K, so we at least merge some patches by Juno. === Some debates occured recently about the scheduler split, so I think it's important to loop back with you all to see where we are and what are the discussions. Again, feel free to express your opinions, they are welcome. Where did this resource tracker discussion come up? Do you have any references that I can read to catch up on it? I would like to see more detail on the proposal for what should stay in Nova vs. be moved. What is the interface between Nova and the scheduler here? -- Russell Bryant ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Nova] [Gantt] Scheduler split status (updated)
Active discussion at the gantt meeting this week, check out the log: http://eavesdrop.openstack.org/meetings/gantt/2014/gantt.2014-07-01-15.00.log.html -- Don Dugger Censeo Toto nos in Kansa esse decisse. - D. Gale Ph: 303/443-3786 -Original Message- From: Russell Bryant [mailto:rbry...@redhat.com] Sent: Thursday, July 10, 2014 7:48 AM To: openstack-dev@lists.openstack.org Subject: Re: [openstack-dev] [Nova] [Gantt] Scheduler split status (updated) On 07/10/2014 05:06 AM, Sylvain Bauza wrote: Hi all, === tl;dr: Now that we agree on waiting for the split prereqs to be done, we debate on if ResourceTracker should be part of the scheduler code and consequently Scheduler should expose ResourceTracker APIs so that Nova wouldn't own compute nodes resources. I'm proposing to first come with RT as Nova resource in Juno and move ResourceTracker in Scheduler for K, so we at least merge some patches by Juno. === Some debates occured recently about the scheduler split, so I think it's important to loop back with you all to see where we are and what are the discussions. Again, feel free to express your opinions, they are welcome. Where did this resource tracker discussion come up? Do you have any references that I can read to catch up on it? I would like to see more detail on the proposal for what should stay in Nova vs. be moved. What is the interface between Nova and the scheduler here? -- Russell Bryant ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Nova] [Gantt] Scheduler split status (updated)
Le 10/07/2014 16:32, Dugger, Donald D a écrit : Active discussion at the gantt meeting this week, check out the log: http://eavesdrop.openstack.org/meetings/gantt/2014/gantt.2014-07-01-15.00.log.html I would prefer to mention this week's meeting : http://eavesdrop.openstack.org/meetings/gantt/2014/gantt.2014-07-08-15.00.log.html -- Don Dugger Censeo Toto nos in Kansa esse decisse. - D. Gale Ph: 303/443-3786 -Original Message- From: Russell Bryant [mailto:rbry...@redhat.com] Sent: Thursday, July 10, 2014 7:48 AM To: openstack-dev@lists.openstack.org Subject: Re: [openstack-dev] [Nova] [Gantt] Scheduler split status (updated) On 07/10/2014 05:06 AM, Sylvain Bauza wrote: Hi all, === tl;dr: Now that we agree on waiting for the split prereqs to be done, we debate on if ResourceTracker should be part of the scheduler code and consequently Scheduler should expose ResourceTracker APIs so that Nova wouldn't own compute nodes resources. I'm proposing to first come with RT as Nova resource in Juno and move ResourceTracker in Scheduler for K, so we at least merge some patches by Juno. === Some debates occured recently about the scheduler split, so I think it's important to loop back with you all to see where we are and what are the discussions. Again, feel free to express your opinions, they are welcome. Where did this resource tracker discussion come up? Do you have any references that I can read to catch up on it? I would like to see more detail on the proposal for what should stay in Nova vs. be moved. What is the interface between Nova and the scheduler here? -- Russell Bryant ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Nova] [Gantt] Scheduler split status (updated)
I blame MicroSoft, cut/paste didn't work so I had to manually type in the (wrong) URL. -- Don Dugger Censeo Toto nos in Kansa esse decisse. - D. Gale Ph: 303/443-3786 -Original Message- From: Sylvain Bauza [mailto:sba...@redhat.com] Sent: Thursday, July 10, 2014 9:05 AM To: openstack-dev@lists.openstack.org Subject: Re: [openstack-dev] [Nova] [Gantt] Scheduler split status (updated) Le 10/07/2014 16:32, Dugger, Donald D a écrit : Active discussion at the gantt meeting this week, check out the log: http://eavesdrop.openstack.org/meetings/gantt/2014/gantt.2014-07-01-15 .00.log.html I would prefer to mention this week's meeting : http://eavesdrop.openstack.org/meetings/gantt/2014/gantt.2014-07-08-15.00.log.html -- Don Dugger Censeo Toto nos in Kansa esse decisse. - D. Gale Ph: 303/443-3786 -Original Message- From: Russell Bryant [mailto:rbry...@redhat.com] Sent: Thursday, July 10, 2014 7:48 AM To: openstack-dev@lists.openstack.org Subject: Re: [openstack-dev] [Nova] [Gantt] Scheduler split status (updated) On 07/10/2014 05:06 AM, Sylvain Bauza wrote: Hi all, === tl;dr: Now that we agree on waiting for the split prereqs to be done, we debate on if ResourceTracker should be part of the scheduler code and consequently Scheduler should expose ResourceTracker APIs so that Nova wouldn't own compute nodes resources. I'm proposing to first come with RT as Nova resource in Juno and move ResourceTracker in Scheduler for K, so we at least merge some patches by Juno. === Some debates occured recently about the scheduler split, so I think it's important to loop back with you all to see where we are and what are the discussions. Again, feel free to express your opinions, they are welcome. Where did this resource tracker discussion come up? Do you have any references that I can read to catch up on it? I would like to see more detail on the proposal for what should stay in Nova vs. be moved. What is the interface between Nova and the scheduler here? -- Russell Bryant ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
