Nice idea. I also think that filters and weighers are FilterScheduler-specific. Thus, that is unnecessary for SolverScheduler to try to translate all filters/weighers into constraints. It would be easier for transition, though. Anyway, we just need some placement logics that will be written as constraints, as the they are currently represented as filters in FilterScheduler. So yes we will need a placement advisor here.
For provisioning engine, isn't scheduler manager (or maybe nova-conductor) will be the one? I still don't figure out after we have gantt, how nova-conductor interact with gantt, or we put its logic into gantt, too. Another though would be the need for Instance Group API [1]. Currently users can only request multiple instances of the same flavors. These requests do not need LP to solve, just placing instances one by one is sufficient. Therefore we need this API so that users can request instances of different flavors, with some relations (constraints) among them. The advantage is that this logic and API will help us add Cinder volumes with ease (not sure how the Cinder-stackers think about it, though). Best regards, Toan [1] https://wiki.openstack.org/wiki/InstanceGroupApiExtension ----- Original Message ----- From: Yathiraj Udupi (yudupi) <yud...@cisco.com> To: OpenStack Development Mailing List (not for usage questions) <openstack-dev@lists.openstack.org> Sent: Thu, 30 Jan 2014 18:13:59 -0000 (UTC) Subject: Re: [openstack-dev] [Nova][Scheduler] Policy Based Scheduler and Solver Scheduler It is really good we are reviving the conversation we started during the last summit in Hongkong during one of the scheduler sessions called “Smart resource placement”. This is the document we used to discuss during the session. Probably you may have seen this before: https://docs.google.com/document/d/1IiPI0sfaWb1bdYiMWzAAx0HYR6UqzOan_Utgml5W1HI/edit The idea is to separate out the logic for the placement decision engine from the actual request and the final provisioning phase. The placement engine itself can be pluggable, and as we show in the solver scheduler blueprint, we show how it fits inside of Nova. The discussions at the summit and in our weekly scheduler meetings led to us starting the “Smart resource placement” idea inside of Nova, and then take it to a unified global level spanning cross services such as cinder and neutron. Like you point out, I do agree the two entities of placement advisor, and the placement engine, but I think there should be a third one – the provisioning engine, which should be responsible for whatever it takes to finally create the instances, after the placement decision has been taken. It is good to take incremental approaches, hence we should try to get patches like these get accepted first within nova, and then slowly split up the logic into separate entities. Thanks, Yathi. On 1/30/14, 7:14 AM, "Gil Rapaport" <g...@il.ibm.com<mailto:g...@il.ibm.com>> wrote: Hi all, Excellent definition of the issue at hand. The recent blueprints of policy-based-scheduler and solver-scheduler indeed highlight a possible weakness in the current design, as despite their completely independent contributions (i.e. which filters to apply per request vs. how to compute a valid placement) their implementation as drivers makes combining them non-trivial. As Alex Glikson hinted a couple of weekly meetings ago, our approach to this is to think of the driver's work as split between two entities: -- A Placement Advisor, that constructs placement problems for scheduling requests (filter-scheduler and policy-based-scheduler) -- A Placement Engine, that solves placement problems (HostManager in get_filtered_hosts() and solver-scheduler with its LP engine). Such modularity should allow developing independent mechanisms that can be combined seamlessly through a unified & well-defined protocol based on constructing "placement problem" objects by the placement advisor and then passing them to the placement engine, which returns the solution. The protocol can be orchestrated by the scheduler manager. As can be seen at this point already, the policy-based-scheduler blueprint can now be positioned as an improvement of the placement advisor. Similarly, the solver-scheduler blueprint can be positioned as an improvement of the placement engine. I'm working on a wiki page that will get into the details. Would appreciate your initial thoughts on this approach. Regards, Gil From: Khanh-Toan Tran <khanh-toan.t...@cloudwatt.com<mailto:khanh-toan.t...@cloudwatt.com>> To: "OpenStack Development Mailing List (not for usage questions)" <openstack-dev@lists.openstack.org<mailto:openstack-dev@lists.openstack.org>>, Date: 01/30/2014 01:43 PM Subject: Re: [openstack-dev] [Nova][Scheduler] Policy Based Scheduler and Solver Scheduler ________________________________ Hi Sylvain, 1) Some Filters such as AggregateCoreFilter, AggregateRAMFilter can change its parameters for aggregates. But what if admin wants to change for all hosts in an availability-zone? Does he have to rewrite all the parameters in all aggregates? Or should we create a new AvailabilityZoneCoreFilter? The Policy Based Scheduler (PBS) blueprint separates the effect (filter according to Core) from its target (all hosts in an aggregate, or in an availability-zone). It will benefit all filters, not just CoreFilter or RAMFilter, so that we can avoid creating for each filter XFilter the AggregateXFilter and AvailabilityZoneWFilter from now on. Beside, if admin wants to apply the a filter to some aggregates (or availability-zone) and not the other (don’t call filters at all, not just modify parameters), he can do it. It help us avoid running all filters on all hosts. 2) In fact, we also prepare for a separated scheduler in which PBS is a very first step of it, that’s why we purposely separate the Policy Based Scheduler from Policy Based Scheduling Module (PBSM) [1] which is the core of our architecture. If you look at our code, you will see that Policy_Based_Scheduler.py is only slightly different from Filter Scheduler. That is because we just want a link from Nova-scheduler to PBSM. We’re trying to push some more management into scheduler without causing too much modification, as you can see in the patch . Thus I’m very happy when Gantt is proposed. As I see it, Gantt is based on Nova-scheduler code, with the planning on replacing nova-scheduler in J. The separation from Nova will be complicated, but not on scheduling part. Thus integrating PBS and PBSM into Gantt would not be a problem. Best regards, [1] https://docs.google.com/document/d/1gr4Pb1ErXymxN9QXR4G_jVjLqNOg2ij9oA0JrLwMVRA Toan De : Sylvain Bauza [mailto:sylvain.ba...@gmail.com] Envoyé : jeudi 30 janvier 2014 11:16 À : OpenStack Development Mailing List (not for usage questions) Objet : Re: [openstack-dev] [Nova][Scheduler] Policy Based Scheduler and Solver Scheduler Hi Khanh-Toan, I only have one comment on your proposal : why are you proposing something new for overcommitments with aggregates while the AggregateCoreFilter [1] and AggregateRAMFilter [2]already exist, which AIUI provide same feature ? I'm also concerned about the scope of changes for scheduler, as Gantt is currently trying to replace it. Can we imagine such big changes to be committed on the Nova side, while it's planned to have a Scheduler service in the next future ? -Sylvain [1] https://github.com/openstack/nova/blob/master/nova/scheduler/filters/core_filter.py#L74 [2] https://github.com/openstack/nova/blob/master/nova/scheduler/filters/ram_filter.py#L75 2014-01-30 Khanh-Toan Tran <khanh-toan.t...@cloudwatt.com<mailto:khanh-toan.t...@cloudwatt.com>> There is an unexpected line break in the middle of the link, so I post it again: https://docs.google.com/document/d/1RfP7jRsw1mXMjd7in72ARjK0fTrsQv1bqolOri IQB2Y<https://docs.google.com/document/d/1RfP7jRsw1mXMjd7in72ARjK0fTrsQv1bqolOriIQB2Y> > -----Message d'origine----- > De : Khanh-Toan Tran > [mailto:khanh-toan.t...@cloudwatt.com<mailto:khanh-toan.t...@cloudwatt.com>] > Envoyé : mercredi 29 janvier 2014 13:25 > À : 'OpenStack Development Mailing List (not for usage questions)' > Objet : [openstack-dev] [Nova][Scheduler] Policy Based Scheduler and Solver > Scheduler > > Dear all, > > As promised in the Scheduler/Gantt meeting, here is our analysis on the > connection between Policy Based Scheduler and Solver Scheduler: > > https://docs.google.com/document/d/1RfP7jRsw1mXMjd7in72ARjK0fTrsQv1bq > olOri > IQB2Y > > This document briefs the mechanism of the two schedulers and the possibility of > cooperation. It is my personal point of view only. > > In a nutshell, Policy Based Scheduler allows admin to define policies for different > physical resources (an aggregate, an availability-zone, or all > infrastructure) or different (classes of) users. Admin can modify > (add/remove/modify) any policy in runtime, and the modification effect is only > in the target (e.g. the aggregate, the users) that the policy is defined to. Solver > Scheduler solves the placement of groups of instances simultaneously by putting > all the known information into a integer linear system and uses Integer Program > solver to solve the latter. Thus relation between VMs and between VMs- > computes are all accounted for. > > If working together, Policy Based Scheduler can supply the filters and weighers > following the policies rules defined for different computes. > These filters and weighers can be converted into constraints & cost function for > Solver Scheduler to solve. More detailed will be found in the doc. > > I look forward for comments and hope that we can work it out. > > Best regards, > > Khanh-Toan TRAN > > > _______________________________________________ > OpenStack-dev mailing list > OpenStack-dev@lists.openstack.org<mailto:OpenStack-dev@lists.openstack.org> > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev _______________________________________________ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org<mailto:OpenStack-dev@lists.openstack.org> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev _______________________________________________ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org<mailto:OpenStack-dev@lists.openstack.org> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev _______________________________________________ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
