Re: [openstack-dev] [OpenStack][Runtime Policy] A proposal for OpenStack run time policy to manage compute/storage resource
Hi Gokul, Thanks for working out how all these policy initiatives relate to each other. I'll be spending some time diving into the ones I hadn't heard about. I made some additional comments about Congress below. Tim - Original Message - From: Jay Lau jay.lau@gmail.com To: OpenStack Development Mailing List (not for usage questions) openstack-dev@lists.openstack.org Sent: Wednesday, March 5, 2014 7:31:55 AM Subject: Re: [openstack-dev] [OpenStack][Runtime Policy] A proposal for OpenStack run time policy to manage compute/storage resource Hi Gokul, 2014-03-05 3:30 GMT+08:00 Gokul Kandiraju gokul4o...@gmail.com : Dear All, We are working on a framework where we want to monitor the system and take certain actions when specific events or situations occur. Here are two examples of ‘different’ situations: Example 1: A VM’s-Owner and N/W’s-owner are different == this could mean a violation == we need to take some action Example 2: A simple policy such as (VM-migrate of all VMs on possible node failure) OR (a more complex Energy Policy that may involve optimization). Both these examples need monitoring and actions to be taken when certain events happen (or through polling). However, the first one falls into the Compliance domain with Boolean conditions getting evaluated while the second one may require a more richer set of expression allowing for sequences or algorithms. So far, based on this discussion, it seems that these are *separate* initiatives in the community. I am understanding the Congress project to be in the domain of Boolean conditions (used for Compliance, etc.) where as the Run-time-policies (Jay's proposal) where policies can be expressed as rules, algorithms with higher-level goals. Is this understanding correct? Also, looking at all the mails, this is what I am reading: 1. Congress -- Focused on Compliance [ is that correct? ] (Boolean constraints and logic) [Tim] Your characterization of boolean constraints for Congress is probably a good one. Congress won't be solving optimization/numeric problems any time soon if ever. However, I could imagine that down the road we could tell Congress here's the policy (optimization or Boolean) that we want to enforce, and it would carve off say the Load-balancing part of the policy and send it to the Runtime-Policies component; or it would carve off the placement policy and send it to the SolverScheduler. Not saying I know how to do this today, but that's always been part of the goal for Congress: to have a central point for admins to control the global policy being enforced throughout the datacenter/cloud. The other delta here is that the Congress policy language is general-purpose, so there's not a list of policy types that it will handle (Load Balancing, Placement, Energy). That generality comes with a price: that Congress must rely on other enforcement points, such as the ones below, to handle complicated policy enforcement problems. blockquote 2. Runtime-Policies -- Jay’s mail -- Focused on Runtime policies for Load Balancing, Availability, Energy, etc. (sequences of actions, rules, algorithms) /blockquote [Jay] Yes, exactly. blockquote 3. SolverScheduler -- Focused on Placement [ static or runtime ] and will be invoked by the (above) policy engines 4. Gantt – Focused on (Holistic) Scheduling /blockquote [Jay] For 3 and 4, I was always thinking Gantt is doing something for implementing SolverScheduler, not sure if run time policy can be included. blockquote 5. Neat -- seems to be a special case of Runtime-Policies (policies based on Load) Would this be correct understanding? We need to understand this to contribute to the right project. :) Thanks! -Gokul On Fri, Feb 28, 2014 at 5:46 PM, Jay Lau jay.lau@gmail.com wrote: blockquote Hi Yathiraj and Tim, Really appreciate your comments here ;-) I will prepare some detailed slides or documents before summit and we can have a review then. It would be great if OpenStack can provide DRS features. Thanks, Jay 2014-03-01 6:00 GMT+08:00 Tim Hinrichs thinri...@vmware.com : blockquote Hi Jay, I think the Solver Scheduler is a better fit for your needs than Congress because you know what kinds of constraints and enforcement you want. I'm not sure this topic deserves an entire design session--maybe just talking a bit at the summit would suffice (I *think* I'll be attending). Tim - Original Message - | From: Jay Lau jay.lau@gmail.com | To: OpenStack Development Mailing List (not for usage questions) openstack-dev@lists.openstack.org | Sent: Wednesday, February 26, 2014 6:30:54 PM | Subject: Re: [openstack-dev] [OpenStack][Runtime Policy] A proposal for OpenStack run time policy to manage | compute/storage resource | | | | | | | Hi Tim, | | I'm not sure if we can put resource monitor and adjust to | solver-scheduler
Re: [openstack-dev] [OpenStack][Runtime Policy] A proposal for OpenStack run time policy to manage compute/storage resource
Dear All, We are working on a framework where we want to monitor the system and take certain actions when specific events or situations occur. Here are two examples of 'different' situations: Example 1: A VM's-Owner and N/W's-owner are different == this could mean a violation == we need to take some action Example 2: A simple policy such as (VM-migrate of all VMs on possible node failure) OR (a more complex Energy Policy that may involve optimization). Both these examples need monitoring and actions to be taken when certain events happen (or through polling). However, the first one falls into the Compliance domain with Boolean conditions getting evaluated while the second one may require a more richer set of expression allowing for sequences or algorithms. So far, based on this discussion, it seems that these are *separate* initiatives in the community. I am understanding the Congress project to be in the domain of Boolean conditions (used for Compliance, etc.) where as the Run-time-policies (Jay's proposal) where policies can be expressed as rules, algorithms with higher-level goals. Is this understanding correct? Also, looking at all the mails, this is what I am reading: 1. Congress -- Focused on Compliance [ is that correct? ] (Boolean constraints and logic) 2. Runtime-Policies -- Jay's mail -- Focused on Runtime policies for Load Balancing, Availability, Energy, etc. (sequences of actions, rules, algorithms) 3. SolverScheduler -- Focused on Placement [ static or runtime ] and will be invoked by the (above) policy engines 4. Gantt - Focused on (Holistic) Scheduling 5. Neat -- seems to be a special case of Runtime-Policies (policies based on Load) Would this be correct understanding? We need to understand this to contribute to the right project. :) Thanks! -Gokul On Fri, Feb 28, 2014 at 5:46 PM, Jay Lau jay.lau@gmail.com wrote: Hi Yathiraj and Tim, Really appreciate your comments here ;-) I will prepare some detailed slides or documents before summit and we can have a review then. It would be great if OpenStack can provide DRS features. Thanks, Jay 2014-03-01 6:00 GMT+08:00 Tim Hinrichs thinri...@vmware.com: Hi Jay, I think the Solver Scheduler is a better fit for your needs than Congress because you know what kinds of constraints and enforcement you want. I'm not sure this topic deserves an entire design session--maybe just talking a bit at the summit would suffice (I *think* I'll be attending). Tim - Original Message - | From: Jay Lau jay.lau@gmail.com | To: OpenStack Development Mailing List (not for usage questions) openstack-dev@lists.openstack.org | Sent: Wednesday, February 26, 2014 6:30:54 PM | Subject: Re: [openstack-dev] [OpenStack][Runtime Policy] A proposal for OpenStack run time policy to manage | compute/storage resource | | | | | | | Hi Tim, | | I'm not sure if we can put resource monitor and adjust to | solver-scheduler (Gantt), but I have proposed this to Gantt design | [1], you can refer to [1] and search jay-lau-513. | | IMHO, Congress does monitoring and also take actions, but the actions | seems mainly for adjusting single VM network or storage. It did not | consider migrating VM according to hypervisor load. | | Not sure if this topic deserved to be a design session for the coming | summit, but I will try to propose. | | | | | [1] https://etherpad.openstack.org/p/icehouse-external-scheduler | | | | Thanks, | | | Jay | | | | 2014-02-27 1:48 GMT+08:00 Tim Hinrichs thinri...@vmware.com : | | | Hi Jay and Sylvain, | | The solver-scheduler sounds like a good fit to me as well. It clearly | provisions resources in accordance with policy. Does it monitor | those resources and adjust them if the system falls out of | compliance with the policy? | | I mentioned Congress for two reasons. (i) It does monitoring. (ii) | There was mention of compute, networking, and storage, and I | couldn't tell if the idea was for policy that spans OS components or | not. Congress was designed for policies spanning OS components. | | | Tim | | - Original Message - | | | From: Jay Lau jay.lau@gmail.com | | To: OpenStack Development Mailing List (not for usage questions) | | openstack-dev@lists.openstack.org | | | | Sent: Tuesday, February 25, 2014 10:13:14 PM | | Subject: Re: [openstack-dev] [OpenStack][Runtime Policy] A proposal | | for OpenStack run time policy to manage | | compute/storage resource | | | | | | | | | | | | Thanks Sylvain and Tim for the great sharing. | | | | @Tim, I also go through with Congress and have the same feeling | | with | | Sylvai, it is likely that Congress is doing something simliar with | | Gantt providing a holistic way for deploying. What I want to do is | | to provide some functions which is very similar with VMWare DRS | | that | | can do some adaptive
Re: [openstack-dev] [OpenStack][Runtime Policy] A proposal for OpenStack run time policy to manage compute/storage resource
Hi Jay, I think the Solver Scheduler is a better fit for your needs than Congress because you know what kinds of constraints and enforcement you want. I'm not sure this topic deserves an entire design session--maybe just talking a bit at the summit would suffice (I *think* I'll be attending). Tim - Original Message - | From: Jay Lau jay.lau@gmail.com | To: OpenStack Development Mailing List (not for usage questions) openstack-dev@lists.openstack.org | Sent: Wednesday, February 26, 2014 6:30:54 PM | Subject: Re: [openstack-dev] [OpenStack][Runtime Policy] A proposal for OpenStack run time policy to manage | compute/storage resource | | | | | | | Hi Tim, | | I'm not sure if we can put resource monitor and adjust to | solver-scheduler (Gantt), but I have proposed this to Gantt design | [1], you can refer to [1] and search jay-lau-513. | | IMHO, Congress does monitoring and also take actions, but the actions | seems mainly for adjusting single VM network or storage. It did not | consider migrating VM according to hypervisor load. | | Not sure if this topic deserved to be a design session for the coming | summit, but I will try to propose. | | | | | [1] https://etherpad.openstack.org/p/icehouse-external-scheduler | | | | Thanks, | | | Jay | | | | 2014-02-27 1:48 GMT+08:00 Tim Hinrichs thinri...@vmware.com : | | | Hi Jay and Sylvain, | | The solver-scheduler sounds like a good fit to me as well. It clearly | provisions resources in accordance with policy. Does it monitor | those resources and adjust them if the system falls out of | compliance with the policy? | | I mentioned Congress for two reasons. (i) It does monitoring. (ii) | There was mention of compute, networking, and storage, and I | couldn't tell if the idea was for policy that spans OS components or | not. Congress was designed for policies spanning OS components. | | | Tim | | - Original Message - | | | From: Jay Lau jay.lau@gmail.com | | To: OpenStack Development Mailing List (not for usage questions) | | openstack-dev@lists.openstack.org | | | | Sent: Tuesday, February 25, 2014 10:13:14 PM | | Subject: Re: [openstack-dev] [OpenStack][Runtime Policy] A proposal | | for OpenStack run time policy to manage | | compute/storage resource | | | | | | | | | | | | Thanks Sylvain and Tim for the great sharing. | | | | @Tim, I also go through with Congress and have the same feeling | | with | | Sylvai, it is likely that Congress is doing something simliar with | | Gantt providing a holistic way for deploying. What I want to do is | | to provide some functions which is very similar with VMWare DRS | | that | | can do some adaptive scheduling automatically. | | | | @Sylvain, can you please show more detail for what Pets vs. | | Cattles | | analogy means? | | | | | | | | | | 2014-02-26 9:11 GMT+08:00 Sylvain Bauza sylvain.ba...@gmail.com | | : | | | | | | | | Hi Tim, | | | | | | As per I'm reading your design document, it sounds more likely | | related to something like Solver Scheduler subteam is trying to | | focus on, ie. intelligent agnostic resources placement on an | | holistic way [1] | | IIRC, Jay is more likely talking about adaptive scheduling | | decisions | | based on feedback with potential counter-measures that can be done | | for decreasing load and preserving QoS of nodes. | | | | | | That said, maybe I'm wrong ? | | | | | | [1] https://blueprints.launchpad.net/nova/+spec/solver-scheduler | | | | | | | | 2014-02-26 1:09 GMT+01:00 Tim Hinrichs thinri...@vmware.com : | | | | | | | | | | Hi Jay, | | | | The Congress project aims to handle something similar to your use | | cases. I just sent a note to the ML with a Congress status update | | with the tag [Congress]. It includes links to our design docs. Let | | me know if you have trouble finding it or want to follow up. | | | | Tim | | | | | | | | - Original Message - | | | From: Sylvain Bauza sylvain.ba...@gmail.com | | | To: OpenStack Development Mailing List (not for usage | | | questions) | | | openstack-dev@lists.openstack.org | | | Sent: Tuesday, February 25, 2014 3:58:07 PM | | | Subject: Re: [openstack-dev] [OpenStack][Runtime Policy] A | | | proposal | | | for OpenStack run time policy to manage | | | compute/storage resource | | | | | | | | | | | | Hi Jay, | | | | | | | | | Currently, the Nova scheduler only acts upon user request (either | | | live migration or boot an instance). IMHO, that's something Gantt | | | should scope later on (or at least there could be some space | | | within | | | the Scheduler) so that Scheduler would be responsible for | | | managing | | | resources on a dynamic way. | | | | | | | | | I'm thinking of the Pets vs. Cattles analogy, and I definitely | | | think | | | that Compute resources could be treated like Pets, provided the | | | Scheduler does a move. | | | | | | | | | -Sylvain | | | | | | | | | | | | 2014-02-26 0:40 GMT+01
Re: [openstack-dev] [OpenStack][Runtime Policy] A proposal for OpenStack run time policy to manage compute/storage resource
Hi Yathiraj and Tim, Really appreciate your comments here ;-) I will prepare some detailed slides or documents before summit and we can have a review then. It would be great if OpenStack can provide DRS features. Thanks, Jay 2014-03-01 6:00 GMT+08:00 Tim Hinrichs thinri...@vmware.com: Hi Jay, I think the Solver Scheduler is a better fit for your needs than Congress because you know what kinds of constraints and enforcement you want. I'm not sure this topic deserves an entire design session--maybe just talking a bit at the summit would suffice (I *think* I'll be attending). Tim - Original Message - | From: Jay Lau jay.lau@gmail.com | To: OpenStack Development Mailing List (not for usage questions) openstack-dev@lists.openstack.org | Sent: Wednesday, February 26, 2014 6:30:54 PM | Subject: Re: [openstack-dev] [OpenStack][Runtime Policy] A proposal for OpenStack run time policy to manage | compute/storage resource | | | | | | | Hi Tim, | | I'm not sure if we can put resource monitor and adjust to | solver-scheduler (Gantt), but I have proposed this to Gantt design | [1], you can refer to [1] and search jay-lau-513. | | IMHO, Congress does monitoring and also take actions, but the actions | seems mainly for adjusting single VM network or storage. It did not | consider migrating VM according to hypervisor load. | | Not sure if this topic deserved to be a design session for the coming | summit, but I will try to propose. | | | | | [1] https://etherpad.openstack.org/p/icehouse-external-scheduler | | | | Thanks, | | | Jay | | | | 2014-02-27 1:48 GMT+08:00 Tim Hinrichs thinri...@vmware.com : | | | Hi Jay and Sylvain, | | The solver-scheduler sounds like a good fit to me as well. It clearly | provisions resources in accordance with policy. Does it monitor | those resources and adjust them if the system falls out of | compliance with the policy? | | I mentioned Congress for two reasons. (i) It does monitoring. (ii) | There was mention of compute, networking, and storage, and I | couldn't tell if the idea was for policy that spans OS components or | not. Congress was designed for policies spanning OS components. | | | Tim | | - Original Message - | | | From: Jay Lau jay.lau@gmail.com | | To: OpenStack Development Mailing List (not for usage questions) | | openstack-dev@lists.openstack.org | | | | Sent: Tuesday, February 25, 2014 10:13:14 PM | | Subject: Re: [openstack-dev] [OpenStack][Runtime Policy] A proposal | | for OpenStack run time policy to manage | | compute/storage resource | | | | | | | | | | | | Thanks Sylvain and Tim for the great sharing. | | | | @Tim, I also go through with Congress and have the same feeling | | with | | Sylvai, it is likely that Congress is doing something simliar with | | Gantt providing a holistic way for deploying. What I want to do is | | to provide some functions which is very similar with VMWare DRS | | that | | can do some adaptive scheduling automatically. | | | | @Sylvain, can you please show more detail for what Pets vs. | | Cattles | | analogy means? | | | | | | | | | | 2014-02-26 9:11 GMT+08:00 Sylvain Bauza sylvain.ba...@gmail.com | | : | | | | | | | | Hi Tim, | | | | | | As per I'm reading your design document, it sounds more likely | | related to something like Solver Scheduler subteam is trying to | | focus on, ie. intelligent agnostic resources placement on an | | holistic way [1] | | IIRC, Jay is more likely talking about adaptive scheduling | | decisions | | based on feedback with potential counter-measures that can be done | | for decreasing load and preserving QoS of nodes. | | | | | | That said, maybe I'm wrong ? | | | | | | [1] https://blueprints.launchpad.net/nova/+spec/solver-scheduler | | | | | | | | 2014-02-26 1:09 GMT+01:00 Tim Hinrichs thinri...@vmware.com : | | | | | | | | | | Hi Jay, | | | | The Congress project aims to handle something similar to your use | | cases. I just sent a note to the ML with a Congress status update | | with the tag [Congress]. It includes links to our design docs. Let | | me know if you have trouble finding it or want to follow up. | | | | Tim | | | | | | | | - Original Message - | | | From: Sylvain Bauza sylvain.ba...@gmail.com | | | To: OpenStack Development Mailing List (not for usage | | | questions) | | | openstack-dev@lists.openstack.org | | | Sent: Tuesday, February 25, 2014 3:58:07 PM | | | Subject: Re: [openstack-dev] [OpenStack][Runtime Policy] A | | | proposal | | | for OpenStack run time policy to manage | | | compute/storage resource | | | | | | | | | | | | Hi Jay, | | | | | | | | | Currently, the Nova scheduler only acts upon user request (either | | | live migration or boot an instance). IMHO, that's something Gantt | | | should scope later on (or at least there could be some space | | | within
Re: [openstack-dev] [OpenStack][Runtime Policy] A proposal for OpenStack run time policy to manage compute/storage resource
Hi Jay and Sylvain, The solver-scheduler sounds like a good fit to me as well. It clearly provisions resources in accordance with policy. Does it monitor those resources and adjust them if the system falls out of compliance with the policy? I mentioned Congress for two reasons. (i) It does monitoring. (ii) There was mention of compute, networking, and storage, and I couldn't tell if the idea was for policy that spans OS components or not. Congress was designed for policies spanning OS components. Tim - Original Message - | From: Jay Lau jay.lau@gmail.com | To: OpenStack Development Mailing List (not for usage questions) openstack-dev@lists.openstack.org | Sent: Tuesday, February 25, 2014 10:13:14 PM | Subject: Re: [openstack-dev] [OpenStack][Runtime Policy] A proposal for OpenStack run time policy to manage | compute/storage resource | | | | | | Thanks Sylvain and Tim for the great sharing. | | @Tim, I also go through with Congress and have the same feeling with | Sylvai, it is likely that Congress is doing something simliar with | Gantt providing a holistic way for deploying. What I want to do is | to provide some functions which is very similar with VMWare DRS that | can do some adaptive scheduling automatically. | | @Sylvain, can you please show more detail for what Pets vs. Cattles | analogy means? | | | | | 2014-02-26 9:11 GMT+08:00 Sylvain Bauza sylvain.ba...@gmail.com : | | | | Hi Tim, | | | As per I'm reading your design document, it sounds more likely | related to something like Solver Scheduler subteam is trying to | focus on, ie. intelligent agnostic resources placement on an | holistic way [1] | IIRC, Jay is more likely talking about adaptive scheduling decisions | based on feedback with potential counter-measures that can be done | for decreasing load and preserving QoS of nodes. | | | That said, maybe I'm wrong ? | | | [1] https://blueprints.launchpad.net/nova/+spec/solver-scheduler | | | | 2014-02-26 1:09 GMT+01:00 Tim Hinrichs thinri...@vmware.com : | | | | | Hi Jay, | | The Congress project aims to handle something similar to your use | cases. I just sent a note to the ML with a Congress status update | with the tag [Congress]. It includes links to our design docs. Let | me know if you have trouble finding it or want to follow up. | | Tim | | | | - Original Message - | | From: Sylvain Bauza sylvain.ba...@gmail.com | | To: OpenStack Development Mailing List (not for usage questions) | | openstack-dev@lists.openstack.org | | Sent: Tuesday, February 25, 2014 3:58:07 PM | | Subject: Re: [openstack-dev] [OpenStack][Runtime Policy] A proposal | | for OpenStack run time policy to manage | | compute/storage resource | | | | | | | | Hi Jay, | | | | | | Currently, the Nova scheduler only acts upon user request (either | | live migration or boot an instance). IMHO, that's something Gantt | | should scope later on (or at least there could be some space within | | the Scheduler) so that Scheduler would be responsible for managing | | resources on a dynamic way. | | | | | | I'm thinking of the Pets vs. Cattles analogy, and I definitely | | think | | that Compute resources could be treated like Pets, provided the | | Scheduler does a move. | | | | | | -Sylvain | | | | | | | | 2014-02-26 0:40 GMT+01:00 Jay Lau jay.lau@gmail.com : | | | | | | | | | | Greetings, | | | | | | Here I want to bring up an old topic here and want to get some | | input | | from you experts. | | | | | | Currently in nova and cinder, we only have some initial placement | | polices to help customer deploy VM instance or create volume | | storage | | to a specified host, but after the VM or the volume was created, | | there was no policy to monitor the hypervisors or the storage | | servers to take some actions in the following case: | | | | | | 1) Load Balance Policy: If the load of one server is too heavy, | | then | | probably we need to migrate some VMs from high load servers to some | | idle servers automatically to make sure the system resource usage | | can be balanced. | | | | 2) HA Policy: If one server get down for some hardware failure or | | whatever reasons, there is no policy to make sure the VMs can be | | evacuated or live migrated (Make sure migrate the VM before server | | goes down) to other available servers to make sure customer | | applications will not be affect too much. | | | | 3) Energy Saving Policy: If a single host load is lower than | | configured threshold, then low down the frequency of the CPU to | | save | | energy; otherwise, increase the CPU frequency. If the average load | | is lower than configured threshold, then shutdown some hypervisors | | to save energy; otherwise, power on some hypervisors to load | | balance. Before power off a hypervisor host, the energy policy need | | to live migrate all VMs on the hypervisor to other available | | hypervisors; After Power on a hypervisor host
Re: [openstack-dev] [OpenStack][Runtime Policy] A proposal for OpenStack run time policy to manage compute/storage resource
Hi Tim, I'm not sure if we can put resource monitor and adjust to solver-scheduler (Gantt), but I have proposed this to Gantt design [1], you can refer to [1] and search jay-lau-513. IMHO, Congress does monitoring and also take actions, but the actions seems mainly for adjusting single VM network or storage. It did not consider migrating VM according to hypervisor load. Not sure if this topic deserved to be a design session for the coming summit, but I will try to propose. [1] https://etherpad.openstack.org/p/icehouse-external-scheduler Thanks, Jay 2014-02-27 1:48 GMT+08:00 Tim Hinrichs thinri...@vmware.com: Hi Jay and Sylvain, The solver-scheduler sounds like a good fit to me as well. It clearly provisions resources in accordance with policy. Does it monitor those resources and adjust them if the system falls out of compliance with the policy? I mentioned Congress for two reasons. (i) It does monitoring. (ii) There was mention of compute, networking, and storage, and I couldn't tell if the idea was for policy that spans OS components or not. Congress was designed for policies spanning OS components. Tim - Original Message - | From: Jay Lau jay.lau@gmail.com | To: OpenStack Development Mailing List (not for usage questions) openstack-dev@lists.openstack.org | Sent: Tuesday, February 25, 2014 10:13:14 PM | Subject: Re: [openstack-dev] [OpenStack][Runtime Policy] A proposal for OpenStack run time policy to manage | compute/storage resource | | | | | | Thanks Sylvain and Tim for the great sharing. | | @Tim, I also go through with Congress and have the same feeling with | Sylvai, it is likely that Congress is doing something simliar with | Gantt providing a holistic way for deploying. What I want to do is | to provide some functions which is very similar with VMWare DRS that | can do some adaptive scheduling automatically. | | @Sylvain, can you please show more detail for what Pets vs. Cattles | analogy means? | | | | | 2014-02-26 9:11 GMT+08:00 Sylvain Bauza sylvain.ba...@gmail.com : | | | | Hi Tim, | | | As per I'm reading your design document, it sounds more likely | related to something like Solver Scheduler subteam is trying to | focus on, ie. intelligent agnostic resources placement on an | holistic way [1] | IIRC, Jay is more likely talking about adaptive scheduling decisions | based on feedback with potential counter-measures that can be done | for decreasing load and preserving QoS of nodes. | | | That said, maybe I'm wrong ? | | | [1] https://blueprints.launchpad.net/nova/+spec/solver-scheduler | | | | 2014-02-26 1:09 GMT+01:00 Tim Hinrichs thinri...@vmware.com : | | | | | Hi Jay, | | The Congress project aims to handle something similar to your use | cases. I just sent a note to the ML with a Congress status update | with the tag [Congress]. It includes links to our design docs. Let | me know if you have trouble finding it or want to follow up. | | Tim | | | | - Original Message - | | From: Sylvain Bauza sylvain.ba...@gmail.com | | To: OpenStack Development Mailing List (not for usage questions) | | openstack-dev@lists.openstack.org | | Sent: Tuesday, February 25, 2014 3:58:07 PM | | Subject: Re: [openstack-dev] [OpenStack][Runtime Policy] A proposal | | for OpenStack run time policy to manage | | compute/storage resource | | | | | | | | Hi Jay, | | | | | | Currently, the Nova scheduler only acts upon user request (either | | live migration or boot an instance). IMHO, that's something Gantt | | should scope later on (or at least there could be some space within | | the Scheduler) so that Scheduler would be responsible for managing | | resources on a dynamic way. | | | | | | I'm thinking of the Pets vs. Cattles analogy, and I definitely | | think | | that Compute resources could be treated like Pets, provided the | | Scheduler does a move. | | | | | | -Sylvain | | | | | | | | 2014-02-26 0:40 GMT+01:00 Jay Lau jay.lau@gmail.com : | | | | | | | | | | Greetings, | | | | | | Here I want to bring up an old topic here and want to get some | | input | | from you experts. | | | | | | Currently in nova and cinder, we only have some initial placement | | polices to help customer deploy VM instance or create volume | | storage | | to a specified host, but after the VM or the volume was created, | | there was no policy to monitor the hypervisors or the storage | | servers to take some actions in the following case: | | | | | | 1) Load Balance Policy: If the load of one server is too heavy, | | then | | probably we need to migrate some VMs from high load servers to some | | idle servers automatically to make sure the system resource usage | | can be balanced. | | | | 2) HA Policy: If one server get down for some hardware failure or | | whatever reasons, there is no policy to make sure the VMs can be | | evacuated or live
Re: [openstack-dev] [OpenStack][Runtime Policy] A proposal for OpenStack run time policy to manage compute/storage resource
Hi Jay, Tim, Sylvain, It is an important topic of run time monitoring and policies for management of compute/storage resources. I agree this is the kind of solution that we should learn from Vmware DRS. Regarding how the Solver Scheduler fits in, we initiated this effort to address complex constraint and policy scenarios, and the kinds of policies and constraints that become relevant for Load balancing, HA, energy saving, etc are all relevant constraints that we care about addressing and solving using the Solver scheduler. However you should consider the solver scheduler as a pure placement decision engine, whether it is used for the initial placement, or if it is used for automatically triggering any action (such as a migration etc.) at runtime based on the constraints/policy checking. As of now, the Solver Scheduler effort has been started to address the initial Nova placement decision, as a new scheduler driver. But eventually with the Gantt efforts, of having scheduler as a separate service, and the resulting new apis, it is heading towards being a true decision engine, that can be triggered at any point of time, to figure out the ideal placements given the current state of the resource capacities. But the kinds of policies you have mentioned here, appear as a true fit for being addressed as the complex constraints and cost metrics that can be used to be solved the Solver Scheduler’s intelligent constraints-based optimization solver. We (the solver scheduler sub-team) will be very happy to work with you to address some of these scenarios, and we can have a session at the upcoming summit regarding this kind of run-time policies. Thanks, Yathi. — On 2/26/14, 6:30 PM, Jay Lau jay.lau@gmail.commailto:jay.lau@gmail.com wrote: Hi Tim, I'm not sure if we can put resource monitor and adjust to solver-scheduler (Gantt), but I have proposed this to Gantt design [1], you can refer to [1] and search jay-lau-513. IMHO, Congress does monitoring and also take actions, but the actions seems mainly for adjusting single VM network or storage. It did not consider migrating VM according to hypervisor load. Not sure if this topic deserved to be a design session for the coming summit, but I will try to propose. [1] https://etherpad.openstack.org/p/icehouse-external-scheduler Thanks, Jay 2014-02-27 1:48 GMT+08:00 Tim Hinrichs thinri...@vmware.commailto:thinri...@vmware.com: Hi Jay and Sylvain, The solver-scheduler sounds like a good fit to me as well. It clearly provisions resources in accordance with policy. Does it monitor those resources and adjust them if the system falls out of compliance with the policy? I mentioned Congress for two reasons. (i) It does monitoring. (ii) There was mention of compute, networking, and storage, and I couldn't tell if the idea was for policy that spans OS components or not. Congress was designed for policies spanning OS components. Tim - Original Message - | From: Jay Lau jay.lau@gmail.commailto:jay.lau@gmail.com | To: OpenStack Development Mailing List (not for usage questions) openstack-dev@lists.openstack.orgmailto:openstack-dev@lists.openstack.org | Sent: Tuesday, February 25, 2014 10:13:14 PM | Subject: Re: [openstack-dev] [OpenStack][Runtime Policy] A proposal for OpenStack run time policy to manage | compute/storage resource | | | | | | Thanks Sylvain and Tim for the great sharing. | | @Tim, I also go through with Congress and have the same feeling with | Sylvai, it is likely that Congress is doing something simliar with | Gantt providing a holistic way for deploying. What I want to do is | to provide some functions which is very similar with VMWare DRS that | can do some adaptive scheduling automatically. | | @Sylvain, can you please show more detail for what Pets vs. Cattles | analogy means? | | | | | 2014-02-26 9:11 GMT+08:00 Sylvain Bauza sylvain.ba...@gmail.commailto:sylvain.ba...@gmail.com : | | | | Hi Tim, | | | As per I'm reading your design document, it sounds more likely | related to something like Solver Scheduler subteam is trying to | focus on, ie. intelligent agnostic resources placement on an | holistic way [1] | IIRC, Jay is more likely talking about adaptive scheduling decisions | based on feedback with potential counter-measures that can be done | for decreasing load and preserving QoS of nodes. | | | That said, maybe I'm wrong ? | | | [1] https://blueprints.launchpad.net/nova/+spec/solver-scheduler | | | | 2014-02-26 1:09 GMT+01:00 Tim Hinrichs thinri...@vmware.commailto:thinri...@vmware.com : | | | | | Hi Jay, | | The Congress project aims to handle something similar to your use | cases. I just sent a note to the ML with a Congress status update | with the tag [Congress]. It includes links to our design docs. Let | me know if you have trouble finding it or want to follow up. | | Tim | | | | - Original Message - | | From: Sylvain Bauza sylvain.ba
[openstack-dev] [OpenStack][Runtime Policy] A proposal for OpenStack run time policy to manage compute/storage resource
Greetings, Here I want to bring up an old topic here and want to get some input from you experts. Currently in nova and cinder, we only have some initial placement polices to help customer deploy VM instance or create volume storage to a specified host, but after the VM or the volume was created, there was no policy to monitor the hypervisors or the storage servers to take some actions in the following case: 1) Load Balance Policy: If the load of one server is too heavy, then probably we need to migrate some VMs from high load servers to some idle servers automatically to make sure the system resource usage can be balanced. 2) HA Policy: If one server get down for some hardware failure or whatever reasons, there is no policy to make sure the VMs can be evacuated or live migrated (Make sure migrate the VM before server goes down) to other available servers to make sure customer applications will not be affect too much. 3) Energy Saving Policy: If a single host load is lower than configured threshold, then low down the frequency of the CPU to save energy; otherwise, increase the CPU frequency. If the average load is lower than configured threshold, then shutdown some hypervisors to save energy; otherwise, power on some hypervisors to load balance. Before power off a hypervisor host, the energy policy need to live migrate all VMs on the hypervisor to other available hypervisors; After Power on a hypervisor host, the Load Balance Policy will help live migrate some VMs to the new powered hypervisor. 4) Customized Policy: Customer can also define some customized policies based on their specified requirement. 5) Some run-time policies for block storage or even network. I borrow the idea from VMWare DRS (Thanks VMWare DRS), and there indeed many customers want such features. I have filed a bp here [1] long ago, but after some discussion with Russell, we think that this should not belong to nova but other projects. Till now, I did not find a good place where we can put this in, can any of you show some comments? [1] https://blueprints.launchpad.net/nova/+spec/resource-optimization-service -- Thanks, Jay ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [OpenStack][Runtime Policy] A proposal for OpenStack run time policy to manage compute/storage resource
Hi Jay, Currently, the Nova scheduler only acts upon user request (either live migration or boot an instance). IMHO, that's something Gantt should scope later on (or at least there could be some space within the Scheduler) so that Scheduler would be responsible for managing resources on a dynamic way. I'm thinking of the Pets vs. Cattles analogy, and I definitely think that Compute resources could be treated like Pets, provided the Scheduler does a move. -Sylvain 2014-02-26 0:40 GMT+01:00 Jay Lau jay.lau@gmail.com: Greetings, Here I want to bring up an old topic here and want to get some input from you experts. Currently in nova and cinder, we only have some initial placement polices to help customer deploy VM instance or create volume storage to a specified host, but after the VM or the volume was created, there was no policy to monitor the hypervisors or the storage servers to take some actions in the following case: 1) Load Balance Policy: If the load of one server is too heavy, then probably we need to migrate some VMs from high load servers to some idle servers automatically to make sure the system resource usage can be balanced. 2) HA Policy: If one server get down for some hardware failure or whatever reasons, there is no policy to make sure the VMs can be evacuated or live migrated (Make sure migrate the VM before server goes down) to other available servers to make sure customer applications will not be affect too much. 3) Energy Saving Policy: If a single host load is lower than configured threshold, then low down the frequency of the CPU to save energy; otherwise, increase the CPU frequency. If the average load is lower than configured threshold, then shutdown some hypervisors to save energy; otherwise, power on some hypervisors to load balance. Before power off a hypervisor host, the energy policy need to live migrate all VMs on the hypervisor to other available hypervisors; After Power on a hypervisor host, the Load Balance Policy will help live migrate some VMs to the new powered hypervisor. 4) Customized Policy: Customer can also define some customized policies based on their specified requirement. 5) Some run-time policies for block storage or even network. I borrow the idea from VMWare DRS (Thanks VMWare DRS), and there indeed many customers want such features. I have filed a bp here [1] long ago, but after some discussion with Russell, we think that this should not belong to nova but other projects. Till now, I did not find a good place where we can put this in, can any of you show some comments? [1] https://blueprints.launchpad.net/nova/+spec/resource-optimization-service -- Thanks, Jay ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [OpenStack][Runtime Policy] A proposal for OpenStack run time policy to manage compute/storage resource
Hi Jay, The Congress project aims to handle something similar to your use cases. I just sent a note to the ML with a Congress status update with the tag [Congress]. It includes links to our design docs. Let me know if you have trouble finding it or want to follow up. Tim - Original Message - | From: Sylvain Bauza sylvain.ba...@gmail.com | To: OpenStack Development Mailing List (not for usage questions) openstack-dev@lists.openstack.org | Sent: Tuesday, February 25, 2014 3:58:07 PM | Subject: Re: [openstack-dev] [OpenStack][Runtime Policy] A proposal for OpenStack run time policy to manage | compute/storage resource | | | | Hi Jay, | | | Currently, the Nova scheduler only acts upon user request (either | live migration or boot an instance). IMHO, that's something Gantt | should scope later on (or at least there could be some space within | the Scheduler) so that Scheduler would be responsible for managing | resources on a dynamic way. | | | I'm thinking of the Pets vs. Cattles analogy, and I definitely think | that Compute resources could be treated like Pets, provided the | Scheduler does a move. | | | -Sylvain | | | | 2014-02-26 0:40 GMT+01:00 Jay Lau jay.lau@gmail.com : | | | | | Greetings, | | | Here I want to bring up an old topic here and want to get some input | from you experts. | | | Currently in nova and cinder, we only have some initial placement | polices to help customer deploy VM instance or create volume storage | to a specified host, but after the VM or the volume was created, | there was no policy to monitor the hypervisors or the storage | servers to take some actions in the following case: | | | 1) Load Balance Policy: If the load of one server is too heavy, then | probably we need to migrate some VMs from high load servers to some | idle servers automatically to make sure the system resource usage | can be balanced. | | 2) HA Policy: If one server get down for some hardware failure or | whatever reasons, there is no policy to make sure the VMs can be | evacuated or live migrated (Make sure migrate the VM before server | goes down) to other available servers to make sure customer | applications will not be affect too much. | | 3) Energy Saving Policy: If a single host load is lower than | configured threshold, then low down the frequency of the CPU to save | energy; otherwise, increase the CPU frequency. If the average load | is lower than configured threshold, then shutdown some hypervisors | to save energy; otherwise, power on some hypervisors to load | balance. Before power off a hypervisor host, the energy policy need | to live migrate all VMs on the hypervisor to other available | hypervisors; After Power on a hypervisor host, the Load Balance | Policy will help live migrate some VMs to the new powered | hypervisor. | | 4) Customized Policy: Customer can also define some customized | policies based on their specified requirement. | | 5) Some run-time policies for block storage or even network. | | | | I borrow the idea from VMWare DRS (Thanks VMWare DRS), and there | indeed many customers want such features. | | | | I have filed a bp here [1] long ago, but after some discussion with | Russell, we think that this should not belong to nova but other | projects. Till now, I did not find a good place where we can put | this in, can any of you show some comments? | | | | [1] | https://blueprints.launchpad.net/nova/+spec/resource-optimization-service | | -- | | | Thanks, | | Jay | | ___ | OpenStack-dev mailing list | OpenStack-dev@lists.openstack.org | http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev | | | | ___ | OpenStack-dev mailing list | OpenStack-dev@lists.openstack.org | https://urldefense.proofpoint.com/v1/url?u=http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-devk=oIvRg1%2BdGAgOoM1BIlLLqw%3D%3D%0Ar=%2FZ35AkRhp2kCW4Q3MPeE%2BxY2bqaf%2FKm29ZfiqAKXxeo%3D%0Am=XDB3hT4WE2iDrNVK0sQ8qKooX2r1T4E%2BVHek3GREhnE%3D%0As=e2346cd017c9d8108c12a101892492e2ac75953e4a5ea5c17394c775cf086d7f | ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [OpenStack][Runtime Policy] A proposal for OpenStack run time policy to manage compute/storage resource
Hi Tim, As per I'm reading your design document, it sounds more likely related to something like Solver Scheduler subteam is trying to focus on, ie. intelligent agnostic resources placement on an holistic way [1] IIRC, Jay is more likely talking about adaptive scheduling decisions based on feedback with potential counter-measures that can be done for decreasing load and preserving QoS of nodes. That said, maybe I'm wrong ? [1]https://blueprints.launchpad.net/nova/+spec/solver-scheduler 2014-02-26 1:09 GMT+01:00 Tim Hinrichs thinri...@vmware.com: Hi Jay, The Congress project aims to handle something similar to your use cases. I just sent a note to the ML with a Congress status update with the tag [Congress]. It includes links to our design docs. Let me know if you have trouble finding it or want to follow up. Tim - Original Message - | From: Sylvain Bauza sylvain.ba...@gmail.com | To: OpenStack Development Mailing List (not for usage questions) openstack-dev@lists.openstack.org | Sent: Tuesday, February 25, 2014 3:58:07 PM | Subject: Re: [openstack-dev] [OpenStack][Runtime Policy] A proposal for OpenStack run time policy to manage | compute/storage resource | | | | Hi Jay, | | | Currently, the Nova scheduler only acts upon user request (either | live migration or boot an instance). IMHO, that's something Gantt | should scope later on (or at least there could be some space within | the Scheduler) so that Scheduler would be responsible for managing | resources on a dynamic way. | | | I'm thinking of the Pets vs. Cattles analogy, and I definitely think | that Compute resources could be treated like Pets, provided the | Scheduler does a move. | | | -Sylvain | | | | 2014-02-26 0:40 GMT+01:00 Jay Lau jay.lau@gmail.com : | | | | | Greetings, | | | Here I want to bring up an old topic here and want to get some input | from you experts. | | | Currently in nova and cinder, we only have some initial placement | polices to help customer deploy VM instance or create volume storage | to a specified host, but after the VM or the volume was created, | there was no policy to monitor the hypervisors or the storage | servers to take some actions in the following case: | | | 1) Load Balance Policy: If the load of one server is too heavy, then | probably we need to migrate some VMs from high load servers to some | idle servers automatically to make sure the system resource usage | can be balanced. | | 2) HA Policy: If one server get down for some hardware failure or | whatever reasons, there is no policy to make sure the VMs can be | evacuated or live migrated (Make sure migrate the VM before server | goes down) to other available servers to make sure customer | applications will not be affect too much. | | 3) Energy Saving Policy: If a single host load is lower than | configured threshold, then low down the frequency of the CPU to save | energy; otherwise, increase the CPU frequency. If the average load | is lower than configured threshold, then shutdown some hypervisors | to save energy; otherwise, power on some hypervisors to load | balance. Before power off a hypervisor host, the energy policy need | to live migrate all VMs on the hypervisor to other available | hypervisors; After Power on a hypervisor host, the Load Balance | Policy will help live migrate some VMs to the new powered | hypervisor. | | 4) Customized Policy: Customer can also define some customized | policies based on their specified requirement. | | 5) Some run-time policies for block storage or even network. | | | | I borrow the idea from VMWare DRS (Thanks VMWare DRS), and there | indeed many customers want such features. | | | | I have filed a bp here [1] long ago, but after some discussion with | Russell, we think that this should not belong to nova but other | projects. Till now, I did not find a good place where we can put | this in, can any of you show some comments? | | | | [1] | https://blueprints.launchpad.net/nova/+spec/resource-optimization-service | | -- | | | Thanks, | | Jay | | ___ | OpenStack-dev mailing list | OpenStack-dev@lists.openstack.org | http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev | | | | ___ | OpenStack-dev mailing list | OpenStack-dev@lists.openstack.org | https://urldefense.proofpoint.com/v1/url?u=http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-devk=oIvRg1%2BdGAgOoM1BIlLLqw%3D%3D%0Ar=%2FZ35AkRhp2kCW4Q3MPeE%2BxY2bqaf%2FKm29ZfiqAKXxeo%3D%0Am=XDB3hT4WE2iDrNVK0sQ8qKooX2r1T4E%2BVHek3GREhnE%3D%0As=e2346cd017c9d8108c12a101892492e2ac75953e4a5ea5c17394c775cf086d7f | ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [OpenStack][Runtime Policy] A proposal for OpenStack run time policy to manage compute/storage resource
Hi, Jay Sylvain: I found the OpenStack-Neat Project (http://openstack-neat.org/) have already aimed to do the things similar to DRS and DPM. Hope it will be helpful. -- Leiqzhang Best Regards From: Sylvain Bauza [mailto:sylvain.ba...@gmail.com] Sent: Wednesday, February 26, 2014 9:11 AM To: OpenStack Development Mailing List (not for usage questions) Subject: Re: [openstack-dev] [OpenStack][Runtime Policy] A proposal for OpenStack run time policy to manage compute/storage resource Hi Tim, As per I'm reading your design document, it sounds more likely related to something like Solver Scheduler subteam is trying to focus on, ie. intelligent agnostic resources placement on an holistic way [1] IIRC, Jay is more likely talking about adaptive scheduling decisions based on feedback with potential counter-measures that can be done for decreasing load and preserving QoS of nodes. That said, maybe I'm wrong ? [1]https://blueprints.launchpad.net/nova/+spec/solver-scheduler 2014-02-26 1:09 GMT+01:00 Tim Hinrichs thinri...@vmware.commailto:thinri...@vmware.com: Hi Jay, The Congress project aims to handle something similar to your use cases. I just sent a note to the ML with a Congress status update with the tag [Congress]. It includes links to our design docs. Let me know if you have trouble finding it or want to follow up. Tim - Original Message - | From: Sylvain Bauza sylvain.ba...@gmail.commailto:sylvain.ba...@gmail.com | To: OpenStack Development Mailing List (not for usage questions) openstack-dev@lists.openstack.orgmailto:openstack-dev@lists.openstack.org | Sent: Tuesday, February 25, 2014 3:58:07 PM | Subject: Re: [openstack-dev] [OpenStack][Runtime Policy] A proposal for OpenStack run time policy to manage | compute/storage resource | | | | Hi Jay, | | | Currently, the Nova scheduler only acts upon user request (either | live migration or boot an instance). IMHO, that's something Gantt | should scope later on (or at least there could be some space within | the Scheduler) so that Scheduler would be responsible for managing | resources on a dynamic way. | | | I'm thinking of the Pets vs. Cattles analogy, and I definitely think | that Compute resources could be treated like Pets, provided the | Scheduler does a move. | | | -Sylvain | | | | 2014-02-26 0:40 GMT+01:00 Jay Lau jay.lau@gmail.commailto:jay.lau@gmail.com : | | | | | Greetings, | | | Here I want to bring up an old topic here and want to get some input | from you experts. | | | Currently in nova and cinder, we only have some initial placement | polices to help customer deploy VM instance or create volume storage | to a specified host, but after the VM or the volume was created, | there was no policy to monitor the hypervisors or the storage | servers to take some actions in the following case: | | | 1) Load Balance Policy: If the load of one server is too heavy, then | probably we need to migrate some VMs from high load servers to some | idle servers automatically to make sure the system resource usage | can be balanced. | | 2) HA Policy: If one server get down for some hardware failure or | whatever reasons, there is no policy to make sure the VMs can be | evacuated or live migrated (Make sure migrate the VM before server | goes down) to other available servers to make sure customer | applications will not be affect too much. | | 3) Energy Saving Policy: If a single host load is lower than | configured threshold, then low down the frequency of the CPU to save | energy; otherwise, increase the CPU frequency. If the average load | is lower than configured threshold, then shutdown some hypervisors | to save energy; otherwise, power on some hypervisors to load | balance. Before power off a hypervisor host, the energy policy need | to live migrate all VMs on the hypervisor to other available | hypervisors; After Power on a hypervisor host, the Load Balance | Policy will help live migrate some VMs to the new powered | hypervisor. | | 4) Customized Policy: Customer can also define some customized | policies based on their specified requirement. | | 5) Some run-time policies for block storage or even network. | | | | I borrow the idea from VMWare DRS (Thanks VMWare DRS), and there | indeed many customers want such features. | | | | I have filed a bp here [1] long ago, but after some discussion with | Russell, we think that this should not belong to nova but other | projects. Till now, I did not find a good place where we can put | this in, can any of you show some comments? | | | | [1] | https://blueprints.launchpad.net/nova/+spec/resource-optimization-service | | -- | | | Thanks, | | Jay | | ___ | OpenStack-dev mailing list | OpenStack-dev@lists.openstack.orgmailto:OpenStack-dev@lists.openstack.org | http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev | | | | ___ | OpenStack-dev mailing list | OpenStack
Re: [openstack-dev] [OpenStack][Runtime Policy] A proposal for OpenStack run time policy to manage compute/storage resource
Thanks Sylvain and Tim for the great sharing. @Tim, I also go through with Congress and have the same feeling with Sylvai, it is likely that Congress is doing something simliar with Gantt providing a holistic way for deploying. What I want to do is to provide some functions which is very similar with VMWare DRS that can do some adaptive scheduling automatically. @Sylvain, can you please show more detail for what Pets vs. Cattles analogy means? 2014-02-26 9:11 GMT+08:00 Sylvain Bauza sylvain.ba...@gmail.com: Hi Tim, As per I'm reading your design document, it sounds more likely related to something like Solver Scheduler subteam is trying to focus on, ie. intelligent agnostic resources placement on an holistic way [1] IIRC, Jay is more likely talking about adaptive scheduling decisions based on feedback with potential counter-measures that can be done for decreasing load and preserving QoS of nodes. That said, maybe I'm wrong ? [1]https://blueprints.launchpad.net/nova/+spec/solver-scheduler 2014-02-26 1:09 GMT+01:00 Tim Hinrichs thinri...@vmware.com: Hi Jay, The Congress project aims to handle something similar to your use cases. I just sent a note to the ML with a Congress status update with the tag [Congress]. It includes links to our design docs. Let me know if you have trouble finding it or want to follow up. Tim - Original Message - | From: Sylvain Bauza sylvain.ba...@gmail.com | To: OpenStack Development Mailing List (not for usage questions) openstack-dev@lists.openstack.org | Sent: Tuesday, February 25, 2014 3:58:07 PM | Subject: Re: [openstack-dev] [OpenStack][Runtime Policy] A proposal for OpenStack run time policy to manage | compute/storage resource | | | | Hi Jay, | | | Currently, the Nova scheduler only acts upon user request (either | live migration or boot an instance). IMHO, that's something Gantt | should scope later on (or at least there could be some space within | the Scheduler) so that Scheduler would be responsible for managing | resources on a dynamic way. | | | I'm thinking of the Pets vs. Cattles analogy, and I definitely think | that Compute resources could be treated like Pets, provided the | Scheduler does a move. | | | -Sylvain | | | | 2014-02-26 0:40 GMT+01:00 Jay Lau jay.lau@gmail.com : | | | | | Greetings, | | | Here I want to bring up an old topic here and want to get some input | from you experts. | | | Currently in nova and cinder, we only have some initial placement | polices to help customer deploy VM instance or create volume storage | to a specified host, but after the VM or the volume was created, | there was no policy to monitor the hypervisors or the storage | servers to take some actions in the following case: | | | 1) Load Balance Policy: If the load of one server is too heavy, then | probably we need to migrate some VMs from high load servers to some | idle servers automatically to make sure the system resource usage | can be balanced. | | 2) HA Policy: If one server get down for some hardware failure or | whatever reasons, there is no policy to make sure the VMs can be | evacuated or live migrated (Make sure migrate the VM before server | goes down) to other available servers to make sure customer | applications will not be affect too much. | | 3) Energy Saving Policy: If a single host load is lower than | configured threshold, then low down the frequency of the CPU to save | energy; otherwise, increase the CPU frequency. If the average load | is lower than configured threshold, then shutdown some hypervisors | to save energy; otherwise, power on some hypervisors to load | balance. Before power off a hypervisor host, the energy policy need | to live migrate all VMs on the hypervisor to other available | hypervisors; After Power on a hypervisor host, the Load Balance | Policy will help live migrate some VMs to the new powered | hypervisor. | | 4) Customized Policy: Customer can also define some customized | policies based on their specified requirement. | | 5) Some run-time policies for block storage or even network. | | | | I borrow the idea from VMWare DRS (Thanks VMWare DRS), and there | indeed many customers want such features. | | | | I have filed a bp here [1] long ago, but after some discussion with | Russell, we think that this should not belong to nova but other | projects. Till now, I did not find a good place where we can put | this in, can any of you show some comments? | | | | [1] | https://blueprints.launchpad.net/nova/+spec/resource-optimization-service | | -- | | | Thanks, | | Jay | | ___ | OpenStack-dev mailing list | OpenStack-dev@lists.openstack.org | http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev | | | | ___ | OpenStack-dev mailing list | OpenStack-dev@lists.openstack.org
Re: [openstack-dev] [OpenStack][Runtime Policy] A proposal for OpenStack run time policy to manage compute/storage resource
@Zhangleiqiang, thanks for the info, yes, it does provide load balance and DPM. What I want to do is not only those two policies but also HA or some customized policies just like openstack nova filters, also I hope that this policy can manage not only compute resource, but also storage, network etc. 2014-02-26 12:16 GMT+08:00 Zhangleiqiang zhangleiqi...@huawei.com: Hi, Jay Sylvain: I found the OpenStack-Neat Project (http://openstack-neat.org/) have already aimed to do the things similar to DRS and DPM. Hope it will be helpful. -- Leiqzhang Best Regards *From:* Sylvain Bauza [mailto:sylvain.ba...@gmail.com] *Sent:* Wednesday, February 26, 2014 9:11 AM *To:* OpenStack Development Mailing List (not for usage questions) *Subject:* Re: [openstack-dev] [OpenStack][Runtime Policy] A proposal for OpenStack run time policy to manage compute/storage resource Hi Tim, As per I'm reading your design document, it sounds more likely related to something like Solver Scheduler subteam is trying to focus on, ie. intelligent agnostic resources placement on an holistic way [1] IIRC, Jay is more likely talking about adaptive scheduling decisions based on feedback with potential counter-measures that can be done for decreasing load and preserving QoS of nodes. That said, maybe I'm wrong ? [1]https://blueprints.launchpad.net/nova/+spec/solver-scheduler 2014-02-26 1:09 GMT+01:00 Tim Hinrichs thinri...@vmware.com: Hi Jay, The Congress project aims to handle something similar to your use cases. I just sent a note to the ML with a Congress status update with the tag [Congress]. It includes links to our design docs. Let me know if you have trouble finding it or want to follow up. Tim - Original Message - | From: Sylvain Bauza sylvain.ba...@gmail.com | To: OpenStack Development Mailing List (not for usage questions) openstack-dev@lists.openstack.org | Sent: Tuesday, February 25, 2014 3:58:07 PM | Subject: Re: [openstack-dev] [OpenStack][Runtime Policy] A proposal for OpenStack run time policy to manage | compute/storage resource | | | | Hi Jay, | | | Currently, the Nova scheduler only acts upon user request (either | live migration or boot an instance). IMHO, that's something Gantt | should scope later on (or at least there could be some space within | the Scheduler) so that Scheduler would be responsible for managing | resources on a dynamic way. | | | I'm thinking of the Pets vs. Cattles analogy, and I definitely think | that Compute resources could be treated like Pets, provided the | Scheduler does a move. | | | -Sylvain | | | | 2014-02-26 0:40 GMT+01:00 Jay Lau jay.lau@gmail.com : | | | | | Greetings, | | | Here I want to bring up an old topic here and want to get some input | from you experts. | | | Currently in nova and cinder, we only have some initial placement | polices to help customer deploy VM instance or create volume storage | to a specified host, but after the VM or the volume was created, | there was no policy to monitor the hypervisors or the storage | servers to take some actions in the following case: | | | 1) Load Balance Policy: If the load of one server is too heavy, then | probably we need to migrate some VMs from high load servers to some | idle servers automatically to make sure the system resource usage | can be balanced. | | 2) HA Policy: If one server get down for some hardware failure or | whatever reasons, there is no policy to make sure the VMs can be | evacuated or live migrated (Make sure migrate the VM before server | goes down) to other available servers to make sure customer | applications will not be affect too much. | | 3) Energy Saving Policy: If a single host load is lower than | configured threshold, then low down the frequency of the CPU to save | energy; otherwise, increase the CPU frequency. If the average load | is lower than configured threshold, then shutdown some hypervisors | to save energy; otherwise, power on some hypervisors to load | balance. Before power off a hypervisor host, the energy policy need | to live migrate all VMs on the hypervisor to other available | hypervisors; After Power on a hypervisor host, the Load Balance | Policy will help live migrate some VMs to the new powered | hypervisor. | | 4) Customized Policy: Customer can also define some customized | policies based on their specified requirement. | | 5) Some run-time policies for block storage or even network. | | | | I borrow the idea from VMWare DRS (Thanks VMWare DRS), and there | indeed many customers want such features. | | | | I have filed a bp here [1] long ago, but after some discussion with | Russell, we think that this should not belong to nova but other | projects. Till now, I did not find a good place where we can put | this in, can any of you show some comments? | | | | [1] | https
