Re: [openstack-dev] [nova][scheduler] Proposal: FairShareScheduler.
Hi Joe, we know the Amazon's hot market is based on a economic model. Suppose to assign to each group a virtual money amount for renting hot instances. So, the money may reflect the share concept in our model. This means that a big money amount may rent potentially more resources (actually it depends by the current resource price, the offer, etc). As in the real World, this economic model emphasizes the difference between rich and poor. So a rich group may rent all resources available for long time. For sure this approach maximizes the resource utilization but it is not so fair. It should guarantees that the usage of the resources is fairly distributed among users and teams according to their mean number of VMs they have got by considering the portion of the resources allocated to them (i.e. share) and the evaluation of the effective resource usage consumed in the recent past. Probably this issue may be solved by defining fair lease contracts between the user and the resource provider. What do you think? To make it more clear, for simplicity let's say that we have only two teams, A and B who are undertaking activities of Type3. The site administrator assigns 1000$ to the group A and just 100$ to B. Suppose A and B need both more resources at the same time. So, for sure B will be able to rent something only when A releases its resources or when A becomes poor. Instead, with a fair-share algorithm, B would be able to rent some resources because the purchasing power of A is adjusted by the recent trading. At the moment I'm still not really sure that this approach can cover this use case. thanks a lot. Cheers, Lisa On Tue, Jul 8, 2014 at 4:18 AM, Lisa lisa.zangrando at pd.infn.it http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev wrote: / Hi Sylvain, // // // On 08/07/2014 09:29, Sylvain Bauza wrote: // // Le 08/07/2014 00:35, Joe Gordon a écrit : // // // On Jul 7, 2014 9:50 AM, Lisa lisa.zangrando at pd.infn.it http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev wrote: // // Hi all, // // during the last IRC meeting, for better understanding our proposal (i.e // the FairShareScheduler), you suggested us to provide (for the tomorrow // meeting) a document which fully describes our use cases. Such document is // attached to this e-mail. // Any comment and feedback is welcome. // // The attached document was very helpful, than you. // // It sounds like Amazon's concept of spot instances ( as a user facing // abstraction) would solve your use case in its entirety. I see spot // instances as the general solution to the question of how to keep a cloud at // full utilization. If so then perhaps we can refocus this discussion on the // best way for Openstack to support Amazon style spot instances. // // // // // Can't agree more. Thanks Lisa for your use-cases, really helpful for // understand your concerns which are really HPC-based. If we want to // translate what you call Type 3 in a non-HPC world where users could compete // for a resource, spot instances model is coming to me as a clear model. // // // our model is similar to the Amazon's spot instances model because both try // to maximize the resource utilization. The main difference is the mechanism // used for assigning resources to the users (the user's offer in terms of // money vs the user's share). They differ even on how they release the // allocated resources. In our model, the user, whenever requires the creation // of a Type 3 VM, she has to select one of the possible types of life time // (short = 4 hours, medium = 24 hours, long = 48 hours). When the time // expires, the VM is automatically released (if not explicitly released by // the user). // Instead, in Amazon, the spot instance is released whenever the spot price // rises. // // /I think you can adapt your model your use case to the spot instance model by allocating different groups 'money' instead of a pre-defined share. If one user tries to use more then there share they will run out of 'money.' Would that fully align the two models? Also why pre-define the different life times for type 3 instances? / // // // I can see that you mention Blazar in your paper, and I appreciate this. // Climate (because that's the former and better known name) has been kick-off // because of such a rationale that you mention : we need to define a contract // (call it SLA if you wish) in between the user and the platform. // And you probably missed it, because I was probably unclear when we // discussed, but the final goal for Climate is *not* to have a start_date and // an end_date, but just *provide a contract in between the user and the // platform* (see // https://wiki.openstack.org/wiki/Blazar#Lease_types_.28concepts.29 ) // // Defining spot instances in OpenStack is a running question, each time // discussed when we presented Climate (now Blazar) at the Summits : what is // Climate? Is it
Re: [openstack-dev] [nova][scheduler] Proposal: FairShareScheduler.
Le 08/07/2014 22:47, Joe Gordon a écrit : I'm not saying that Climate (now Blazar) would be the only project involved for managing spot instances. By looking at a draft a couple of months before, I thought that this scenario would possibly involve Climate for best-effort leases (see again the Lease concepts in the wiki above), but also the Nova scheduler (for accounting the lease requests) and probably Ceilometer (for the auditing and metering side). Blazar is now in a turn where we're missing contributors because we are a Stackforge project, so we work with a minimal bandwidth and we don't have time for implementing best-effort leases but maybe that's something we could discuss. If you're willing to contribute to an Openstack-style project, I'm personnally thinking Blazar is a good one because of its little complexity as of now. I think the current thinking around how to sport spot instances is somewhat backwards. We should first identify the user facing requirements (I.E. API changes) then identify the missing pieces needed to support that API, and lastly figure out where those missing pieces should live. I don't think we can say Blazer is the answer without fully understanding the problem. My bad, I think you misunderstood me. I just said that part of the problem can possibly land in Blazar because of Blazar already implements some concepts close to the spot instances model (what we call leases, ie. a contract or SLA expressed by a REST API), but I'm also thinking that it would require other components involved. +1 on defining API specs for spot instance model, that's what we raised during the last Gantt meeting. ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [nova][scheduler] Proposal: FairShareScheduler.
Le 08/07/2014 00:35, Joe Gordon a écrit : On Jul 7, 2014 9:50 AM, Lisa lisa.zangra...@pd.infn.it mailto:lisa.zangra...@pd.infn.it wrote: Hi all, during the last IRC meeting, for better understanding our proposal (i.e the FairShareScheduler), you suggested us to provide (for the tomorrow meeting) a document which fully describes our use cases. Such document is attached to this e-mail. Any comment and feedback is welcome. The attached document was very helpful, than you. It sounds like Amazon's concept of spot instances ( as a user facing abstraction) would solve your use case in its entirety. I see spot instances as the general solution to the question of how to keep a cloud at full utilization. If so then perhaps we can refocus this discussion on the best way for Openstack to support Amazon style spot instances. Can't agree more. Thanks Lisa for your use-cases, really helpful for understand your concerns which are really HPC-based. If we want to translate what you call Type 3 in a non-HPC world where users could compete for a resource, spot instances model is coming to me as a clear model. I can see that you mention Blazar in your paper, and I appreciate this. Climate (because that's the former and better known name) has been kick-off because of such a rationale that you mention : we need to define a contract (call it SLA if you wish) in between the user and the platform. And you probably missed it, because I was probably unclear when we discussed, but the final goal for Climate is *not* to have a start_date and an end_date, but just *provide a contract in between the user and the platform* (see https://wiki.openstack.org/wiki/Blazar#Lease_types_.28concepts.29 ) Defining spot instances in OpenStack is a running question, each time discussed when we presented Climate (now Blazar) at the Summits : what is Climate? Is it something planning to provide spot instances ? Can Climate provide spot instances ? I'm not saying that Climate (now Blazar) would be the only project involved for managing spot instances. By looking at a draft a couple of months before, I thought that this scenario would possibly involve Climate for best-effort leases (see again the Lease concepts in the wiki above), but also the Nova scheduler (for accounting the lease requests) and probably Ceilometer (for the auditing and metering side). Blazar is now in a turn where we're missing contributors because we are a Stackforge project, so we work with a minimal bandwidth and we don't have time for implementing best-effort leases but maybe that's something we could discuss. If you're willing to contribute to an Openstack-style project, I'm personnally thinking Blazar is a good one because of its little complexity as of now. Thanks, -Sylvain Thanks a lot. Cheers, Lisa ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org mailto:OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [nova][scheduler] Proposal: FairShareScheduler.
Hi Sylvain, On 08/07/2014 09:29, Sylvain Bauza wrote: Le 08/07/2014 00:35, Joe Gordon a écrit : On Jul 7, 2014 9:50 AM, Lisa lisa.zangra...@pd.infn.it mailto:lisa.zangra...@pd.infn.it wrote: Hi all, during the last IRC meeting, for better understanding our proposal (i.e the FairShareScheduler), you suggested us to provide (for the tomorrow meeting) a document which fully describes our use cases. Such document is attached to this e-mail. Any comment and feedback is welcome. The attached document was very helpful, than you. It sounds like Amazon's concept of spot instances ( as a user facing abstraction) would solve your use case in its entirety. I see spot instances as the general solution to the question of how to keep a cloud at full utilization. If so then perhaps we can refocus this discussion on the best way for Openstack to support Amazon style spot instances. Can't agree more. Thanks Lisa for your use-cases, really helpful for understand your concerns which are really HPC-based. If we want to translate what you call Type 3 in a non-HPC world where users could compete for a resource, spot instances model is coming to me as a clear model. our model is similar to the Amazon's spot instances model because both try to maximize the resource utilization. The main difference is the mechanism used for assigning resources to the users (the user's offer in terms of money vs the user's share). They differ even on how they release the allocated resources. In our model, the user, whenever requires the creation of a Type 3 VM, she has to select one of the possible types of life time (short = 4 hours, medium = 24 hours, long = 48 hours). When the time expires, the VM is automatically released (if not explicitly released by the user). Instead, in Amazon, the spot instance is released whenever the spot price rises. I can see that you mention Blazar in your paper, and I appreciate this. Climate (because that's the former and better known name) has been kick-off because of such a rationale that you mention : we need to define a contract (call it SLA if you wish) in between the user and the platform. And you probably missed it, because I was probably unclear when we discussed, but the final goal for Climate is *not* to have a start_date and an end_date, but just *provide a contract in between the user and the platform* (see https://wiki.openstack.org/wiki/Blazar#Lease_types_.28concepts.29 ) Defining spot instances in OpenStack is a running question, each time discussed when we presented Climate (now Blazar) at the Summits : what is Climate? Is it something planning to provide spot instances ? Can Climate provide spot instances ? I'm not saying that Climate (now Blazar) would be the only project involved for managing spot instances. By looking at a draft a couple of months before, I thought that this scenario would possibly involve Climate for best-effort leases (see again the Lease concepts in the wiki above), but also the Nova scheduler (for accounting the lease requests) and probably Ceilometer (for the auditing and metering side). Blazar is now in a turn where we're missing contributors because we are a Stackforge project, so we work with a minimal bandwidth and we don't have time for implementing best-effort leases but maybe that's something we could discuss. If you're willing to contribute to an Openstack-style project, I'm personnally thinking Blazar is a good one because of its little complexity as of now. Just few questions. We read your use cases and it seems you had some issues with the quota handling. How did you solved it? About the Blazar's architecture (https://wiki.openstack.org/w/images/c/cb/Climate_architecture.png): the resource plug-in interacts even with the nova-scheduler? Such scheduler has been (or will be) extended for supporting the Blazar's requests? Which relationship there is between nova-scheduler and Gantt? It would be nice to discuss with you in details. Thanks a lot for your feedback. Cheers, Lisa Thanks, -Sylvain Thanks a lot. Cheers, Lisa ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org mailto:OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [nova][scheduler] Proposal: FairShareScheduler.
Le 08/07/2014 13:18, Lisa a écrit : Hi Sylvain, On 08/07/2014 09:29, Sylvain Bauza wrote: Le 08/07/2014 00:35, Joe Gordon a écrit : On Jul 7, 2014 9:50 AM, Lisa lisa.zangra...@pd.infn.it mailto:lisa.zangra...@pd.infn.it wrote: Hi all, during the last IRC meeting, for better understanding our proposal (i.e the FairShareScheduler), you suggested us to provide (for the tomorrow meeting) a document which fully describes our use cases. Such document is attached to this e-mail. Any comment and feedback is welcome. The attached document was very helpful, than you. It sounds like Amazon's concept of spot instances ( as a user facing abstraction) would solve your use case in its entirety. I see spot instances as the general solution to the question of how to keep a cloud at full utilization. If so then perhaps we can refocus this discussion on the best way for Openstack to support Amazon style spot instances. Can't agree more. Thanks Lisa for your use-cases, really helpful for understand your concerns which are really HPC-based. If we want to translate what you call Type 3 in a non-HPC world where users could compete for a resource, spot instances model is coming to me as a clear model. our model is similar to the Amazon's spot instances model because both try to maximize the resource utilization. The main difference is the mechanism used for assigning resources to the users (the user's offer in terms of money vs the user's share). They differ even on how they release the allocated resources. In our model, the user, whenever requires the creation of a Type 3 VM, she has to select one of the possible types of life time (short = 4 hours, medium = 24 hours, long = 48 hours). When the time expires, the VM is automatically released (if not explicitly released by the user). Instead, in Amazon, the spot instance is released whenever the spot price rises. That's just another trigger so the model is still good for defining what you say Type 3 :-) I can see that you mention Blazar in your paper, and I appreciate this. Climate (because that's the former and better known name) has been kick-off because of such a rationale that you mention : we need to define a contract (call it SLA if you wish) in between the user and the platform. And you probably missed it, because I was probably unclear when we discussed, but the final goal for Climate is *not* to have a start_date and an end_date, but just *provide a contract in between the user and the platform* (see https://wiki.openstack.org/wiki/Blazar#Lease_types_.28concepts.29 ) Defining spot instances in OpenStack is a running question, each time discussed when we presented Climate (now Blazar) at the Summits : what is Climate? Is it something planning to provide spot instances ? Can Climate provide spot instances ? I'm not saying that Climate (now Blazar) would be the only project involved for managing spot instances. By looking at a draft a couple of months before, I thought that this scenario would possibly involve Climate for best-effort leases (see again the Lease concepts in the wiki above), but also the Nova scheduler (for accounting the lease requests) and probably Ceilometer (for the auditing and metering side). Blazar is now in a turn where we're missing contributors because we are a Stackforge project, so we work with a minimal bandwidth and we don't have time for implementing best-effort leases but maybe that's something we could discuss. If you're willing to contribute to an Openstack-style project, I'm personnally thinking Blazar is a good one because of its little complexity as of now. Just few questions. We read your use cases and it seems you had some issues with the quota handling. How did you solved it? About the Blazar's architecture (https://wiki.openstack.org/w/images/c/cb/Climate_architecture.png): the resource plug-in interacts even with the nova-scheduler? Such scheduler has been (or will be) extended for supporting the Blazar's requests? Which relationship there is between nova-scheduler and Gantt? It would be nice to discuss with you in details. Thanks a lot for your feedback. Cheers, Lisa As said above, there are still some identified lacks in Blazar, but we miss resources for implementing these. Quotas is one of them, but some people in Yahoo! expressed their interest in Climate for implementing deferred quotas, so it could be done in the next cycle. As nova-scheduler is not enduser-facing (no API), we're driving a call to nova-api for placing resources thanks to aggregates. That's also why we're looking at Gantt, because it would be less tricky for doing this. -Sylvain Thanks, -Sylvain Thanks a lot. Cheers, Lisa ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org mailto:OpenStack-dev@lists.openstack.org
Re: [openstack-dev] [nova][scheduler] Proposal: FairShareScheduler.
On Tue, Jul 8, 2014 at 4:18 AM, Lisa lisa.zangra...@pd.infn.it wrote: Hi Sylvain, On 08/07/2014 09:29, Sylvain Bauza wrote: Le 08/07/2014 00:35, Joe Gordon a écrit : On Jul 7, 2014 9:50 AM, Lisa lisa.zangra...@pd.infn.it wrote: Hi all, during the last IRC meeting, for better understanding our proposal (i.e the FairShareScheduler), you suggested us to provide (for the tomorrow meeting) a document which fully describes our use cases. Such document is attached to this e-mail. Any comment and feedback is welcome. The attached document was very helpful, than you. It sounds like Amazon's concept of spot instances ( as a user facing abstraction) would solve your use case in its entirety. I see spot instances as the general solution to the question of how to keep a cloud at full utilization. If so then perhaps we can refocus this discussion on the best way for Openstack to support Amazon style spot instances. Can't agree more. Thanks Lisa for your use-cases, really helpful for understand your concerns which are really HPC-based. If we want to translate what you call Type 3 in a non-HPC world where users could compete for a resource, spot instances model is coming to me as a clear model. our model is similar to the Amazon's spot instances model because both try to maximize the resource utilization. The main difference is the mechanism used for assigning resources to the users (the user's offer in terms of money vs the user's share). They differ even on how they release the allocated resources. In our model, the user, whenever requires the creation of a Type 3 VM, she has to select one of the possible types of life time (short = 4 hours, medium = 24 hours, long = 48 hours). When the time expires, the VM is automatically released (if not explicitly released by the user). Instead, in Amazon, the spot instance is released whenever the spot price rises. I think you can adapt your model your use case to the spot instance model by allocating different groups 'money' instead of a pre-defined share. If one user tries to use more then there share they will run out of 'money.' Would that fully align the two models? Also why pre-define the different life times for type 3 instances? I can see that you mention Blazar in your paper, and I appreciate this. Climate (because that's the former and better known name) has been kick-off because of such a rationale that you mention : we need to define a contract (call it SLA if you wish) in between the user and the platform. And you probably missed it, because I was probably unclear when we discussed, but the final goal for Climate is *not* to have a start_date and an end_date, but just *provide a contract in between the user and the platform* (see https://wiki.openstack.org/wiki/Blazar#Lease_types_.28concepts.29 ) Defining spot instances in OpenStack is a running question, each time discussed when we presented Climate (now Blazar) at the Summits : what is Climate? Is it something planning to provide spot instances ? Can Climate provide spot instances ? I'm not saying that Climate (now Blazar) would be the only project involved for managing spot instances. By looking at a draft a couple of months before, I thought that this scenario would possibly involve Climate for best-effort leases (see again the Lease concepts in the wiki above), but also the Nova scheduler (for accounting the lease requests) and probably Ceilometer (for the auditing and metering side). Blazar is now in a turn where we're missing contributors because we are a Stackforge project, so we work with a minimal bandwidth and we don't have time for implementing best-effort leases but maybe that's something we could discuss. If you're willing to contribute to an Openstack-style project, I'm personnally thinking Blazar is a good one because of its little complexity as of now. Just few questions. We read your use cases and it seems you had some issues with the quota handling. How did you solved it? About the Blazar's architecture ( https://wiki.openstack.org/w/images/c/cb/Climate_architecture.png): the resource plug-in interacts even with the nova-scheduler? Such scheduler has been (or will be) extended for supporting the Blazar's requests? Which relationship there is between nova-scheduler and Gantt? It would be nice to discuss with you in details. Thanks a lot for your feedback. Cheers, Lisa Thanks, -Sylvain Thanks a lot. Cheers, Lisa ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing listOpenStack-dev@lists.openstack.orghttp://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing
Re: [openstack-dev] [nova][scheduler] Proposal: FairShareScheduler.
On Tue, Jul 8, 2014 at 6:55 AM, Sylvain Bauza sba...@redhat.com wrote: Le 08/07/2014 13:18, Lisa a écrit : Hi Sylvain, On 08/07/2014 09:29, Sylvain Bauza wrote: Le 08/07/2014 00:35, Joe Gordon a écrit : On Jul 7, 2014 9:50 AM, Lisa lisa.zangra...@pd.infn.it wrote: Hi all, during the last IRC meeting, for better understanding our proposal (i.e the FairShareScheduler), you suggested us to provide (for the tomorrow meeting) a document which fully describes our use cases. Such document is attached to this e-mail. Any comment and feedback is welcome. The attached document was very helpful, than you. It sounds like Amazon's concept of spot instances ( as a user facing abstraction) would solve your use case in its entirety. I see spot instances as the general solution to the question of how to keep a cloud at full utilization. If so then perhaps we can refocus this discussion on the best way for Openstack to support Amazon style spot instances. Can't agree more. Thanks Lisa for your use-cases, really helpful for understand your concerns which are really HPC-based. If we want to translate what you call Type 3 in a non-HPC world where users could compete for a resource, spot instances model is coming to me as a clear model. our model is similar to the Amazon's spot instances model because both try to maximize the resource utilization. The main difference is the mechanism used for assigning resources to the users (the user's offer in terms of money vs the user's share). They differ even on how they release the allocated resources. In our model, the user, whenever requires the creation of a Type 3 VM, she has to select one of the possible types of life time (short = 4 hours, medium = 24 hours, long = 48 hours). When the time expires, the VM is automatically released (if not explicitly released by the user). Instead, in Amazon, the spot instance is released whenever the spot price rises. That's just another trigger so the model is still good for defining what you say Type 3 :-) I can see that you mention Blazar in your paper, and I appreciate this. Climate (because that's the former and better known name) has been kick-off because of such a rationale that you mention : we need to define a contract (call it SLA if you wish) in between the user and the platform. And you probably missed it, because I was probably unclear when we discussed, but the final goal for Climate is *not* to have a start_date and an end_date, but just *provide a contract in between the user and the platform* (see https://wiki.openstack.org/wiki/Blazar#Lease_types_.28concepts.29 ) Defining spot instances in OpenStack is a running question, each time discussed when we presented Climate (now Blazar) at the Summits : what is Climate? Is it something planning to provide spot instances ? Can Climate provide spot instances ? I'm not saying that Climate (now Blazar) would be the only project involved for managing spot instances. By looking at a draft a couple of months before, I thought that this scenario would possibly involve Climate for best-effort leases (see again the Lease concepts in the wiki above), but also the Nova scheduler (for accounting the lease requests) and probably Ceilometer (for the auditing and metering side). Blazar is now in a turn where we're missing contributors because we are a Stackforge project, so we work with a minimal bandwidth and we don't have time for implementing best-effort leases but maybe that's something we could discuss. If you're willing to contribute to an Openstack-style project, I'm personnally thinking Blazar is a good one because of its little complexity as of now. I think the current thinking around how to sport spot instances is somewhat backwards. We should first identify the user facing requirements (I.E. API changes) then identify the missing pieces needed to support that API, and lastly figure out where those missing pieces should live. I don't think we can say Blazer is the answer without fully understanding the problem. Just few questions. We read your use cases and it seems you had some issues with the quota handling. How did you solved it? About the Blazar's architecture ( https://wiki.openstack.org/w/images/c/cb/Climate_architecture.png): the resource plug-in interacts even with the nova-scheduler? Such scheduler has been (or will be) extended for supporting the Blazar's requests? Which relationship there is between nova-scheduler and Gantt? It would be nice to discuss with you in details. Thanks a lot for your feedback. Cheers, Lisa As said above, there are still some identified lacks in Blazar, but we miss resources for implementing these. Quotas is one of them, but some people in Yahoo! expressed their interest in Climate for implementing deferred quotas, so it could be done in the next cycle. As nova-scheduler is not enduser-facing (no API), we're driving a call to nova-api
Re: [openstack-dev] [nova][scheduler] Proposal: FairShareScheduler.
On Jul 7, 2014 9:50 AM, Lisa lisa.zangra...@pd.infn.it wrote: Hi all, during the last IRC meeting, for better understanding our proposal (i.e the FairShareScheduler), you suggested us to provide (for the tomorrow meeting) a document which fully describes our use cases. Such document is attached to this e-mail. Any comment and feedback is welcome. The attached document was very helpful, than you. It sounds like Amazon's concept of spot instances ( as a user facing abstraction) would solve your use case in its entirety. I see spot instances as the general solution to the question of how to keep a cloud at full utilization. If so then perhaps we can refocus this discussion on the best way for Openstack to support Amazon style spot instances. Thanks a lot. Cheers, Lisa ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [nova][scheduler] Proposal: FairShareScheduler.
Eric,
Thanks for sharing your work, it looks like an interesting development.
I was wondering how the Keystone token expiry is handled since the tokens
generally have a 1 day validity. If the request is scheduling for more than one
day, it would no longer have a valid token. We have similar scenarios with
Kerberos/AFS credentials in the CERN batch system. There are some interesting
proxy renew approaches used by Heat to get tokens at a later date which may be
useful for this problem.
$ nova credentials
+---+-+
| Token | Value |
+---+-+
| expires | 2014-07-02T06:39:59Z|
| id| 1a819279121f4235a8d85c694dea5e9e|
| issued_at | 2014-07-01T06:39:59.385417 |
| tenant| {id: 841615a3-ece9-4622-9fa0-fdc178ed34f8, enabled: true, |
| | description: Personal Project for user timbell, name: |
| | Personal timbell} |
+---+-+
Tim
-Original Message-
From: Eric Frizziero [mailto:eric.frizzi...@pd.infn.it]
Sent: 30 June 2014 16:05
To: OpenStack Development Mailing List (not for usage questions)
Subject: [openstack-dev] [nova][scheduler] Proposal: FairShareScheduler.
Hi All,
we have analyzed the nova-scheduler component (FilterScheduler) in our
Openstack installation used by some scientific teams.
In our scenario, the cloud resources need to be distributed among the teams by
considering the predefined share (e.g. quota) assigned to each team, the
portion
of the resources currently used and the resources they have already consumed.
We have observed that:
1) User requests are sequentially processed (FIFO scheduling), i.e.
FilterScheduler doesn't provide any dynamic priority algorithm;
2) User requests that cannot be satisfied (e.g. if resources are not
available) fail and will be lost, i.e. on that scenario nova-scheduler doesn't
provide any queuing of the requests;
3) OpenStack simply provides a static partitioning of resources among various
projects / teams (use of quotas). If project/team 1 in a period is
systematically
underutilizing its quota and the project/team 2 instead is systematically
saturating its quota, the only solution to give more resource to project/team
2 is
a manual change (to be done by the admin) to the related quotas.
The need to find a better approach to enable a more effective scheduling in
Openstack becomes more and more evident when the number of the user
requests to be handled increases significantly. This is a well known problem
which has already been solved in the past for the Batch Systems.
In order to solve those issues in our usage scenario of Openstack, we have
developed a prototype of a pluggable scheduler, named FairShareScheduler,
with the objective to extend the existing OpenStack scheduler
(FilterScheduler)
by integrating a (batch like) dynamic priority algorithm.
The architecture of the FairShareScheduler is explicitly designed to provide a
high scalability level. To all user requests will be assigned a priority value
calculated by considering the share allocated to the user by the administrator
and the evaluation of the effective resource usage consumed in the recent
past.
All requests will be inserted in a priority queue, and processed in parallel
by a
configurable pool of workers without interfering with the priority order.
Moreover all significant information (e.g. priority queue) will be stored in a
persistence layer in order to provide a fault tolerance mechanism while a
proper
logging system will annotate all relevant events, useful for auditing
processing.
In more detail, some features of the FairshareScheduler are:
a) It assigns dynamically the proper priority to every new user requests;
b) The priority of the queued requests will be recalculated periodically
using the
fairshare algorithm. This feature guarantees the usage of the cloud resources
is
distributed among users and groups by considering the portion of the cloud
resources allocated to them (i.e. share) and the resources already consumed;
c) all user requests will be inserted in a (persistent) priority queue and
then
processed asynchronously by the dedicated process (filtering + weighting
phase)
when compute resources are available;
d) From the client point of view the queued requests remain in Scheduling
state till the compute resources are available. No new states added: this
prevents any possible interaction issue with the Openstack clients;
e) User requests are dequeued by a pool of WorkerThreads (configurable), i.e.
no sequential processing of
Re: [openstack-dev] [nova][scheduler] Proposal: FairShareScheduler.
On 06/30/2014 04:35 PM, Jesse Pretorius wrote: On 30 June 2014 16:05, Eric Frizziero eric.frizzi...@pd.infn.it mailto:eric.frizzi...@pd.infn.it wrote: In more detail, some features of the FairshareScheduler are: a) It assigns dynamically the proper priority to every new user requests; b) The priority of the queued requests will be recalculated periodically using the fairshare algorithm. This feature guarantees the usage of the cloud resources is distributed among users and groups by considering the portion of the cloud resources allocated to them (i.e. share) and the resources already consumed; c) all user requests will be inserted in a (persistent) priority queue and then processed asynchronously by the dedicated process (filtering + weighting phase) when compute resources are available; d) From the client point of view the queued requests remain in Scheduling state till the compute resources are available. No new states added: this prevents any possible interaction issue with the Openstack clients; e) User requests are dequeued by a pool of WorkerThreads (configurable), i.e. no sequential processing of the requests; f) The failed requests at filtering + weighting phase may be inserted again in the queue for n-times (configurable). We have integrated the FairShareScheduler in our Openstack installation (release HAVANA). We're now working to adapt the FairShareScheduler to the new release IceHouse. Does anyone have experiences in those issues found in our cloud scenario? Could the FairShareScheduler be useful for the Openstack community? In that case, we'll be happy to share our work. Sounds like an interesting option to have available. Thank you very much for your feedback! Cheers, Eric. ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [nova][scheduler] Proposal: FairShareScheduler.
Hi Belmiro, On 06/30/2014 11:42 PM, Belmiro Moreira wrote: Hi Eric, definitely... In my view a FairShareScheduler could be a very interesting option for private clouds that support scientific communities. Basically this is the model used by batch systems in order to fully use the available resources. Yes, it is. I'm very curious about the work that you are doing. You can find more info at the following link: https://agenda.infn.it/getFile.py/access?contribId=17sessionId=3resId=0materialId=slidesconfId=7915 Is it available in github? At slide 18, you can find the pointer to the FairShareScheduler. Cheers, Eric. Belmiro -- Belmiro Moreira CERN Email: belmiro.more...@cern.ch mailto:belmiro.more...@cern.ch IRC: belmoreira On Mon, Jun 30, 2014 at 4:05 PM, Eric Frizziero eric.frizzi...@pd.infn.it mailto:eric.frizzi...@pd.infn.it wrote: Hi All, we have analyzed the nova-scheduler component (FilterScheduler) in our Openstack installation used by some scientific teams. In our scenario, the cloud resources need to be distributed among the teams by considering the predefined share (e.g. quota) assigned to each team, the portion of the resources currently used and the resources they have already consumed. We have observed that: 1) User requests are sequentially processed (FIFO scheduling), i.e. FilterScheduler doesn't provide any dynamic priority algorithm; 2) User requests that cannot be satisfied (e.g. if resources are not available) fail and will be lost, i.e. on that scenario nova-scheduler doesn't provide any queuing of the requests; 3) OpenStack simply provides a static partitioning of resources among various projects / teams (use of quotas). If project/team 1 in a period is systematically underutilizing its quota and the project/team 2 instead is systematically saturating its quota, the only solution to give more resource to project/team 2 is a manual change (to be done by the admin) to the related quotas. The need to find a better approach to enable a more effective scheduling in Openstack becomes more and more evident when the number of the user requests to be handled increases significantly. This is a well known problem which has already been solved in the past for the Batch Systems. In order to solve those issues in our usage scenario of Openstack, we have developed a prototype of a pluggable scheduler, named FairShareScheduler, with the objective to extend the existing OpenStack scheduler (FilterScheduler) by integrating a (batch like) dynamic priority algorithm. The architecture of the FairShareScheduler is explicitly designed to provide a high scalability level. To all user requests will be assigned a priority value calculated by considering the share allocated to the user by the administrator and the evaluation of the effective resource usage consumed in the recent past. All requests will be inserted in a priority queue, and processed in parallel by a configurable pool of workers without interfering with the priority order. Moreover all significant information (e.g. priority queue) will be stored in a persistence layer in order to provide a fault tolerance mechanism while a proper logging system will annotate all relevant events, useful for auditing processing. In more detail, some features of the FairshareScheduler are: a) It assigns dynamically the proper priority to every new user requests; b) The priority of the queued requests will be recalculated periodically using the fairshare algorithm. This feature guarantees the usage of the cloud resources is distributed among users and groups by considering the portion of the cloud resources allocated to them (i.e. share) and the resources already consumed; c) all user requests will be inserted in a (persistent) priority queue and then processed asynchronously by the dedicated process (filtering + weighting phase) when compute resources are available; d) From the client point of view the queued requests remain in Scheduling state till the compute resources are available. No new states added: this prevents any possible interaction issue with the Openstack clients; e) User requests are dequeued by a pool of WorkerThreads (configurable), i.e. no sequential processing of the requests; f) The failed requests at filtering + weighting phase may be inserted again in the queue for n-times (configurable). We have integrated the FairShareScheduler in our Openstack installation (release HAVANA). We're now working to adapt the FairShareScheduler to the new release IceHouse. Does anyone have experiences in those issues found in our cloud scenario? Could the FairShareScheduler be useful for the Openstack community? In that case, we'll
Re: [openstack-dev] [nova][scheduler] Proposal: FairShareScheduler.
Hi Tim,
for sure this is one of the main issues we are facing and the approach
you suggested is the same we are investigating on.
Could you provide some details about the Heat proxy renew mechanism?
Thank you very much for your feedback.
Cheers,
Lisa
On 01/07/2014 08:46, Tim Bell wrote:
Eric,
Thanks for sharing your work, it looks like an interesting development.
I was wondering how the Keystone token expiry is handled since the tokens
generally have a 1 day validity. If the request is scheduling for more than one
day, it would no longer have a valid token. We have similar scenarios with
Kerberos/AFS credentials in the CERN batch system. There are some interesting
proxy renew approaches used by Heat to get tokens at a later date which may be
useful for this problem.
$ nova credentials
+---+-+
| Token | Value |
+---+-+
| expires | 2014-07-02T06:39:59Z|
| id| 1a819279121f4235a8d85c694dea5e9e|
| issued_at | 2014-07-01T06:39:59.385417 |
| tenant| {id: 841615a3-ece9-4622-9fa0-fdc178ed34f8, enabled: true, |
| | description: Personal Project for user timbell, name: |
| | Personal timbell} |
+---+-+
Tim
-Original Message-
From: Eric Frizziero [mailto:eric.frizzi...@pd.infn.it]
Sent: 30 June 2014 16:05
To: OpenStack Development Mailing List (not for usage questions)
Subject: [openstack-dev] [nova][scheduler] Proposal: FairShareScheduler.
Hi All,
we have analyzed the nova-scheduler component (FilterScheduler) in our
Openstack installation used by some scientific teams.
In our scenario, the cloud resources need to be distributed among the teams by
considering the predefined share (e.g. quota) assigned to each team, the portion
of the resources currently used and the resources they have already consumed.
We have observed that:
1) User requests are sequentially processed (FIFO scheduling), i.e.
FilterScheduler doesn't provide any dynamic priority algorithm;
2) User requests that cannot be satisfied (e.g. if resources are not
available) fail and will be lost, i.e. on that scenario nova-scheduler doesn't
provide any queuing of the requests;
3) OpenStack simply provides a static partitioning of resources among various
projects / teams (use of quotas). If project/team 1 in a period is
systematically
underutilizing its quota and the project/team 2 instead is systematically
saturating its quota, the only solution to give more resource to project/team 2
is
a manual change (to be done by the admin) to the related quotas.
The need to find a better approach to enable a more effective scheduling in
Openstack becomes more and more evident when the number of the user
requests to be handled increases significantly. This is a well known problem
which has already been solved in the past for the Batch Systems.
In order to solve those issues in our usage scenario of Openstack, we have
developed a prototype of a pluggable scheduler, named FairShareScheduler,
with the objective to extend the existing OpenStack scheduler (FilterScheduler)
by integrating a (batch like) dynamic priority algorithm.
The architecture of the FairShareScheduler is explicitly designed to provide a
high scalability level. To all user requests will be assigned a priority value
calculated by considering the share allocated to the user by the administrator
and the evaluation of the effective resource usage consumed in the recent past.
All requests will be inserted in a priority queue, and processed in parallel by
a
configurable pool of workers without interfering with the priority order.
Moreover all significant information (e.g. priority queue) will be stored in a
persistence layer in order to provide a fault tolerance mechanism while a proper
logging system will annotate all relevant events, useful for auditing
processing.
In more detail, some features of the FairshareScheduler are:
a) It assigns dynamically the proper priority to every new user requests;
b) The priority of the queued requests will be recalculated periodically using
the
fairshare algorithm. This feature guarantees the usage of the cloud resources is
distributed among users and groups by considering the portion of the cloud
resources allocated to them (i.e. share) and the resources already consumed;
c) all user requests will be inserted in a (persistent) priority queue and then
processed asynchronously by the dedicated process (filtering + weighting phase)
when compute resources are available;
d) From the client point of view the queued requests remain in Scheduling
state till the compute resources
Re: [openstack-dev] [nova][scheduler] Proposal: FairShareScheduler.
Hi Don, I and my colleague Lisa attend both IRC meetings (today and next week). In the meantime you can find more info about the FairShareScheduler at the following link: https://agenda.infn.it/getFile.py/access?contribId=17sessionId=3resId=0materialId=slidesconfId=7915 Thank you very much! Cheers, Eric. On 07/01/2014 01:48 AM, Dugger, Donald D wrote: Eric- We have a weekly scheduler sub-group (code name gantt) IRC meeting at 1500 UTC on Tuesdays. This would be an excellent topic to bring up at one of those meetings as a lot of people with interest in the scheduler will be there. It's a little short notice for tomorrow but do you think you could attend next week, 7/8, to talk about this? -- Don Dugger Censeo Toto nos in Kansa esse decisse. - D. Gale Ph: 303/443-3786 -Original Message- From: Eric Frizziero [mailto:eric.frizzi...@pd.infn.it] Sent: Monday, June 30, 2014 8:05 AM To: OpenStack Development Mailing List (not for usage questions) Subject: [openstack-dev] [nova][scheduler] Proposal: FairShareScheduler. Hi All, we have analyzed the nova-scheduler component (FilterScheduler) in our Openstack installation used by some scientific teams. In our scenario, the cloud resources need to be distributed among the teams by considering the predefined share (e.g. quota) assigned to each team, the portion of the resources currently used and the resources they have already consumed. We have observed that: 1) User requests are sequentially processed (FIFO scheduling), i.e. FilterScheduler doesn't provide any dynamic priority algorithm; 2) User requests that cannot be satisfied (e.g. if resources are not available) fail and will be lost, i.e. on that scenario nova-scheduler doesn't provide any queuing of the requests; 3) OpenStack simply provides a static partitioning of resources among various projects / teams (use of quotas). If project/team 1 in a period is systematically underutilizing its quota and the project/team 2 instead is systematically saturating its quota, the only solution to give more resource to project/team 2 is a manual change (to be done by the admin) to the related quotas. The need to find a better approach to enable a more effective scheduling in Openstack becomes more and more evident when the number of the user requests to be handled increases significantly. This is a well known problem which has already been solved in the past for the Batch Systems. In order to solve those issues in our usage scenario of Openstack, we have developed a prototype of a pluggable scheduler, named FairShareScheduler, with the objective to extend the existing OpenStack scheduler (FilterScheduler) by integrating a (batch like) dynamic priority algorithm. The architecture of the FairShareScheduler is explicitly designed to provide a high scalability level. To all user requests will be assigned a priority value calculated by considering the share allocated to the user by the administrator and the evaluation of the effective resource usage consumed in the recent past. All requests will be inserted in a priority queue, and processed in parallel by a configurable pool of workers without interfering with the priority order. Moreover all significant information (e.g. priority queue) will be stored in a persistence layer in order to provide a fault tolerance mechanism while a proper logging system will annotate all relevant events, useful for auditing processing. In more detail, some features of the FairshareScheduler are: a) It assigns dynamically the proper priority to every new user requests; b) The priority of the queued requests will be recalculated periodically using the fairshare algorithm. This feature guarantees the usage of the cloud resources is distributed among users and groups by considering the portion of the cloud resources allocated to them (i.e. share) and the resources already consumed; c) all user requests will be inserted in a (persistent) priority queue and then processed asynchronously by the dedicated process (filtering + weighting phase) when compute resources are available; d) From the client point of view the queued requests remain in Scheduling state till the compute resources are available. No new states added: this prevents any possible interaction issue with the Openstack clients; e) User requests are dequeued by a pool of WorkerThreads (configurable), i.e. no sequential processing of the requests; f) The failed requests at filtering + weighting phase may be inserted again in the queue for n-times (configurable). We have integrated the FairShareScheduler in our Openstack installation (release HAVANA). We're now working to adapt the FairShareScheduler to the new release IceHouse. Does anyone have experiences in those issues found in our cloud scenario? Could the FairShareScheduler be useful for the Openstack community? In that case, we'll be happy to share our work. Any feedback/comment is welcome! Cheers, Eric.
Re: [openstack-dev] [nova][scheduler] Proposal: FairShareScheduler.
Le 01/07/2014 10:45, Lisa a écrit :
Hi Tim,
for sure this is one of the main issues we are facing and the approach
you suggested is the same we are investigating on.
Could you provide some details about the Heat proxy renew mechanism?
Thank you very much for your feedback.
Cheers,
Lisa
Keystone already provides support for that problem, that's called trusts.
https://wiki.openstack.org/wiki/Keystone/Trusts
I would be also interested in reviewing your implementation, because
Blazar [1] is also providing some reservation mechanism based on trusts.
One long-term objective of splitting the Scheduler into a separate
project (called Gantt) would be to interoperate Blazar and Gantt for
scheduling non-immediate requests, so your project is going into the
right direction.
As Don said, feel free to join us today at 3pm UTC to discuss about your
proposal.
[1] https://wiki.openstack.org/wiki/Blazar
On 01/07/2014 08:46, Tim Bell wrote:
Eric,
Thanks for sharing your work, it looks like an interesting development.
I was wondering how the Keystone token expiry is handled since the
tokens generally have a 1 day validity. If the request is scheduling
for more than one day, it would no longer have a valid token. We have
similar scenarios with Kerberos/AFS credentials in the CERN batch
system. There are some interesting proxy renew approaches used by
Heat to get tokens at a later date which may be useful for this problem.
$ nova credentials
+---+-+
| Token |
Value |
+---+-+
| expires |
2014-07-02T06:39:59Z|
| id|
1a819279121f4235a8d85c694dea5e9e|
| issued_at |
2014-07-01T06:39:59.385417 |
| tenant| {id: 841615a3-ece9-4622-9fa0-fdc178ed34f8,
enabled: true, |
| | description: Personal Project for user timbell,
name: |
| | Personal
timbell} |
+---+-+
Tim
-Original Message-
From: Eric Frizziero [mailto:eric.frizzi...@pd.infn.it]
Sent: 30 June 2014 16:05
To: OpenStack Development Mailing List (not for usage questions)
Subject: [openstack-dev] [nova][scheduler] Proposal:
FairShareScheduler.
Hi All,
we have analyzed the nova-scheduler component (FilterScheduler) in our
Openstack installation used by some scientific teams.
In our scenario, the cloud resources need to be distributed among
the teams by
considering the predefined share (e.g. quota) assigned to each team,
the portion
of the resources currently used and the resources they have already
consumed.
We have observed that:
1) User requests are sequentially processed (FIFO scheduling), i.e.
FilterScheduler doesn't provide any dynamic priority algorithm;
2) User requests that cannot be satisfied (e.g. if resources are not
available) fail and will be lost, i.e. on that scenario
nova-scheduler doesn't
provide any queuing of the requests;
3) OpenStack simply provides a static partitioning of resources
among various
projects / teams (use of quotas). If project/team 1 in a period is
systematically
underutilizing its quota and the project/team 2 instead is
systematically
saturating its quota, the only solution to give more resource to
project/team 2 is
a manual change (to be done by the admin) to the related quotas.
The need to find a better approach to enable a more effective
scheduling in
Openstack becomes more and more evident when the number of the user
requests to be handled increases significantly. This is a well known
problem
which has already been solved in the past for the Batch Systems.
In order to solve those issues in our usage scenario of Openstack,
we have
developed a prototype of a pluggable scheduler, named
FairShareScheduler,
with the objective to extend the existing OpenStack scheduler
(FilterScheduler)
by integrating a (batch like) dynamic priority algorithm.
The architecture of the FairShareScheduler is explicitly designed to
provide a
high scalability level. To all user requests will be assigned a
priority value
calculated by considering the share allocated to the user by the
administrator
and the evaluation of the effective resource usage consumed in the
recent past.
All requests will be inserted in a priority queue, and processed in
parallel by a
configurable pool of workers without interfering with the priority
order.
Moreover all significant information (e.g. priority queue) will be
stored in a
persistence layer in order to provide a fault tolerance mechanism
while a proper
logging system will annotate all relevant events, useful for
auditing processing.
In more
Re: [openstack-dev] [nova][scheduler] Proposal: FairShareScheduler.
On 30 June 2014 16:05, Eric Frizziero eric.frizzi...@pd.infn.it wrote: In more detail, some features of the FairshareScheduler are: a) It assigns dynamically the proper priority to every new user requests; b) The priority of the queued requests will be recalculated periodically using the fairshare algorithm. This feature guarantees the usage of the cloud resources is distributed among users and groups by considering the portion of the cloud resources allocated to them (i.e. share) and the resources already consumed; c) all user requests will be inserted in a (persistent) priority queue and then processed asynchronously by the dedicated process (filtering + weighting phase) when compute resources are available; d) From the client point of view the queued requests remain in “Scheduling” state till the compute resources are available. No new states added: this prevents any possible interaction issue with the Openstack clients; e) User requests are dequeued by a pool of WorkerThreads (configurable), i.e. no sequential processing of the requests; f) The failed requests at filtering + weighting phase may be inserted again in the queue for n-times (configurable). We have integrated the FairShareScheduler in our Openstack installation (release HAVANA). We're now working to adapt the FairShareScheduler to the new release IceHouse. Does anyone have experiences in those issues found in our cloud scenario? Could the FairShareScheduler be useful for the Openstack community? In that case, we'll be happy to share our work. Sounds like an interesting option to have available. ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [nova][scheduler] Proposal: FairShareScheduler.
Hi Eric, definitely... In my view a FairShareScheduler could be a very interesting option for private clouds that support scientific communities. Basically this is the model used by batch systems in order to fully use the available resources. I'm very curious about the work that you are doing. Is it available in github? Belmiro -- Belmiro Moreira CERN Email: belmiro.more...@cern.ch IRC: belmoreira On Mon, Jun 30, 2014 at 4:05 PM, Eric Frizziero eric.frizzi...@pd.infn.it wrote: Hi All, we have analyzed the nova-scheduler component (FilterScheduler) in our Openstack installation used by some scientific teams. In our scenario, the cloud resources need to be distributed among the teams by considering the predefined share (e.g. quota) assigned to each team, the portion of the resources currently used and the resources they have already consumed. We have observed that: 1) User requests are sequentially processed (FIFO scheduling), i.e. FilterScheduler doesn't provide any dynamic priority algorithm; 2) User requests that cannot be satisfied (e.g. if resources are not available) fail and will be lost, i.e. on that scenario nova-scheduler doesn't provide any queuing of the requests; 3) OpenStack simply provides a static partitioning of resources among various projects / teams (use of quotas). If project/team 1 in a period is systematically underutilizing its quota and the project/team 2 instead is systematically saturating its quota, the only solution to give more resource to project/team 2 is a manual change (to be done by the admin) to the related quotas. The need to find a better approach to enable a more effective scheduling in Openstack becomes more and more evident when the number of the user requests to be handled increases significantly. This is a well known problem which has already been solved in the past for the Batch Systems. In order to solve those issues in our usage scenario of Openstack, we have developed a prototype of a pluggable scheduler, named FairShareScheduler, with the objective to extend the existing OpenStack scheduler (FilterScheduler) by integrating a (batch like) dynamic priority algorithm. The architecture of the FairShareScheduler is explicitly designed to provide a high scalability level. To all user requests will be assigned a priority value calculated by considering the share allocated to the user by the administrator and the evaluation of the effective resource usage consumed in the recent past. All requests will be inserted in a priority queue, and processed in parallel by a configurable pool of workers without interfering with the priority order. Moreover all significant information (e.g. priority queue) will be stored in a persistence layer in order to provide a fault tolerance mechanism while a proper logging system will annotate all relevant events, useful for auditing processing. In more detail, some features of the FairshareScheduler are: a) It assigns dynamically the proper priority to every new user requests; b) The priority of the queued requests will be recalculated periodically using the fairshare algorithm. This feature guarantees the usage of the cloud resources is distributed among users and groups by considering the portion of the cloud resources allocated to them (i.e. share) and the resources already consumed; c) all user requests will be inserted in a (persistent) priority queue and then processed asynchronously by the dedicated process (filtering + weighting phase) when compute resources are available; d) From the client point of view the queued requests remain in “Scheduling” state till the compute resources are available. No new states added: this prevents any possible interaction issue with the Openstack clients; e) User requests are dequeued by a pool of WorkerThreads (configurable), i.e. no sequential processing of the requests; f) The failed requests at filtering + weighting phase may be inserted again in the queue for n-times (configurable). We have integrated the FairShareScheduler in our Openstack installation (release HAVANA). We're now working to adapt the FairShareScheduler to the new release IceHouse. Does anyone have experiences in those issues found in our cloud scenario? Could the FairShareScheduler be useful for the Openstack community? In that case, we'll be happy to share our work. Any feedback/comment is welcome! Cheers, Eric. ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
