Eric,
Thanks for sharing your work, it looks like an interesting development.
I was wondering how the Keystone token expiry is handled since the tokens
generally have a 1 day validity. If the request is scheduling for more than one
day, it would no longer have a valid token. We have similar scenarios with
Kerberos/AFS credentials in the CERN batch system. There are some interesting
proxy renew approaches used by Heat to get tokens at a later date which may be
useful for this problem.
$ nova credentials
+-----------+-----------------------------------------------------------------+
| Token | Value |
+-----------+-----------------------------------------------------------------+
| expires | 2014-07-02T06:39:59Z |
| id | 1a819279121f4235a8d85c694dea5e9e |
| issued_at | 2014-07-01T06:39:59.385417 |
| tenant | {"id": "841615a3-ece9-4622-9fa0-fdc178ed34f8", "enabled": true, |
| | "description": "Personal Project for user timbell", "name": |
| | "Personal timbell"} |
+-----------+-----------------------------------------------------------------+
Tim
> -----Original Message-----
> From: Eric Frizziero [mailto:eric.frizzi...@pd.infn.it]
> Sent: 30 June 2014 16:05
> To: OpenStack Development Mailing List (not for usage questions)
> Subject: [openstack-dev] [nova][scheduler] Proposal: FairShareScheduler.
>
> Hi All,
>
> we have analyzed the nova-scheduler component (FilterScheduler) in our
> Openstack installation used by some scientific teams.
>
> In our scenario, the cloud resources need to be distributed among the teams by
> considering the predefined share (e.g. quota) assigned to each team, the
> portion
> of the resources currently used and the resources they have already consumed.
>
> We have observed that:
> 1) User requests are sequentially processed (FIFO scheduling), i.e.
> FilterScheduler doesn't provide any dynamic priority algorithm;
> 2) User requests that cannot be satisfied (e.g. if resources are not
> available) fail and will be lost, i.e. on that scenario nova-scheduler doesn't
> provide any queuing of the requests;
> 3) OpenStack simply provides a static partitioning of resources among various
> projects / teams (use of quotas). If project/team 1 in a period is
> systematically
> underutilizing its quota and the project/team 2 instead is systematically
> saturating its quota, the only solution to give more resource to project/team
> 2 is
> a manual change (to be done by the admin) to the related quotas.
>
> The need to find a better approach to enable a more effective scheduling in
> Openstack becomes more and more evident when the number of the user
> requests to be handled increases significantly. This is a well known problem
> which has already been solved in the past for the Batch Systems.
>
> In order to solve those issues in our usage scenario of Openstack, we have
> developed a prototype of a pluggable scheduler, named FairShareScheduler,
> with the objective to extend the existing OpenStack scheduler
> (FilterScheduler)
> by integrating a (batch like) dynamic priority algorithm.
>
> The architecture of the FairShareScheduler is explicitly designed to provide a
> high scalability level. To all user requests will be assigned a priority value
> calculated by considering the share allocated to the user by the administrator
> and the evaluation of the effective resource usage consumed in the recent
> past.
> All requests will be inserted in a priority queue, and processed in parallel
> by a
> configurable pool of workers without interfering with the priority order.
> Moreover all significant information (e.g. priority queue) will be stored in a
> persistence layer in order to provide a fault tolerance mechanism while a
> proper
> logging system will annotate all relevant events, useful for auditing
> processing.
>
> In more detail, some features of the FairshareScheduler are:
> a) It assigns dynamically the proper priority to every new user requests;
> b) The priority of the queued requests will be recalculated periodically
> using the
> fairshare algorithm. This feature guarantees the usage of the cloud resources
> is
> distributed among users and groups by considering the portion of the cloud
> resources allocated to them (i.e. share) and the resources already consumed;
> c) all user requests will be inserted in a (persistent) priority queue and
> then
> processed asynchronously by the dedicated process (filtering + weighting
> phase)
> when compute resources are available;
> d) From the client point of view the queued requests remain in "Scheduling"
> state till the compute resources are available. No new states added: this
> prevents any possible interaction issue with the Openstack clients;
> e) User requests are dequeued by a pool of WorkerThreads (configurable), i.e.
> no sequential processing of the requests;
> f) The failed requests at filtering + weighting phase may be inserted again
> in the
> queue for n-times (configurable).
>
> We have integrated the FairShareScheduler in our Openstack installation
> (release "HAVANA"). We're now working to adapt the FairShareScheduler to the
> new release "IceHouse".
>
> Does anyone have experiences in those issues found in our cloud scenario?
>
> Could the FairShareScheduler be useful for the Openstack community?
> In that case, we'll be happy to share our work.
>
> Any feedback/comment is welcome!
>
> Cheers,
> Eric.
>
>
> _______________________________________________
> OpenStack-dev mailing list
> OpenStack-dev@lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
_______________________________________________
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
