Hello Mark,
First of all, thank you for the reply !
The design you briefed me, a very good way to control the allocations,
the insight will definitely help me address few issue's for this service.
I have derived some use-cases after discussing with Sudhakar, Are these
use-cases exhaustive enough to generalize them to CIPRES, Please review
and suggest improvements if any.
Use-cases can be found here :
https://cwiki.apache.org/confluence/display/AIRAVATA/Airavata+Resource+Allocation+Service
Yes Suresh,
I have created a page to document the development phase for this module,
link is mentioned above.
Thanks and best regards,
Anuj Bhandar
On 09/23/2016 03:38 PM, Miller, Mark wrote:
Hi Anuj,
Just to follow up on Suresh’s comment.
We do not use an allocation protocol, though Sudhakar does, and
probably can help with that.
We do enforce both storage and usage limits on users.
The former is done at login, and users over a certain threshold aren’t
allowed to submit jobs until their storage levels are reduced.
The latter is enforced, but manually in our main site (at least at
present).
However, our rest site has a better and more sophisticated method.
Our policy is to allow users to access 50,000 su if they are at a US
institution, and 30,000 su if they are at a foreign institution.
When the user hits 10,20, 30, and 50 thousand SU, they receive an
email to warn them about the process.
If they go above, I can turn off their account manually, but I usually
ask them how they are doing first.
In the REST site, each time a user submits a job, we calculate the
amount of total SUs it would consume if it ran to completion.
We subtract this amount from the amount of allocation the user has
remaining. If the potential cost is greater than the amount they have
remaining, the submission is rejected. That way, a user cannot submit
100 jobs each requiring 6800 SU. Which is possible, though it would
take a long time.
The idea of holding a reserve against potential expenses by a running
job is not a new idea, but it prevents things from getting out of hand.
Let me know if you have any questions about how this works.
Best,
Mark
*From:*Suresh Marru [mailto:[email protected]]
*Sent:* Friday, September 23, 2016 12:00 PM
*To:* Airavata Dev <[email protected]>
*Subject:* Re: Building a Resource Allocation Service For Airavata
Hi Anuj,
This is great summary. Since this is a new capability for Airavata, it
will be useful to have dedicated wiki space to record all information.
We can start with a high-level problem description (you have some of
that in the form of use cases), architecture and design thoughts and
Implementation plan (pointers to JIRA’s).
May be you can put it under -
https://cwiki.apache.org/confluence/display/AIRAVATA/Architecture+Documentation
Mark and Terri might have some input on this task since CIPRES
enforces soft quotas for users (but I guess there is no formal
allocation request process). Once you have the wiki we can gather the
feedback also there.
Thanks,
Suresh
On Sep 23, 2016, at 2:39 PM, Anuj Bhandar <[email protected]
<mailto:[email protected]>> wrote:
Hi Dev,
I have undertaken this project of designing and developing an
resource allocation service for Airavata, to give you some
context, this service takes inspiration from XRAS (XSEDE Resource
Allocation Service).
Here is the preliminary design document, which was written as a
part of GSOC-2016,
https://cwiki.apache.org/confluence/display/AIRAVATA/%5BGsoc+Proposal%5D+Resource+Allocation+Manager+for+Apache+Airavata.
I have derived some use cases by discussing with Sudhakar
Pamidighantam ( PI - GridChem), here is the link :
https://issues.apache.org/jira/browse/AIRAVATA-2108.
Any comments and improvements are gladly welcome.
Thanks and best regards,
Anuj Bhandar
