Hi Anuj, Just to follow up on Suresh’s comment. We do not use an allocation protocol, though Sudhakar does, and probably can help with that. We do enforce both storage and usage limits on users.
The former is done at login, and users over a certain threshold aren’t allowed to submit jobs until their storage levels are reduced. The latter is enforced, but manually in our main site (at least at present). However, our rest site has a better and more sophisticated method. Our policy is to allow users to access 50,000 su if they are at a US institution, and 30,000 su if they are at a foreign institution. When the user hits 10,20, 30, and 50 thousand SU, they receive an email to warn them about the process. If they go above, I can turn off their account manually, but I usually ask them how they are doing first. In the REST site, each time a user submits a job, we calculate the amount of total SUs it would consume if it ran to completion. We subtract this amount from the amount of allocation the user has remaining. If the potential cost is greater than the amount they have remaining, the submission is rejected. That way, a user cannot submit 100 jobs each requiring 6800 SU. Which is possible, though it would take a long time. The idea of holding a reserve against potential expenses by a running job is not a new idea, but it prevents things from getting out of hand. Let me know if you have any questions about how this works. Best, Mark From: Suresh Marru [mailto:[email protected]] Sent: Friday, September 23, 2016 12:00 PM To: Airavata Dev <[email protected]> Subject: Re: Building a Resource Allocation Service For Airavata Hi Anuj, This is great summary. Since this is a new capability for Airavata, it will be useful to have dedicated wiki space to record all information. We can start with a high-level problem description (you have some of that in the form of use cases), architecture and design thoughts and Implementation plan (pointers to JIRA’s). May be you can put it under - https://cwiki.apache.org/confluence/display/AIRAVATA/Architecture+Documentation Mark and Terri might have some input on this task since CIPRES enforces soft quotas for users (but I guess there is no formal allocation request process). Once you have the wiki we can gather the feedback also there. Thanks, Suresh On Sep 23, 2016, at 2:39 PM, Anuj Bhandar <[email protected]<mailto:[email protected]>> wrote: Hi Dev, I have undertaken this project of designing and developing an resource allocation service for Airavata, to give you some context, this service takes inspiration from XRAS (XSEDE Resource Allocation Service). Here is the preliminary design document, which was written as a part of GSOC-2016, https://cwiki.apache.org/confluence/display/AIRAVATA/%5BGsoc+Proposal%5D+Resource+Allocation+Manager+for+Apache+Airavata. I have derived some use cases by discussing with Sudhakar Pamidighantam ( PI - GridChem), here is the link : https://issues.apache.org/jira/browse/AIRAVATA-2108. Any comments and improvements are gladly welcome. Thanks and best regards, Anuj Bhandar
