Hi Isuru,
Thanks for the updates. Great to see progress is being made.
I plan to dive into the UltraScan side of this starting in January,
perhaps we can setup a meeting early January to see what is implemented,
how the new UltraScan microservice can be integrated & begin
planning/running some tests?
Best wishes,
Emre
Isuru Ranawaka wrote:
*EXTERNAL EMAIL*
Hi all,
We have implemented the initial phase of the Airavata Metascheduler.
Following is the finalized architecture at this phase.
image.png
Airavata Experiments can be configured for auto-scheduling and for
such Experiments Orchestrator invokes meta scheduler to check whether
the Experiment is schedulable. If it is schedulable Orchestrator
submits the respective Process of the Experiment to Helix Framework
for instant scheduling otherwise Experiment and Process are Queued.
*Metascheduler Architecture*
image.png
Process scanner scans for already queued Processes
periodically and sends those Process information to the MetaScheduler
Engine. Metascheduler Engine evaluates the chance of successful
submission based on configured scheduling policies. If it is
submittable MetaScheduler engine invokes Orchestrator APIs to initiate
the Job Submission for those Processes through Helix Framework.
* Compute Resource Monitoring Service
Compute resource monitoring service uses Airavata Group Resource
Profile information to periodically monitor configured Remote Compute
Resources and update the status of the Queues in the registry. So
the information is consumed by Metacheduler Engine to schedule or
reschedule Processes.
thanks
Isuru
On Tue, Oct 4, 2022 at 7:41 AM Isuru Ranawaka <irjan...@gmail.com
<mailto:irjan...@gmail.com>> wrote:
Hi all,
Airavata MetaSchedular is a managed job submission framework for
High-Performance Computing resources. Basically, It handles
*
User-based load balancing of Jobs between resource queues.
*
Dynamic Job scheduling across compute resources
*
Job queueing and managed submissions
and the following document [1] contains the captured use cases.
Please have a look and make suggestions, comments are highly
appreciated.
I will update the thread as I proceed with the implementation.
thanks
Isuru
[1]https://docs.google.com/document/d/1i_cAR7ulNpkEATt0VW4-xoYZyZTQbdnNAJVbnAlMI4I/edit?usp=sharing
<https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdocs.google.com%2Fdocument%2Fd%2F1i_cAR7ulNpkEATt0VW4-xoYZyZTQbdnNAJVbnAlMI4I%2Fedit%3Fusp%3Dsharing&data=05%7C01%7Cemre.brookes%40mso.umt.edu%7Cef364bd1eacf40f9084708dad7c05f78%7C68407ce503da49ffaf0a724be0d37c9d%7C0%7C0%7C638059518517375866%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=TakGZiJUBDCypMJMjsD0yNrjayREybgM8JBtQVu9hJM%3D&reserved=0>
--
Research Software Engineer
Indiana University, IN
--
Research Software Engineer
Indiana University, IN
CAUTION: This message originated outside of UT Health San Antonio.
Please exercise caution when clicking on links or opening attachments.
