Hi Shameera, Can you please map this to the diagram at [1]? Will the HPCPullMonitor be equivalent to the BufferedQueue we discussed on the architecture list?
Suresh [1] - https://cwiki.apache.org/confluence/display/AIRAVATA/Airavata+Metascheduler On Sep 11, 2014, at 10:29 AM, Shameera Rathnayaka <[email protected]> wrote: > Hi devs, > > I am going to implement the $Subject > > Requirement: Introduce a max job submission count for a given resource under > a given username. > > Abstraction: When user submits a new experiment to the airavata, user selects > the resource (Machine) where airavata should run that experiment (Job). That > resource may have job count restriction like under one user there can only be > have X number of jobs either in Q or R state. So we need to handle this at > Orchestrator level rather than handing over the experiment to GFac to submit > the jobs where it gets rejected because of that restriction. To do that > Orchestrator need to know the job count of particular user in that given > resource. > > > Implementation: HPCPullMonitor will write stat data to zookeeper, zookeeper > path would be something like /stat/{username}/{machine}/jobs/{count}. > Orchestrator will register a watcher for this data change and that watcher > will trigger when any GFac node(Monitor component) update the job status > realtime. Finished jobs will immediately decrement the count and these > changes will replicate in Orchestrator with ZK watches. > > Thanks, > Shameera. > > -- > Best Regards, > Shameera Rathnayaka. > > email: shameera AT apache.org , shameerainfo AT gmail.com > Blog : http://shameerarathnayaka.blogspot.com/
