Sounds great! The all-revocable vs. all-guaranteed strategy seems like the easiest to implement (just add the Capability to FrameworkInfo), but making Myriad itself smart about high-priority vs. best-effort backfill jobs.
On Wed, Dec 2, 2015 at 6:27 AM, John Omernik <[email protected]> wrote: > Hey all, just curious if there has been any discussion around supporting > oversubscription in Myriad. Based on my reading of things, Myriad would be > an awesome use case for over subscription, especially when you combine it > with the FGS. Based on what I've read on oversubscription, if Myriad was > aware of oversubscription, we could have Myriad be smart about various Yarn > containers, and have some jobs that may be production jobs, they could run > on non-revocable resources, but could we have yarn jobs with certain > users/flags, especially in FGS mode be submitted using the revocable > resources? These are the jobs that would be adhoc in nature, and in > addition to not using resources when no jobs are running, the node > managers, when they did run certain jobs would run on the revocable > resources. > > I am speaking now not from a Dev perspective, so this may be a lot harder > than it seems. > > Another approach would be once we have the the multi-tenancy built in, have > a whole myriad framework dedicate to adhoc type jobs, and have another > myriad framework dedicated to production jobs. > > I see use cases for both, this just seems to add another layer of awesome > flexibility as it pertains to jobs on the cluster. > > I'd be interested in the group's thoughts here. > > John >
