Haven't tried it but this might be possible using label based scheduling (MapR/YARN). I am thinking if all CGS nodes are labeled 'Production' and FGS ones 'Non Production' and there are two queues one labeled 'Production' and the other 'Non Production'. If a job is submitted to the Non Production queue, its containers will only run on the FGS nodes.
Regards Swapnil On Wed, Dec 2, 2015 at 10:04 PM, Adam Bordelon <[email protected]> wrote: > Sounds great! The all-revocable vs. all-guaranteed strategy seems like the > easiest to implement (just add the Capability to FrameworkInfo), but making > Myriad itself smart about high-priority vs. best-effort backfill jobs. > > On Wed, Dec 2, 2015 at 6:27 AM, John Omernik <[email protected]> wrote: > > > Hey all, just curious if there has been any discussion around supporting > > oversubscription in Myriad. Based on my reading of things, Myriad would > be > > an awesome use case for over subscription, especially when you combine it > > with the FGS. Based on what I've read on oversubscription, if Myriad was > > aware of oversubscription, we could have Myriad be smart about various > Yarn > > containers, and have some jobs that may be production jobs, they could > run > > on non-revocable resources, but could we have yarn jobs with certain > > users/flags, especially in FGS mode be submitted using the revocable > > resources? These are the jobs that would be adhoc in nature, and in > > addition to not using resources when no jobs are running, the node > > managers, when they did run certain jobs would run on the revocable > > resources. > > > > I am speaking now not from a Dev perspective, so this may be a lot harder > > than it seems. > > > > Another approach would be once we have the the multi-tenancy built in, > have > > a whole myriad framework dedicate to adhoc type jobs, and have another > > myriad framework dedicated to production jobs. > > > > I see use cases for both, this just seems to add another layer of awesome > > flexibility as it pertains to jobs on the cluster. > > > > I'd be interested in the group's thoughts here. > > > > John > > >
