Thank you for that suggestion, it seems to be exactly what we're looking to do. Does mesos-execute support constraints?
Thanks, June Taylor System Administrator, Minnesota Population Center University of Minnesota On Sat, Apr 16, 2016 at 9:57 AM, haosdent <[email protected]> wrote: > >We expect these tasks to only go on nodes with the "production" resource > role. > > Actually most frameworks support constraints(spark/marathon/chronos). It > could be used to limit tasks executed on the Mesos Agent that satisfied > conditions. For your scenario, you could restart Mesos agent by adding > `--attributes=env:production` flag. And launch your tasks with > `"constraints": [["env", "LIKE", "production"]]`. > > For further details, you could checkout > https://github.com/mesosphere/marathon/blob/master/docs/docs/constraints.md > > > On Sat, Apr 16, 2016 at 10:10 AM, Klaus Ma <[email protected]> wrote: > >> Which version are you using? For your requirement, I think you can try >> Quota; currently, the resources beyond quota will not offer to the >> framework whose quota satisfied. Quota will also include reserved resources. >> >> ---- >> Da (Klaus), Ma (马达) | PMP® | Advisory Software Engineer >> Platform OpenSource Technology, STG, IBM GCG >> +86-10-8245 4084 | [email protected] | http://k82.me >> >> On Sat, Apr 16, 2016 at 4:54 AM, Rodrick Brown < >> [email protected]> wrote: >> >>> You can try setting constraints on tasks in both Chronos and marathon >>> that will limit deployment to only a certain set of nodes. >>> >>> Sent from Outlook for iPhone <https://aka.ms/wp8k5y> >>> >>> >>> >>> >>> On Fri, Apr 15, 2016 at 1:35 PM -0700, "June Taylor" <[email protected]> >>> wrote: >>> >>> Evan, >>>> >>>> I'm not sure about it. We're new to the Mesos system and still >>>> learning. We want to be able to classify resources so that our developers >>>> can run tasks against them easily, without using more than they are >>>> permitted. It seemed like resource roles were the appropriate solution, but >>>> they may not go far enough if Mesos will still spill over into default >>>> resources. >>>> >>>> >>>> Thanks, >>>> June Taylor >>>> System Administrator, Minnesota Population Center >>>> University of Minnesota >>>> >>>> On Fri, Apr 15, 2016 at 3:27 PM, Evan Krall <[email protected]> wrote: >>>> >>>>> My understanding is that your framework would have to know not to >>>>> accept offers for * resources. Marathon has an option to specify which >>>>> roles to accept for a particular app, and has command line options for >>>>> controlling the default. Maybe pyspark has something similar? >>>>> >>>>> On Fri, Apr 15, 2016 at 1:24 PM, June Taylor <[email protected]> wrote: >>>>> >>>>>> Yep - we're waiting for it. >>>>>> >>>>>> >>>>>> Thanks, >>>>>> June Taylor >>>>>> System Administrator, Minnesota Population Center >>>>>> University of Minnesota >>>>>> >>>>>> On Fri, Apr 15, 2016 at 3:23 PM, Anand Mazumdar <[email protected]> >>>>>> wrote: >>>>>> >>>>>>> FWIW, we recently fixed `mesos-execute` (command scheduler) to add >>>>>>> support for roles. It should be available in the next release (0.29). >>>>>>> >>>>>>> https://issues.apache.org/jira/browse/MESOS-4744 >>>>>>> >>>>>>> -anand >>>>>>> >>>>>>> On Apr 15, 2016, at 11:41 AM, June Taylor <[email protected]> wrote: >>>>>>> >>>>>>> Ken, >>>>>>> >>>>>>> Thanks for your reply. >>>>>>> >>>>>>> Is there a way to ensure a framework only receives the reserved >>>>>>> resources? >>>>>>> >>>>>>> I would go ahead and take everything out of the * role, however, the >>>>>>> 'mesos-execute' command doesn't support specifying a role, so that's the >>>>>>> only way we can currently get mesos-execute to co-exist with pyspark. >>>>>>> >>>>>>> Any other thoughts from the group? >>>>>>> >>>>>>> >>>>>>> Thanks, >>>>>>> June Taylor >>>>>>> System Administrator, Minnesota Population Center >>>>>>> University of Minnesota >>>>>>> >>>>>>> On Fri, Apr 15, 2016 at 11:54 AM, Ken Sipe <[email protected]> >>>>>>> wrote: >>>>>>> >>>>>>>> The framework with role “production” will receive production >>>>>>>> resources and * resources >>>>>>>> All other frameworks (assuming no role) will only receive * >>>>>>>> resources >>>>>>>> >>>>>>>> ken >>>>>>>> >>>>>>>> > On Apr 15, 2016, at 11:38 AM, June Taylor <[email protected]> wrote: >>>>>>>> > >>>>>>>> > We have a small cluster with 3 nodes in the * resource role >>>>>>>> default, and 3 nodes in a "production" resource role. >>>>>>>> > >>>>>>>> > Starting up a framework which requests "production" properly >>>>>>>> executes on the expected nodes, however, today we noticed that this job >>>>>>>> also started up executors under the * resource role as well. >>>>>>>> > >>>>>>>> > We expect these tasks to only go on nodes with the "production" >>>>>>>> resource role. Can you advise further? >>>>>>>> > >>>>>>>> > Thanks, >>>>>>>> > June Taylor >>>>>>>> > System Administrator, Minnesota Population Center >>>>>>>> > University of Minnesota >>>>>>>> >>>>>>>> >>>>>>> >>>>>>> >>>>>> >>>>> >>>> >>> *NOTICE TO RECIPIENTS*: This communication is confidential and intended >>> for the use of the addressee only. If you are not an intended recipient of >>> this communication, please delete it immediately and notify the sender >>> by return email. Unauthorized reading, dissemination, distribution or >>> copying of this communication is prohibited. This communication does not >>> constitute >>> an offer to sell or a solicitation of an indication of interest to purchase >>> any loan, security or any other financial product or instrument, nor is it >>> an offer to sell or a solicitation of an indication of interest to purchase >>> any products or services to any persons who are prohibited from receiving >>> such information under applicable law. The contents of this communication >>> may not be accurate or complete and are subject to change without notice. >>> As such, Orchard App, Inc. (including its subsidiaries and affiliates, >>> "Orchard") makes no representation regarding the accuracy or >>> completeness of the information contained herein. The intended recipient is >>> advised to consult its own professional advisors, including those >>> specializing in legal, tax and accounting matters. Orchard does not >>> provide legal, tax or accounting advice. >>> >> >> > > > -- > Best Regards, > Haosdent Huang >
