I wonder if you might be better off just having 2 clusters, it might make things easier to manage in the long term.
> On Sep 28, 2016, at 5:01 PM, Jacob Scott <[email protected]> wrote: > > Thanks much (and thanks for fielding a version of this question in IRC!). > > Jacob > > On Wed, Sep 28, 2016 at 1:24 PM Zameer Manji <[email protected] > <mailto:[email protected]>> wrote: > +1 > > I've found the complexity of managing mesos roles to be high. For a fix that > has less overhead, I think scheduling on attributes via Aurora's constraint > system would be easiest. > > On Wed, Sep 28, 2016 at 1:20 PM, Jacob Scott <[email protected] > <mailto:[email protected]>> wrote: > I'm running Chronos and Aurora, with an existing workload which has high > reliability requirements. I'm adding in a new, more experimental workload, > and would like the different workloads to run on different hardware to reduce > risk. Some of the risk comes from me playing fast and loose with per-job > cpu/mem/disk, but I expect it to be easier to run for now with different > agent classes than untangle that ball of wax. There's also some secondary > benefits like better attribution for billing. > > Based on further reading on roles, and the complexity of reserving resources > for roles, I'm currently planning to use attributes -- one Aurora, one > Chronos, one Mesos Master, agents classified via attribute). > > > Thanks, > > Jacob > > On Wed, Sep 28, 2016 at 1:11 PM, Erb, Stephan <[email protected] > <mailto:[email protected]>> wrote: > Hi Jacob, > > > > could you elaborate why you want to establish two different classes of > agents? Do those have different hardware specs or is this due to a different > network setup or something like that? > > > > Furthermore, do you plan to run other frameworks in addition to Aurora? > > > > If you only plan to use Aurora for now, I would start with a single Aurora > scheduler and see how this works out for you. Aurora does not immediately > reject offers but it holds onto them for a default of 5 minutes (IIRC). The > single scheduler then has a complete view of all available resources and > should be able to schedule your jobs rather quickly. > > > > Best Regards, > > Stephan > > > > From: Jacob Scott <[email protected] <mailto:[email protected]>> > Reply-To: "[email protected] <mailto:[email protected]>" > <[email protected] <mailto:[email protected]>> > Date: Tuesday 27 September 2016 at 07:34 > To: "[email protected] <mailto:[email protected]>" > <[email protected] <mailto:[email protected]>> > Subject: roles or attributes for virtual clusters > > > > I have a mesos cluster running on EC2 that I want to split into two "virtual > clusters" to run Aurora jobs on -- e.g. blue nodes/instances for blue jobs > and red nodes/instances for red jobs. > > > > After reading the documentation I can see two ways of doing this: > > Set color:red and color:blue attributes on mesos agents[1] and similarly in > Aurora job config [2] > Using mesos roles [3], Run one blue Aurora scheduler [4] and one red scheduler > Is one approach here better, or more in line with best practices? Two > schedulers seems like more operational overhead, but using attributes might > result in a higher offer reject rate (Aurora receiving and rejecting red > offers when trying to run a blue job)? > > > > I appreciate any advice or pointers to relevant resources. > > > > Thanks, > > > > Jacob > > > > > > [1] http://mesos.apache.org/documentation/latest/attributes-resources/ > <http://mesos.apache.org/documentation/latest/attributes-resources/> > [2] http://aurora.apache.org/documentation/latest/features/constraints/ > <http://aurora.apache.org/documentation/latest/features/constraints/> > [3] http://mesos.apache.org/documentation/latest/roles/ > <http://mesos.apache.org/documentation/latest/roles/> > [4] > http://aurora.apache.org/documentation/latest/reference/scheduler-configuration/ > > <http://aurora.apache.org/documentation/latest/reference/scheduler-configuration/> >
signature.asc
Description: Message signed with OpenPGP using GPGMail
