Thanks, Adam.. Sounds like it is going to be pretty effective when all the framework running a short tasks, then mesos can balance the resource allocation based on the DRF among the framework quickly. If one framework is happening to run some long tasks and take too many resources, mesos have to wait until some resource being free up to assign to other framework. Is it correct? Best Regards Jun Feng Liu IBM China Systems & Technology Laboratory in Beijing
Phone: 86-10-82452683 E-mail: [email protected] BLD 28,ZGC Software Park No.8 Rd.Dong Bei Wang West, Dist.Haidian Beijing 100193 China Adam Bordelon <[email protected]> 2014/08/18 16:26 Please respond to [email protected] To "[email protected]" <[email protected]>, cc Jay Buffington <[email protected]> Subject Re: mesos scheduling Mesos uses a fair-sharing algorithm[1] to ensure that each framework registered with Mesos is ensured its fair share of resources. If you want more control over the groupings and weights of different frameworks, check out the roles and weights parameters: mesos-master --roles="services,batch" and --weights="services=2,batch=1" as described at http://mesosphere.io/docs/mesos/deep-dive/mesos-master/ Mesos uses these algorithms and parameters to decide which framework gets the next offer, so it won't affect already running tasks if one framework is already hogging the cluster when you start a new framework. But if you start killing tasks from the over-provisioned framework, those resources will be offered to the new framework(s) until it reaches its fair share. [1] http://static.usenix.org/event/nsdi11/tech/full_papers/Ghodsi.pdf On Sun, Aug 17, 2014 at 7:06 PM, Jun Feng Liu <[email protected]> wrote: Thanks Jay.. Dose it mean if one of scheduler/frame need a lot resource and keep ask for more resources from mesos, then it will cause other framework/scheduler hard to get resources? Any way I can configure the mesos to setup a resource consuming boundary for each framework? Best Regards Jun Feng Liu IBM China Systems & Technology Laboratory in Beijing Phone: 86-10-82452683 E-mail: [email protected] BLD 28,ZGC Software Park No.8 Rd.Dong Bei Wang West, Dist.Haidian Beijing 100193 China Jay Buffington <[email protected]> Sent by: [email protected] 2014/08/18 02:44 Please respond to [email protected] To [email protected], cc Subject Re: mesos scheduling On Sun, Aug 17, 2014 at 6:13 AM, Jun Feng Liu <[email protected]> wrote: I am trying to better understand how mesos allocator works. In the offer resource model, will mesos send the same offer to multiple framework? Or it just send all resource to one framework then wait the response from the the framework then try the next one? Mesos sends an offer to one scheduler (a scheduler is part of a framework) at a time. That scheduler will have the offer until it uses it, gives it back or mesos rescinds it. This strategy was referred to as "pessimistic" by Google's Omega paper [1] and has drawbacks. In order to address these points a new type of offer, an Optimistic Offer, is being considered. See https://issues.apache.org/jira/browse/MESOS-1607 Jay [1] http://eurosys2013.tudos.org/wp-content/uploads/2013/paper/Schwarzkopf.pdf
