Hi Niklas, I am using Apache spark with mesos 0.19.1. I have limited resources and when I submit a job which takes all of the resources. This is fine as when no one is using them, but when one of my colleagues submit his job, I would like mesos allows some part of resources assigned to his job when part of my jobs are finished, But it seems currently it waits until my whole job is finished before starting the other job. Is it due to mesos or you think Spark is the one who is blocking the job.
- Gurvinder On 07/31/2014 06:12 PM, Niklas Nielsen wrote: > Hi Gurvinder, > > The frameworks competing for resources will get their (weighted) fair > share of the cluster. The allocator in the master uses the Dominant > Resource Fairness algorithm to do this > (http://static.usenix.org/event/nsdi11/tech/full_papers/Ghodsi.pdf). > Regarding FIFO, are you referring to 'local' scheduler policies? How > tasks are dispatched is up to the individual framework. > > Cheers, > Niklas > > > On 31 July 2014 07:28, Gurvinder Singh <[email protected] > <mailto:[email protected]>> wrote: > > Hi, > > I am wondering how mesos handle the task scheduling when the resource > are limited and multiple users want to access them at the same time. Is > there any kind of fair scheduling as I see currently mainly FIFO. If > there is how can I specify that. > > Thanks, > Gurvinder > >
