Sorry for the tardy reply. Spark seems to be holding on to those resources
then - are you running Spark in coarse grained mode?
I am not too Spark savvy, but running in fine grained mode should allocate
and free resources on-demand rather than allocating a temporary static
partition up front.
Hi Niklas,
I am using Apache spark with mesos 0.19.1. I have limited resources and
when I submit a job which takes all of the resources. This is fine as
when no one is using them, but when one of my colleagues submit his job,
I would like mesos allows some part of resources assigned to his job
Hi,
I am wondering how mesos handle the task scheduling when the resource
are limited and multiple users want to access them at the same time. Is
there any kind of fair scheduling as I see currently mainly FIFO. If
there is how can I specify that.
Thanks,
Gurvinder
Hi Gurvinder,
The frameworks competing for resources will get their (weighted) fair share
of the cluster. The allocator in the master uses the Dominant Resource
Fairness algorithm to do this (
http://static.usenix.org/event/nsdi11/tech/full_papers/Ghodsi.pdf).
Regarding FIFO, are you referring to
4 matches
Mail list logo
