Hi all,
According to the documentation, spark standalone currently only supports
a FIFO scheduling system.
I understand its possible to limit the number of cores a job uses by
setting |spark.cores.max.
|When running a job, will spark try using the max number of cores on
each machine until it reaches the set limit, or will it do this round
robin style - utilize a single core on each machine - if its already
used a core on all of the slaves and the limit has not been reached,
spark will utilize an additional core on each machine and so on.
I think the latter make more sense, but I want to be sure that is the case.
Thanks,
Yadid
