Just configure <pool name="default"> with <schedulingMode>FAIR</schedulingMode> in fairscheduler.xml (or in spark.scheduler.allocation.file if you have over-riden the default name for the config file.) `buildDefaultPool()` will only build the pool named "default" with the default properties (such as schedulingMode = DEFAULT_SCHEDULING_MODE -- i.e. SchedulingMode.FAIR) if that pool name is not already built ( https://github.com/apache/spark/blob/master/core/src/main/scala/org/apache/spark/scheduler/SchedulableBuilder.scala#L90 ).
On Tue, Jan 5, 2016 at 4:15 PM, Jeff Zhang <zjf...@gmail.com> wrote: > Sorry, I don't make it clearly. What I want is the default pool is fair > scheduling. But seems if I want to use fair scheduling now, I have to set > spark.scheduler.pool explicitly. > > On Wed, Jan 6, 2016 at 2:03 AM, Mark Hamstra <m...@clearstorydata.com> > wrote: > >> I don't understand. If you're using fair scheduling and don't set a >> pool, the default pool will be used. >> >> On Tue, Jan 5, 2016 at 1:57 AM, Jeff Zhang <zjf...@gmail.com> wrote: >> >>> >>> It seems currently spark.scheduler.pool must be set as localProperties >>> (associate with thread). Any reason why spark.scheduler.pool can not be >>> used globally. My scenario is that I want my thriftserver started with >>> fair scheduler as the default pool without using set command to set the >>> pool. Is there anyway to do that ? Or do I miss anything here ? >>> >>> -- >>> Best Regards >>> >>> Jeff Zhang >>> >> >> > > > -- > Best Regards > > Jeff Zhang >
