Hi Fabian, Thanks for sharing the issue. SparkSqlInterpreter set scheduler to "fair" depends on interpreter property [1]. I think we can do the similar for SparkInterpreter. Do you mind file a new JIRA issue for it?
Regards, moon [1] https://github.com/apache/zeppelin/blob/0e1964877654c56c72473ad07dac1de6f9646816/spark/src/main/java/org/apache/zeppelin/spark/SparkSqlInterpreter.java#L98 On Tue, Mar 28, 2017 at 5:24 AM Fabian Böhnlein <fabian.boehnl...@gmail.com> wrote: > Hi all, > > how can I change (globally, for Zeppelin) the default scheduler pool which > SparkInterpreter submits jobs to. Currently all jobs go into the pool > 'default' but I want them to go into the pool 'fair'. > We use "Per Note" and "scoped" processes for best resource sharing. > > "spark.scheduler.pool"="fair" in Interpreter Settings does not work, > should it? > > What works is > sc.setLocalProperty("spark.scheduler.pool","fair") > but it's required in every *note* (not just notebook) since it's on > thread level. > > Is there a possibility to globally/per notebook set the 'fair' pool as the > default pool? > > Zeppelin brings two (hardcoded?) sheduler pools 'default' and 'fair'. > Between them, the scheduling is FAIR. 'default' is FIFO, 'fair' is FAIR. > > This is awesome and together with dynamicAllocation allows for super > flexible usage for multiple users but above behavior is a bit complicated. > > Thanks, > Fabian > > > >
