Hua Zhang created SPARK-27104: --------------------------------- Summary: Spark Fair scheduler across applications in standalone mode Key: SPARK-27104 URL: https://issues.apache.org/jira/browse/SPARK-27104 Project: Spark Issue Type: Wish Components: Scheduler Affects Versions: 2.4.0, 2.3.3, 2.2.3 Reporter: Hua Zhang
Spark in standalone mode currently only supports FIFO (first-in-first-out) scheduler across applications. It will be great that a fair scheduler is supported. +A fair scheduler across applications, not in a application.+ Use case (for example with the integration of zeppelin) At certain moment, user A submits an heavy application en consumes all the resources of the spark cluster. At a later moment, user B submits a second application. No matter how many work nodes you added now, all the resources go to user A due to the FIFO. User B will never get any resource until user A release its allocated resources. A fair scheduler should distribute extra resources in a fair way on all running applications, which demands resources. -- This message was sent by Atlassian JIRA (v7.6.3#76005) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org