GitHub user sirpkt opened a pull request:
https://github.com/apache/spark/pull/7119
[SPARK-8728] Add configuration for limiting the maximum number of active
stages in a fair scheduling queueî
It only takes predefined number of Schedulables in getSortedTaskSetQueue.
'maxRunning' is added in fair scheduler configuration to limit the maximum
number of concurrently running stages.
If 'maxRunning' is not set, no limitation is applied.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/sirpkt/spark SPARK-8728
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/7119.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #7119
----
commit 4988a721a284bc5cb2ce05a3d4df8040cc98d872
Author: Keuntae Park <[email protected]>
Date: 2015-06-26T04:40:16Z
add limitation on the number of TaskSetManagers scheduled concurrently
commit e59ee3e44d03d4ad8505a32333ae89956c7be9d3
Author: Keuntae Park <[email protected]>
Date: 2015-06-30T08:12:57Z
Merge remote-tracking branch 'upstream/master' into limitMaxRunning
commit 7f620705efb11241312a0916580b28c97a4f24bc
Author: Keuntae Park <[email protected]>
Date: 2015-06-30T08:13:40Z
reflect in web UI and refine the test code
commit a8392108749cc93d7bdf1355bb065614fd4b1c07
Author: Keuntae Park <[email protected]>
Date: 2015-06-30T08:47:22Z
change test name
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
