[GitHub] spark pull request #19194: [SPARK-20589] Allow limiting task concurrency per...

2017-09-27 Thread dhruve
Github user dhruve commented on a diff in the pull request: https://github.com/apache/spark/pull/19194#discussion_r141482741 --- Diff: core/src/main/scala/org/apache/spark/scheduler/TaskSetManager.scala --- @@ -512,6 +535,9 @@ private[spark] class TaskSetManager(

[GitHub] spark pull request #19194: [SPARK-20589] Allow limiting task concurrency per...

2017-09-27 Thread squito
Github user squito commented on a diff in the pull request: https://github.com/apache/spark/pull/19194#discussion_r141475205 --- Diff: core/src/main/scala/org/apache/spark/scheduler/TaskSetManager.scala --- @@ -512,6 +535,9 @@ private[spark] class TaskSetManager(

[GitHub] spark pull request #19194: [SPARK-20589] Allow limiting task concurrency per...

2017-09-21 Thread dhruve
Github user dhruve commented on a diff in the pull request: https://github.com/apache/spark/pull/19194#discussion_r140361162 --- Diff: core/src/main/scala/org/apache/spark/ExecutorAllocationManager.scala --- @@ -619,6 +625,47 @@ private[spark] class ExecutorAllocationManager(

[GitHub] spark pull request #19194: [SPARK-20589] Allow limiting task concurrency per...

2017-09-21 Thread markhamstra
Github user markhamstra commented on a diff in the pull request: https://github.com/apache/spark/pull/19194#discussion_r140340018 --- Diff: core/src/main/scala/org/apache/spark/ExecutorAllocationManager.scala --- @@ -619,6 +625,47 @@ private[spark] class ExecutorAllocationManager(

[GitHub] spark pull request #19194: [SPARK-20589] Allow limiting task concurrency per...

2017-09-21 Thread squito
Github user squito commented on a diff in the pull request: https://github.com/apache/spark/pull/19194#discussion_r140337893 --- Diff: core/src/main/scala/org/apache/spark/ExecutorAllocationManager.scala --- @@ -619,6 +625,47 @@ private[spark] class ExecutorAllocationManager(

[GitHub] spark pull request #19194: [SPARK-20589] Allow limiting task concurrency per...

2017-09-21 Thread tgravescs
Github user tgravescs commented on a diff in the pull request: https://github.com/apache/spark/pull/19194#discussion_r140332294 --- Diff: core/src/main/scala/org/apache/spark/ExecutorAllocationManager.scala --- @@ -619,6 +625,47 @@ private[spark] class ExecutorAllocationManager(

[GitHub] spark pull request #19194: [SPARK-20589] Allow limiting task concurrency per...

2017-09-21 Thread dhruve
Github user dhruve commented on a diff in the pull request: https://github.com/apache/spark/pull/19194#discussion_r140321640 --- Diff: core/src/main/scala/org/apache/spark/ExecutorAllocationManager.scala --- @@ -619,6 +625,47 @@ private[spark] class ExecutorAllocationManager(

[GitHub] spark pull request #19194: [SPARK-20589] Allow limiting task concurrency per...

2017-09-21 Thread squito
Github user squito commented on a diff in the pull request: https://github.com/apache/spark/pull/19194#discussion_r140296863 --- Diff: core/src/main/scala/org/apache/spark/ExecutorAllocationManager.scala --- @@ -619,6 +625,47 @@ private[spark] class ExecutorAllocationManager(

[GitHub] spark pull request #19194: [SPARK-20589] Allow limiting task concurrency per...

2017-09-21 Thread dhruve
Github user dhruve commented on a diff in the pull request: https://github.com/apache/spark/pull/19194#discussion_r140293577 --- Diff: core/src/main/scala/org/apache/spark/ExecutorAllocationManager.scala --- @@ -619,6 +625,47 @@ private[spark] class ExecutorAllocationManager(

[GitHub] spark pull request #19194: [SPARK-20589] Allow limiting task concurrency per...

2017-09-20 Thread squito
Github user squito commented on a diff in the pull request: https://github.com/apache/spark/pull/19194#discussion_r140136101 --- Diff: core/src/main/scala/org/apache/spark/ExecutorAllocationManager.scala --- @@ -619,6 +625,47 @@ private[spark] class ExecutorAllocationManager(

[GitHub] spark pull request #19194: [SPARK-20589] Allow limiting task concurrency per...

2017-09-20 Thread dhruve
Github user dhruve commented on a diff in the pull request: https://github.com/apache/spark/pull/19194#discussion_r140122744 --- Diff: core/src/test/scala/org/apache/spark/scheduler/TaskSetManagerSuite.scala --- @@ -1255,6 +1255,97 @@ class TaskSetManagerSuite extends

[GitHub] spark pull request #19194: [SPARK-20589] Allow limiting task concurrency per...

2017-09-20 Thread dhruve
Github user dhruve commented on a diff in the pull request: https://github.com/apache/spark/pull/19194#discussion_r140124886 --- Diff: core/src/main/scala/org/apache/spark/ExecutorAllocationManager.scala --- @@ -619,6 +625,47 @@ private[spark] class ExecutorAllocationManager(

[GitHub] spark pull request #19194: [SPARK-20589] Allow limiting task concurrency per...

2017-09-20 Thread dhruve
Github user dhruve commented on a diff in the pull request: https://github.com/apache/spark/pull/19194#discussion_r140122769 --- Diff: core/src/test/scala/org/apache/spark/scheduler/TaskSetManagerSuite.scala --- @@ -1255,6 +1255,97 @@ class TaskSetManagerSuite extends

[GitHub] spark pull request #19194: [SPARK-20589] Allow limiting task concurrency per...

2017-09-20 Thread dhruve
Github user dhruve commented on a diff in the pull request: https://github.com/apache/spark/pull/19194#discussion_r140123047 --- Diff: core/src/main/scala/org/apache/spark/ExecutorAllocationManager.scala --- @@ -758,11 +825,52 @@ private[spark] class ExecutorAllocationManager(

[GitHub] spark pull request #19194: [SPARK-20589] Allow limiting task concurrency per...

2017-09-19 Thread squito
Github user squito commented on a diff in the pull request: https://github.com/apache/spark/pull/19194#discussion_r139819360 --- Diff: core/src/test/scala/org/apache/spark/scheduler/TaskSetManagerSuite.scala --- @@ -1255,6 +1255,97 @@ class TaskSetManagerSuite extends

[GitHub] spark pull request #19194: [SPARK-20589] Allow limiting task concurrency per...

2017-09-19 Thread squito
Github user squito commented on a diff in the pull request: https://github.com/apache/spark/pull/19194#discussion_r139819793 --- Diff: core/src/test/scala/org/apache/spark/scheduler/TaskSetManagerSuite.scala --- @@ -1255,6 +1255,97 @@ class TaskSetManagerSuite extends

[GitHub] spark pull request #19194: [SPARK-20589] Allow limiting task concurrency per...

2017-09-19 Thread squito
Github user squito commented on a diff in the pull request: https://github.com/apache/spark/pull/19194#discussion_r139821899 --- Diff: core/src/main/scala/org/apache/spark/ExecutorAllocationManager.scala --- @@ -619,6 +625,47 @@ private[spark] class ExecutorAllocationManager(

[GitHub] spark pull request #19194: [SPARK-20589] Allow limiting task concurrency per...

2017-09-19 Thread squito
Github user squito commented on a diff in the pull request: https://github.com/apache/spark/pull/19194#discussion_r139818031 --- Diff: core/src/main/scala/org/apache/spark/ExecutorAllocationManager.scala --- @@ -758,11 +825,52 @@ private[spark] class ExecutorAllocationManager(

[GitHub] spark pull request #19194: [SPARK-20589] Allow limiting task concurrency per...

2017-09-15 Thread dhruve
Github user dhruve commented on a diff in the pull request: https://github.com/apache/spark/pull/19194#discussion_r139163830 --- Diff: core/src/test/scala/org/apache/spark/scheduler/TaskSetManagerSuite.scala --- @@ -1255,6 +1255,97 @@ class TaskSetManagerSuite extends

[GitHub] spark pull request #19194: [SPARK-20589] Allow limiting task concurrency per...

2017-09-15 Thread dhruve
Github user dhruve commented on a diff in the pull request: https://github.com/apache/spark/pull/19194#discussion_r139163463 --- Diff: core/src/test/scala/org/apache/spark/ExecutorAllocationManagerSuite.scala --- @@ -210,22 +216,282 @@ class ExecutorAllocationManagerSuite

[GitHub] spark pull request #19194: [SPARK-20589] Allow limiting task concurrency per...

2017-09-15 Thread dhruve
Github user dhruve commented on a diff in the pull request: https://github.com/apache/spark/pull/19194#discussion_r139163529 --- Diff: core/src/test/scala/org/apache/spark/ExecutorAllocationManagerSuite.scala --- @@ -210,22 +216,282 @@ class ExecutorAllocationManagerSuite

[GitHub] spark pull request #19194: [SPARK-20589] Allow limiting task concurrency per...

2017-09-15 Thread dhruve
Github user dhruve commented on a diff in the pull request: https://github.com/apache/spark/pull/19194#discussion_r139163109 --- Diff: core/src/main/scala/org/apache/spark/ExecutorAllocationManager.scala --- @@ -758,11 +812,58 @@ private[spark] class ExecutorAllocationManager(

[GitHub] spark pull request #19194: [SPARK-20589] Allow limiting task concurrency per...

2017-09-15 Thread dhruve
Github user dhruve commented on a diff in the pull request: https://github.com/apache/spark/pull/19194#discussion_r139162871 --- Diff: core/src/main/scala/org/apache/spark/ExecutorAllocationManager.scala --- @@ -758,11 +812,58 @@ private[spark] class ExecutorAllocationManager(

[GitHub] spark pull request #19194: [SPARK-20589] Allow limiting task concurrency per...

2017-09-15 Thread tgravescs
Github user tgravescs commented on a diff in the pull request: https://github.com/apache/spark/pull/19194#discussion_r139157280 --- Diff: core/src/test/scala/org/apache/spark/scheduler/TaskSetManagerSuite.scala --- @@ -1255,6 +1255,97 @@ class TaskSetManagerSuite extends

[GitHub] spark pull request #19194: [SPARK-20589] Allow limiting task concurrency per...

2017-09-15 Thread tgravescs
Github user tgravescs commented on a diff in the pull request: https://github.com/apache/spark/pull/19194#discussion_r139156205 --- Diff: core/src/test/scala/org/apache/spark/ExecutorAllocationManagerSuite.scala --- @@ -210,22 +216,282 @@ class ExecutorAllocationManagerSuite

[GitHub] spark pull request #19194: [SPARK-20589] Allow limiting task concurrency per...

2017-09-15 Thread tgravescs
Github user tgravescs commented on a diff in the pull request: https://github.com/apache/spark/pull/19194#discussion_r139156014 --- Diff: core/src/test/scala/org/apache/spark/ExecutorAllocationManagerSuite.scala --- @@ -210,22 +216,282 @@ class ExecutorAllocationManagerSuite

[GitHub] spark pull request #19194: [SPARK-20589] Allow limiting task concurrency per...

2017-09-15 Thread tgravescs
Github user tgravescs commented on a diff in the pull request: https://github.com/apache/spark/pull/19194#discussion_r139155530 --- Diff: core/src/test/scala/org/apache/spark/ExecutorAllocationManagerSuite.scala --- @@ -210,22 +216,282 @@ class ExecutorAllocationManagerSuite

[GitHub] spark pull request #19194: [SPARK-20589] Allow limiting task concurrency per...

2017-09-15 Thread tgravescs
Github user tgravescs commented on a diff in the pull request: https://github.com/apache/spark/pull/19194#discussion_r139149660 --- Diff: core/src/test/scala/org/apache/spark/ExecutorAllocationManagerSuite.scala --- @@ -210,22 +216,282 @@ class ExecutorAllocationManagerSuite

[GitHub] spark pull request #19194: [SPARK-20589] Allow limiting task concurrency per...

2017-09-14 Thread tgravescs
Github user tgravescs commented on a diff in the pull request: https://github.com/apache/spark/pull/19194#discussion_r138993979 --- Diff: core/src/main/scala/org/apache/spark/ExecutorAllocationManager.scala --- @@ -758,11 +812,58 @@ private[spark] class ExecutorAllocationManager(

[GitHub] spark pull request #19194: [SPARK-20589] Allow limiting task concurrency per...

2017-09-14 Thread tgravescs
Github user tgravescs commented on a diff in the pull request: https://github.com/apache/spark/pull/19194#discussion_r138987948 --- Diff: core/src/main/scala/org/apache/spark/ExecutorAllocationManager.scala --- @@ -758,11 +812,58 @@ private[spark] class ExecutorAllocationManager(

[GitHub] spark pull request #19194: [SPARK-20589] Allow limiting task concurrency per...

2017-09-14 Thread tgravescs
Github user tgravescs commented on a diff in the pull request: https://github.com/apache/spark/pull/19194#discussion_r138980299 --- Diff: core/src/main/scala/org/apache/spark/ExecutorAllocationManager.scala --- @@ -758,11 +812,58 @@ private[spark] class ExecutorAllocationManager(

[GitHub] spark pull request #19194: [SPARK-20589] Allow limiting task concurrency per...

2017-09-11 Thread dhruve
GitHub user dhruve opened a pull request: https://github.com/apache/spark/pull/19194 [SPARK-20589] Allow limiting task concurrency per stage ## What changes were proposed in this pull request? This change allows the user to specify the maximum no. of tasks running in a given