[GitHub] spark pull request: [SPARK-10515] When killing executor, the pendi...

2015-10-16 Thread andrewor14
Github user andrewor14 commented on the pull request: https://github.com/apache/spark/pull/8945#issuecomment-148866912 Forgot to add: merged into master 1.5. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-10515] When killing executor, the pendi...

2015-10-15 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/8945 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark pull request: [SPARK-10515] When killing executor, the pendi...

2015-10-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8945#issuecomment-147625660 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-10515] When killing executor, the pendi...

2015-10-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8945#issuecomment-147625645 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-10515] When killing executor, the pendi...

2015-10-13 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/8945#issuecomment-147626144 [Test build #43628 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/43628/consoleFull) for PR 8945 at commit

[GitHub] spark pull request: [SPARK-10515] When killing executor, the pendi...

2015-10-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8945#issuecomment-147661007 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-10515] When killing executor, the pendi...

2015-10-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8945#issuecomment-147661009 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-10515] When killing executor, the pendi...

2015-10-13 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/8945#issuecomment-147660586 [Test build #43628 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/43628/console) for PR 8945 at commit

[GitHub] spark pull request: [SPARK-10515] When killing executor, the pendi...

2015-10-13 Thread KaiXinXiaoLei
Github user KaiXinXiaoLei commented on a diff in the pull request: https://github.com/apache/spark/pull/8945#discussion_r41947268 --- Diff: core/src/test/scala/org/apache/spark/deploy/StandaloneDynamicAllocationSuite.scala --- @@ -369,6 +369,38 @@ class

[GitHub] spark pull request: [SPARK-10515] When killing executor, the pendi...

2015-10-13 Thread vanzin
Github user vanzin commented on the pull request: https://github.com/apache/spark/pull/8945#issuecomment-147900244 LGTM, will let Andrew have a final look. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project

[GitHub] spark pull request: [SPARK-10515] When killing executor, the pendi...

2015-10-12 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/8945#discussion_r41775279 --- Diff: core/src/test/scala/org/apache/spark/deploy/StandaloneDynamicAllocationSuite.scala --- @@ -369,6 +369,38 @@ class StandaloneDynamicAllocationSuite

[GitHub] spark pull request: [SPARK-10515] When killing executor, the pendi...

2015-10-12 Thread KaiXinXiaoLei
Github user KaiXinXiaoLei commented on a diff in the pull request: https://github.com/apache/spark/pull/8945#discussion_r41752408 --- Diff: core/src/test/scala/org/apache/spark/deploy/StandaloneDynamicAllocationSuite.scala --- @@ -369,6 +369,38 @@ class

[GitHub] spark pull request: [SPARK-10515] When killing executor, the pendi...

2015-10-12 Thread KaiXinXiaoLei
Github user KaiXinXiaoLei commented on a diff in the pull request: https://github.com/apache/spark/pull/8945#discussion_r41820348 --- Diff: core/src/test/scala/org/apache/spark/deploy/StandaloneDynamicAllocationSuite.scala --- @@ -369,6 +369,38 @@ class

[GitHub] spark pull request: [SPARK-10515] When killing executor, the pendi...

2015-10-12 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/8945#discussion_r41820574 --- Diff: core/src/test/scala/org/apache/spark/deploy/StandaloneDynamicAllocationSuite.scala --- @@ -369,6 +369,38 @@ class StandaloneDynamicAllocationSuite

[GitHub] spark pull request: [SPARK-10515] When killing executor, the pendi...

2015-10-09 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/8945#issuecomment-146807928 [Test build #43464 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/43464/consoleFull) for PR 8945 at commit

[GitHub] spark pull request: [SPARK-10515] When killing executor, the pendi...

2015-10-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8945#issuecomment-146806550 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-10515] When killing executor, the pendi...

2015-10-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8945#issuecomment-146806538 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-10515] When killing executor, the pendi...

2015-10-09 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/8945#discussion_r41674241 --- Diff: core/src/test/scala/org/apache/spark/deploy/StandaloneDynamicAllocationSuite.scala --- @@ -369,6 +369,38 @@ class StandaloneDynamicAllocationSuite

[GitHub] spark pull request: [SPARK-10515] When killing executor, the pendi...

2015-10-09 Thread KaiXinXiaoLei
Github user KaiXinXiaoLei commented on a diff in the pull request: https://github.com/apache/spark/pull/8945#discussion_r41692752 --- Diff: core/src/test/scala/org/apache/spark/deploy/StandaloneDynamicAllocationSuite.scala --- @@ -369,6 +369,38 @@ class

[GitHub] spark pull request: [SPARK-10515] When killing executor, the pendi...

2015-10-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8945#issuecomment-146834200 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-10515] When killing executor, the pendi...

2015-10-09 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/8945#issuecomment-146834103 [Test build #43464 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/43464/console) for PR 8945 at commit

[GitHub] spark pull request: [SPARK-10515] When killing executor, the pendi...

2015-10-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8945#issuecomment-146834201 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-10515] When killing executor, the pendi...

2015-10-09 Thread andrewor14
Github user andrewor14 commented on a diff in the pull request: https://github.com/apache/spark/pull/8945#discussion_r41660042 --- Diff: core/src/test/scala/org/apache/spark/deploy/StandaloneDynamicAllocationSuite.scala --- @@ -369,6 +369,38 @@ class

[GitHub] spark pull request: [SPARK-10515] When killing executor, the pendi...

2015-10-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8945#issuecomment-146436500 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-10515] When killing executor, the pendi...

2015-10-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8945#issuecomment-146436545 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-10515] When killing executor, the pendi...

2015-10-08 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/8945#issuecomment-146440195 [Test build #43389 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/43389/console) for PR 8945 at commit

[GitHub] spark pull request: [SPARK-10515] When killing executor, the pendi...

2015-10-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8945#issuecomment-146440227 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-10515] When killing executor, the pendi...

2015-10-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8945#issuecomment-146440226 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-10515] When killing executor, the pendi...

2015-10-08 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/8945#issuecomment-146437914 [Test build #43389 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/43389/consoleFull) for PR 8945 at commit

[GitHub] spark pull request: [SPARK-10515] When killing executor, the pendi...

2015-10-08 Thread KaiXinXiaoLei
Github user KaiXinXiaoLei commented on the pull request: https://github.com/apache/spark/pull/8945#issuecomment-146446834 jenkins test please. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [SPARK-10515] When killing executor, the pendi...

2015-10-08 Thread andrewor14
Github user andrewor14 commented on the pull request: https://github.com/apache/spark/pull/8945#issuecomment-146730978 retest this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark pull request: [SPARK-10515] When killing executor, the pendi...

2015-10-08 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/8945#issuecomment-146732399 [Test build #43453 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/43453/consoleFull) for PR 8945 at commit

[GitHub] spark pull request: [SPARK-10515] When killing executor, the pendi...

2015-10-08 Thread andrewor14
Github user andrewor14 commented on the pull request: https://github.com/apache/spark/pull/8945#issuecomment-146737544 Thanks LGTM. I'll merge this once tests pass. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-10515] When killing executor, the pendi...

2015-10-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8945#issuecomment-146747357 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-10515] When killing executor, the pendi...

2015-10-08 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/8945#issuecomment-146747096 [Test build #43453 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/43453/console) for PR 8945 at commit

[GitHub] spark pull request: [SPARK-10515] When killing executor, the pendi...

2015-10-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8945#issuecomment-146747359 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-10515] When killing executor, the pendi...

2015-10-08 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/8945#discussion_r41591815 --- Diff: core/src/test/scala/org/apache/spark/deploy/StandaloneDynamicAllocationSuite.scala --- @@ -369,6 +369,35 @@ class StandaloneDynamicAllocationSuite

[GitHub] spark pull request: [SPARK-10515] When killing executor, the pendi...

2015-10-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8945#issuecomment-146731465 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-10515] When killing executor, the pendi...

2015-10-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8945#issuecomment-146731448 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-10515] When killing executor, the pendi...

2015-10-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8945#issuecomment-146754992 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-10515] When killing executor, the pendi...

2015-10-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8945#issuecomment-146754984 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-10515] When killing executor, the pendi...

2015-10-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8945#issuecomment-146755686 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-10515] When killing executor, the pendi...

2015-10-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8945#issuecomment-146755685 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-10515] When killing executor, the pendi...

2015-10-08 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/8945#issuecomment-146755683 [Test build #43458 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/43458/console) for PR 8945 at commit

[GitHub] spark pull request: [SPARK-10515] When killing executor, the pendi...

2015-10-08 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/8945#issuecomment-146755528 [Test build #43458 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/43458/consoleFull) for PR 8945 at commit

[GitHub] spark pull request: [SPARK-10515] When killing executor, the pendi...

2015-10-07 Thread KaiXinXiaoLei
Github user KaiXinXiaoLei commented on the pull request: https://github.com/apache/spark/pull/8668#issuecomment-146389770 see https://github.com/apache/spark/pull/8945 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If

[GitHub] spark pull request: [SPARK-10515] When killing executor, the pendi...

2015-10-07 Thread KaiXinXiaoLei
Github user KaiXinXiaoLei closed the pull request at: https://github.com/apache/spark/pull/8668 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature

[GitHub] spark pull request: [SPARK-10515] When killing executor, the pendi...

2015-10-01 Thread srowen
Github user srowen commented on the pull request: https://github.com/apache/spark/pull/8668#issuecomment-144707186 @KaiXinXiaoLei do you mind closing this PR? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-10515] When killing executor, the pendi...

2015-09-30 Thread andrewor14
Github user andrewor14 commented on a diff in the pull request: https://github.com/apache/spark/pull/8945#discussion_r40827345 --- Diff: core/src/test/scala/org/apache/spark/deploy/StandaloneDynamicAllocationSuite.scala --- @@ -369,6 +369,35 @@ class

[GitHub] spark pull request: [SPARK-10515] When killing executor, the pendi...

2015-09-30 Thread andrewor14
Github user andrewor14 commented on the pull request: https://github.com/apache/spark/pull/8668#issuecomment-144490428 @KaiXinXiaoLei in the future please just have one PR open for the same issue. You can do this by pushing to the same branch instead of creating a whole new one (i.e.

[GitHub] spark pull request: [SPARK-10515] When killing executor, the pendi...

2015-09-30 Thread KaiXinXiaoLei
GitHub user KaiXinXiaoLei opened a pull request: https://github.com/apache/spark/pull/8945 [SPARK-10515] When killing executor, the pending replacement executors should not be lost If the heartbeat receiver kills executors (and new ones are not registered to replace them), the

[GitHub] spark pull request: [SPARK-10515] When killing executor, the pendi...

2015-09-30 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8945#issuecomment-144311524 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-10515] When killing executor, the pendi...

2015-09-30 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8945#issuecomment-144311537 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-10515] When killing executor, the pendi...

2015-09-30 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/8945#issuecomment-144311863 [Test build #43124 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/43124/consoleFull) for PR 8945 at commit

[GitHub] spark pull request: [SPARK-10515] When killing executor, the pendi...

2015-09-30 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/8945#issuecomment-144337201 [Test build #43124 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/43124/console) for PR 8945 at commit

[GitHub] spark pull request: [SPARK-10515] When killing executor, the pendi...

2015-09-30 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8945#issuecomment-144337257 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-10515] When killing executor, the pendi...

2015-09-30 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8945#issuecomment-144337259 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-10515] When killing executor, the pendi...

2015-09-30 Thread KaiXinXiaoLei
Github user KaiXinXiaoLei commented on the pull request: https://github.com/apache/spark/pull/8668#issuecomment-144338392 @andrewor14 I change code according to your suggest. see: https://github.com/apache/spark/pull/8945 --- If your project is set up for it, you can reply to this

[GitHub] spark pull request: [SPARK-10515] When killing executor, the pendi...

2015-09-30 Thread vanzin
Github user vanzin commented on the pull request: https://github.com/apache/spark/pull/8945#issuecomment-144389195 Hmm, test failure looks like it might be related. Also, does this replace #8668? If so, could you close one of them? --- If your project is set up for it, you

[GitHub] spark pull request: [SPARK-10515] When killing executor, the pendi...

2015-09-30 Thread andrewor14
Github user andrewor14 commented on a diff in the pull request: https://github.com/apache/spark/pull/8945#discussion_r40827411 --- Diff: core/src/test/scala/org/apache/spark/deploy/StandaloneDynamicAllocationSuite.scala --- @@ -369,6 +369,35 @@ class

[GitHub] spark pull request: [SPARK-10515] When killing executor, the pendi...

2015-09-29 Thread andrewor14
Github user andrewor14 commented on a diff in the pull request: https://github.com/apache/spark/pull/8668#discussion_r40715442 --- Diff: core/src/main/scala/org/apache/spark/scheduler/cluster/CoarseGrainedSchedulerBackend.scala --- @@ -431,7 +439,10 @@ class

[GitHub] spark pull request: [SPARK-10515] When killing executor, the pendi...

2015-09-29 Thread andrewor14
Github user andrewor14 commented on a diff in the pull request: https://github.com/apache/spark/pull/8668#discussion_r40713413 --- Diff: core/src/test/scala/org/apache/spark/deploy/StandaloneDynamicAllocationSuite.scala --- @@ -303,6 +303,32 @@ class

[GitHub] spark pull request: [SPARK-10515] When killing executor, the pendi...

2015-09-29 Thread andrewor14
Github user andrewor14 commented on a diff in the pull request: https://github.com/apache/spark/pull/8668#discussion_r40715552 --- Diff: core/src/test/scala/org/apache/spark/deploy/StandaloneDynamicAllocationSuite.scala --- @@ -303,6 +303,32 @@ class

[GitHub] spark pull request: [SPARK-10515] When killing executor, the pendi...

2015-09-29 Thread andrewor14
Github user andrewor14 commented on the pull request: https://github.com/apache/spark/pull/8668#issuecomment-144160428 @KaiXinXiaoLei The problem makes sense now and I think this fix is correct, though I suggested a way to make it simpler. Once you address the comments would you mind

[GitHub] spark pull request: [SPARK-10515] When killing executor, the pendi...

2015-09-25 Thread vanzin
Github user vanzin commented on the pull request: https://github.com/apache/spark/pull/8668#issuecomment-143307328 FYI: #8914 makes some changes to these tests to avoid the races I alluded to in my last comment. --- If your project is set up for it, you can reply to this email and

[GitHub] spark pull request: [SPARK-10515] When killing executor, the pendi...

2015-09-23 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/8668#discussion_r40228114 --- Diff: core/src/test/scala/org/apache/spark/deploy/StandaloneDynamicAllocationSuite.scala --- @@ -303,6 +303,32 @@ class StandaloneDynamicAllocationSuite

[GitHub] spark pull request: [SPARK-10515] When killing executor, the pendi...

2015-09-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8668#issuecomment-142504404 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-10515] When killing executor, the pendi...

2015-09-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8668#issuecomment-142504403 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-10515] When killing executor, the pendi...

2015-09-23 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/8668#issuecomment-142504363 [Test build #42881 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/42881/console) for PR 8668 at commit

[GitHub] spark pull request: [SPARK-10515] When killing executor, the pendi...

2015-09-23 Thread KaiXinXiaoLei
Github user KaiXinXiaoLei commented on the pull request: https://github.com/apache/spark/pull/8668#issuecomment-142507056 @vanzin I add a unit test for this problem. Thanks. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as

[GitHub] spark pull request: [SPARK-10515] When killing executor, the pendi...

2015-09-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8668#issuecomment-142363716 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-10515] When killing executor, the pendi...

2015-09-22 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/8668#discussion_r40121163 --- Diff: core/src/main/scala/org/apache/spark/scheduler/cluster/CoarseGrainedSchedulerBackend.scala --- @@ -147,6 +150,10 @@ class

[GitHub] spark pull request: [SPARK-10515] When killing executor, the pendi...

2015-09-22 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/8668#discussion_r40121097 --- Diff: core/src/main/scala/org/apache/spark/scheduler/cluster/CoarseGrainedSchedulerBackend.scala --- @@ -66,6 +66,9 @@ class

[GitHub] spark pull request: [SPARK-10515] When killing executor, the pendi...

2015-09-22 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/8668#issuecomment-142363549 [Test build #42839 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/42839/console) for PR 8668 at commit

[GitHub] spark pull request: [SPARK-10515] When killing executor, the pendi...

2015-09-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8668#issuecomment-142363712 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-10515] When killing executor, the pendi...

2015-09-22 Thread vanzin
Github user vanzin commented on the pull request: https://github.com/apache/spark/pull/8668#issuecomment-142369819 The change LGTM; I'd be more comfortable if there was a unit test for this code, but I tried to craft one and `CoarseGrainedSchedulerBackend` is not very unit-testable

[GitHub] spark pull request: [SPARK-10515] When killing executor, the pendi...

2015-09-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8668#issuecomment-142482195 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-10515] When killing executor, the pendi...

2015-09-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8668#issuecomment-142482194 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-10515] When killing executor, the pendi...

2015-09-22 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/8668#issuecomment-142485675 [Test build #42883 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/42883/consoleFull) for PR 8668 at commit

[GitHub] spark pull request: [SPARK-10515] When killing executor, the pendi...

2015-09-22 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/8668#issuecomment-142490404 [Test build #42883 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/42883/console) for PR 8668 at commit

[GitHub] spark pull request: [SPARK-10515] When killing executor, the pendi...

2015-09-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8668#issuecomment-142490420 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-10515] When killing executor, the pendi...

2015-09-22 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/8668#issuecomment-142481582 [Test build #42881 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/42881/consoleFull) for PR 8668 at commit

[GitHub] spark pull request: [SPARK-10515] When killing executor, the pendi...

2015-09-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8668#issuecomment-142481452 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-10515] When killing executor, the pendi...

2015-09-22 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/8668#issuecomment-142482191 [Test build #42882 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/42882/console) for PR 8668 at commit

[GitHub] spark pull request: [SPARK-10515] When killing executor, the pendi...

2015-09-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8668#issuecomment-142481446 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-10515] When killing executor, the pendi...

2015-09-22 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/8668#issuecomment-142481985 [Test build #42882 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/42882/consoleFull) for PR 8668 at commit

[GitHub] spark pull request: [SPARK-10515] When killing executor, the pendi...

2015-09-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8668#issuecomment-142481939 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-10515] When killing executor, the pendi...

2015-09-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8668#issuecomment-142481925 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-10515] When killing executor, the pendi...

2015-09-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8668#issuecomment-142485209 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-10515] When killing executor, the pendi...

2015-09-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8668#issuecomment-142485169 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-10515] When killing executor, the pendi...

2015-09-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8668#issuecomment-142490419 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-10515] When killing executor, the pendi...

2015-09-22 Thread KaiXinXiaoLei
Github user KaiXinXiaoLei commented on the pull request: https://github.com/apache/spark/pull/8668#issuecomment-142201407 @vanzin Yes, ```addExecutors``` will often send message to AM, but the total number of executors in ```spark-dynamic-executor-allocation``` will be the same with

[GitHub] spark pull request: [SPARK-10515] When killing executor, the pendi...

2015-09-22 Thread KaiXinXiaoLei
Github user KaiXinXiaoLei commented on a diff in the pull request: https://github.com/apache/spark/pull/8668#discussion_r40096873 --- Diff: core/src/main/scala/org/apache/spark/scheduler/cluster/CoarseGrainedSchedulerBackend.scala --- @@ -236,6 +246,12 @@ class

[GitHub] spark pull request: [SPARK-10515] When killing executor, the pendi...

2015-09-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8668#issuecomment-142317605 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-10515] When killing executor, the pendi...

2015-09-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8668#issuecomment-142317529 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-10515] When killing executor, the pendi...

2015-09-22 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/8668#issuecomment-142320061 [Test build #42839 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/42839/consoleFull) for PR 8668 at commit

[GitHub] spark pull request: [SPARK-10515] When killing executor, the pendi...

2015-09-22 Thread KaiXinXiaoLei
Github user KaiXinXiaoLei commented on the pull request: https://github.com/apache/spark/pull/8668#issuecomment-142322746 @vanzin I think the code I changed can resolve the problem about [SPARK-10515]. Thanks --- If your project is set up for it, you can reply to this email and

[GitHub] spark pull request: [SPARK-10515] When killing executor, the pendi...

2015-09-21 Thread vanzin
Github user vanzin commented on the pull request: https://github.com/apache/spark/pull/8668#issuecomment-142073376 There's one possible race in this current code; let's say executor `X` fails a heartbeat and the driver kills it expecting a replacement. Slightly later, the allocation

[GitHub] spark pull request: [SPARK-10515] When killing executor, the pendi...

2015-09-21 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/8668#discussion_r40009333 --- Diff: core/src/main/scala/org/apache/spark/scheduler/cluster/CoarseGrainedSchedulerBackend.scala --- @@ -236,6 +246,12 @@ class

[GitHub] spark pull request: [SPARK-10515] When killing executor, the pendi...

2015-09-21 Thread vanzin
Github user vanzin commented on the pull request: https://github.com/apache/spark/pull/8668#issuecomment-142106259 @KaiXinXiaoLei I think it understood what the change you propose in your last comment would do. I think it would work, except for a very minor issue where it might

  1   2   >