[GitHub] spark pull request: [SPARK-8297] [YARN] Scheduler backend is not n...

2015-07-30 Thread vanzin
Github user vanzin commented on the pull request: https://github.com/apache/spark/pull/7431#issuecomment-126413387 Merged to master. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark pull request: [SPARK-8297] [YARN] Scheduler backend is not n...

2015-07-30 Thread vanzin
Github user vanzin closed the pull request at: https://github.com/apache/spark/pull/7431 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark pull request: [SPARK-8297] [YARN] Scheduler backend is not n...

2015-07-29 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/7431#issuecomment-126117367 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-8297] [YARN] Scheduler backend is not n...

2015-07-29 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/7431#issuecomment-126117395 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-8297] [YARN] Scheduler backend is not n...

2015-07-29 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/7431#issuecomment-126118115 [Test build #38900 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/38900/consoleFull) for PR 7431 at commit

[GitHub] spark pull request: [SPARK-8297] [YARN] Scheduler backend is not n...

2015-07-29 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/7431#issuecomment-126143104 [Test build #38900 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/38900/console) for PR 7431 at commit

[GitHub] spark pull request: [SPARK-8297] [YARN] Scheduler backend is not n...

2015-07-29 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/7431#issuecomment-126143167 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-8297] [YARN] Scheduler backend is not n...

2015-07-28 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/7431#issuecomment-125772365 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-8297] [YARN] Scheduler backend is not n...

2015-07-28 Thread vanzin
Github user vanzin commented on the pull request: https://github.com/apache/spark/pull/7431#issuecomment-125772304 Jenkins retest this please. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [SPARK-8297] [YARN] Scheduler backend is not n...

2015-07-28 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/7431#issuecomment-125772389 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-8297] [YARN] Scheduler backend is not n...

2015-07-28 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/7431#issuecomment-125815206 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-8297] [YARN] Scheduler backend is not n...

2015-07-28 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/7431#issuecomment-125815258 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-8297] [YARN] Scheduler backend is not n...

2015-07-28 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/7431#issuecomment-125816335 [Test build #38794 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/38794/consoleFull) for PR 7431 at commit

[GitHub] spark pull request: [SPARK-8297] [YARN] Scheduler backend is not n...

2015-07-28 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/7431#issuecomment-125790635 [Test build #142 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SlowSparkPullRequestBuilder/142/console) for PR 7431 at commit

[GitHub] spark pull request: [SPARK-8297] [YARN] Scheduler backend is not n...

2015-07-28 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/7431#issuecomment-125790662 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-8297] [YARN] Scheduler backend is not n...

2015-07-28 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/7431#issuecomment-125773212 [Test build #142 has started](https://amplab.cs.berkeley.edu/jenkins/job/SlowSparkPullRequestBuilder/142/consoleFull) for PR 7431 at commit

[GitHub] spark pull request: [SPARK-8297] [YARN] Scheduler backend is not n...

2015-07-28 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/7431#issuecomment-125790368 [Test build #38772 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/38772/console) for PR 7431 at commit

[GitHub] spark pull request: [SPARK-8297] [YARN] Scheduler backend is not n...

2015-07-28 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/7431#issuecomment-125790390 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-8297] [YARN] Scheduler backend is not n...

2015-07-28 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/7431#issuecomment-125835441 [Test build #38794 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/38794/console) for PR 7431 at commit

[GitHub] spark pull request: [SPARK-8297] [YARN] Scheduler backend is not n...

2015-07-28 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/7431#issuecomment-125835460 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-8297] [YARN] Scheduler backend is not n...

2015-07-28 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/7431#issuecomment-125717223 [Test build #38747 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/38747/consoleFull) for PR 7431 at commit

[GitHub] spark pull request: [SPARK-8297] [YARN] Scheduler backend is not n...

2015-07-28 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/7431#issuecomment-125714866 [Test build #139 has started](https://amplab.cs.berkeley.edu/jenkins/job/SlowSparkPullRequestBuilder/139/consoleFull) for PR 7431 at commit

[GitHub] spark pull request: [SPARK-8297] [YARN] Scheduler backend is not n...

2015-07-28 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/7431#issuecomment-125714681 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-8297] [YARN] Scheduler backend is not n...

2015-07-28 Thread vanzin
Github user vanzin commented on the pull request: https://github.com/apache/spark/pull/7431#issuecomment-125714523 Jenkins retest this please. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [SPARK-8297] [YARN] Scheduler backend is not n...

2015-07-28 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/7431#issuecomment-125714753 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-8297] [YARN] Scheduler backend is not n...

2015-07-28 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/7431#issuecomment-125707780 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-8297] [YARN] Scheduler backend is not n...

2015-07-28 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/7431#issuecomment-125707813 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-8297] [YARN] Scheduler backend is not n...

2015-07-28 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/7431#issuecomment-125708458 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-8297] [YARN] Scheduler backend is not n...

2015-07-28 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/7431#issuecomment-125716194 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-8297] [YARN] Scheduler backend is not n...

2015-07-28 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/7431#issuecomment-125716114 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-8297] [YARN] Scheduler backend is not n...

2015-07-28 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/7431#issuecomment-125772845 [Test build #38772 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/38772/consoleFull) for PR 7431 at commit

[GitHub] spark pull request: [SPARK-8297] [YARN] Scheduler backend is not n...

2015-07-28 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/7431#issuecomment-125773058 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-8297] [YARN] Scheduler backend is not n...

2015-07-28 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/7431#issuecomment-125773089 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-8297] [YARN] Scheduler backend is not n...

2015-07-28 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/7431#issuecomment-125748948 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-8297] [YARN] Scheduler backend is not n...

2015-07-28 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/7431#issuecomment-125748844 [Test build #139 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SlowSparkPullRequestBuilder/139/console) for PR 7431 at commit

[GitHub] spark pull request: [SPARK-8297] [YARN] Scheduler backend is not n...

2015-07-28 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/7431#issuecomment-125750668 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-8297] [YARN] Scheduler backend is not n...

2015-07-28 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/7431#issuecomment-125750602 [Test build #38747 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/38747/console) for PR 7431 at commit

[GitHub] spark pull request: [SPARK-8297] [YARN] Scheduler backend is not n...

2015-07-27 Thread tgravescs
Github user tgravescs commented on the pull request: https://github.com/apache/spark/pull/7431#issuecomment-125199663 the last change was just log level change so this lgtm. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as

[GitHub] spark pull request: [SPARK-8297] [YARN] Scheduler backend is not n...

2015-07-23 Thread vanzin
Github user vanzin commented on the pull request: https://github.com/apache/spark/pull/7431#issuecomment-124136120 Jenkins retest this please. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [SPARK-8297] [YARN] Scheduler backend is not n...

2015-07-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/7431#issuecomment-124136938 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-8297] [YARN] Scheduler backend is not n...

2015-07-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/7431#issuecomment-124136897 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-8297] [YARN] Scheduler backend is not n...

2015-07-23 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/7431#issuecomment-124137016 [Test build #81 has started](https://amplab.cs.berkeley.edu/jenkins/job/SlowSparkPullRequestBuilder/81/consoleFull) for PR 7431 at commit

[GitHub] spark pull request: [SPARK-8297] [YARN] Scheduler backend is not n...

2015-07-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/7431#issuecomment-124137146 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-8297] [YARN] Scheduler backend is not n...

2015-07-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/7431#issuecomment-124137177 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-8297] [YARN] Scheduler backend is not n...

2015-07-23 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/7431#issuecomment-124138163 [Test build #38233 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/38233/consoleFull) for PR 7431 at commit

[GitHub] spark pull request: [SPARK-8297] [YARN] Scheduler backend is not n...

2015-07-23 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/7431#issuecomment-124170637 [Test build #38233 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/38233/console) for PR 7431 at commit

[GitHub] spark pull request: [SPARK-8297] [YARN] Scheduler backend is not n...

2015-07-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/7431#issuecomment-124170680 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-8297] [YARN] Scheduler backend is not n...

2015-07-23 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/7431#issuecomment-124174313 [Test build #81 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SlowSparkPullRequestBuilder/81/console) for PR 7431 at commit

[GitHub] spark pull request: [SPARK-8297] [YARN] Scheduler backend is not n...

2015-07-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/7431#issuecomment-124174492 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-8297] [YARN] Scheduler backend is not n...

2015-07-21 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/7431#issuecomment-123412629 [Test build #44 has started](https://amplab.cs.berkeley.edu/jenkins/job/SlowSparkPullRequestBuilder/44/consoleFull) for PR 7431 at commit

[GitHub] spark pull request: [SPARK-8297] [YARN] Scheduler backend is not n...

2015-07-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/7431#issuecomment-123412562 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-8297] [YARN] Scheduler backend is not n...

2015-07-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/7431#issuecomment-123412591 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-8297] [YARN] Scheduler backend is not n...

2015-07-21 Thread vanzin
Github user vanzin commented on the pull request: https://github.com/apache/spark/pull/7431#issuecomment-123411798 Jenkins retest this please. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [SPARK-8297] [YARN] Scheduler backend is not n...

2015-07-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/7431#issuecomment-123412335 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-8297] [YARN] Scheduler backend is not n...

2015-07-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/7431#issuecomment-123412360 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-8297] [YARN] Scheduler backend is not n...

2015-07-21 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/7431#issuecomment-123412943 [Test build #37967 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/37967/consoleFull) for PR 7431 at commit

[GitHub] spark pull request: [SPARK-8297] [YARN] Scheduler backend is not n...

2015-07-21 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/7431#issuecomment-123437909 [Test build #37967 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/37967/console) for PR 7431 at commit

[GitHub] spark pull request: [SPARK-8297] [YARN] Scheduler backend is not n...

2015-07-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/7431#issuecomment-123438063 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-8297] [YARN] Scheduler backend is not n...

2015-07-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/7431#issuecomment-123439636 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-8297] [YARN] Scheduler backend is not n...

2015-07-21 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/7431#issuecomment-123439577 [Test build #44 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SlowSparkPullRequestBuilder/44/console) for PR 7431 at commit

[GitHub] spark pull request: [SPARK-8297] [YARN] Scheduler backend is not n...

2015-07-20 Thread tgravescs
Github user tgravescs commented on the pull request: https://github.com/apache/spark/pull/7431#issuecomment-122892629 It doesn't necessarily mean executor is in bad state but it could be in a bad state. I've seen very weird things happen on machines before so personally I don't

[GitHub] spark pull request: [SPARK-8297] [YARN] Scheduler backend is not n...

2015-07-20 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/7431#issuecomment-122965976 [Test build #37851 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/37851/consoleFull) for PR 7431 at commit

[GitHub] spark pull request: [SPARK-8297] [YARN] Scheduler backend is not n...

2015-07-20 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/7431#issuecomment-122965691 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-8297] [YARN] Scheduler backend is not n...

2015-07-20 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/7431#issuecomment-122965717 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-8297] [YARN] Scheduler backend is not n...

2015-07-20 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/7431#issuecomment-123003866 [Test build #37851 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/37851/console) for PR 7431 at commit

[GitHub] spark pull request: [SPARK-8297] [YARN] Scheduler backend is not n...

2015-07-20 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/7431#issuecomment-123003974 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-8297] [YARN] Scheduler backend is not n...

2015-07-17 Thread vanzin
Github user vanzin commented on the pull request: https://github.com/apache/spark/pull/7431#issuecomment-122402046 If that executor was somehow in a weird state we would want it removed. Feels to me like Spark's own heartbeat would take care of that failure mode. The NM

[GitHub] spark pull request: [SPARK-8297] [YARN] Scheduler backend is not n...

2015-07-17 Thread tgravescs
Github user tgravescs commented on the pull request: https://github.com/apache/spark/pull/7431#issuecomment-122399579 thats not necessarily true. if a NM goes away for long enough the RM will assume all containers on it are gone too so you would get the notification saying they are

[GitHub] spark pull request: [SPARK-8297] [YARN] Scheduler backend is not n...

2015-07-17 Thread mridulm
Github user mridulm closed the pull request at: https://github.com/apache/spark/pull/7243 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark pull request: [SPARK-8297] [YARN] Scheduler backend is not n...

2015-07-16 Thread vanzin
Github user vanzin commented on the pull request: https://github.com/apache/spark/pull/7431#issuecomment-122034172 I can try it out (I'll also add some logging in my internal build to make sure the code is actually being exercised). --- If your project is set up for it, you can

[GitHub] spark pull request: [SPARK-8297] [YARN] Scheduler backend is not n...

2015-07-16 Thread tgravescs
Github user tgravescs commented on the pull request: https://github.com/apache/spark/pull/7431#issuecomment-121980479 The changes look good. To manually test this can't you just remove/kill a nodemanager that has an executor on it? Perhaps there is a race between when yarn tells us

[GitHub] spark pull request: [SPARK-8297] [YARN] Scheduler backend is not n...

2015-07-16 Thread tgravescs
Github user tgravescs commented on the pull request: https://github.com/apache/spark/pull/7243#issuecomment-122001174 @mridulm can you close this then in favor of #7431 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well.

[GitHub] spark pull request: [SPARK-8297] [YARN] Scheduler backend is not n...

2015-07-16 Thread vanzin
Github user vanzin commented on the pull request: https://github.com/apache/spark/pull/7431#issuecomment-122065006 (BTW to test this, killing the NM is not enough, since that doesn't seem to cause child containers to be killed.) --- If your project is set up for it, you can reply to

[GitHub] spark pull request: [SPARK-8297] [YARN] Scheduler backend is not n...

2015-07-16 Thread vanzin
Github user vanzin commented on the pull request: https://github.com/apache/spark/pull/7431#issuecomment-122062974 So now that I tried the new code path (which works), I'm a little skeptical that sending a message back to the driver is really needed. The driver already removes the

[GitHub] spark pull request: [SPARK-8297] [YARN] Scheduler backend is not n...

2015-07-16 Thread tgravescs
Github user tgravescs commented on the pull request: https://github.com/apache/spark/pull/7431#issuecomment-122150692 https://issues.apache.org/jira/browse/SPARK-8297?focusedCommentId=14583560page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14583560

[GitHub] spark pull request: [SPARK-8297] [YARN] Scheduler backend is not n...

2015-07-16 Thread tgravescs
Github user tgravescs commented on a diff in the pull request: https://github.com/apache/spark/pull/7431#discussion_r34857066 --- Diff: yarn/src/main/scala/org/apache/spark/deploy/yarn/YarnAllocator.scala --- @@ -430,6 +435,18 @@ private[yarn] class YarnAllocator(

[GitHub] spark pull request: [SPARK-8297] [YARN] Scheduler backend is not n...

2015-07-16 Thread vanzin
Github user vanzin commented on the pull request: https://github.com/apache/spark/pull/7431#issuecomment-122156071 I'm guessing there might be situations it doesn't recognize the disconnect right away. That's the only case in which this code might help; but does that really

[GitHub] spark pull request: [SPARK-8297] [YARN] Scheduler backend is not n...

2015-07-16 Thread andrewor14
Github user andrewor14 commented on a diff in the pull request: https://github.com/apache/spark/pull/7431#discussion_r34853110 --- Diff: yarn/src/main/scala/org/apache/spark/deploy/yarn/YarnAllocator.scala --- @@ -430,6 +435,18 @@ private[yarn] class YarnAllocator(

[GitHub] spark pull request: [SPARK-8297] [YARN] Scheduler backend is not n...

2015-07-16 Thread andrewor14
Github user andrewor14 commented on the pull request: https://github.com/apache/spark/pull/7431#issuecomment-122138316 Yeah it seems like a race condition between the two detection code paths. If they are both guaranteed to be called, then we're always going to end up with `ERROR:

[GitHub] spark pull request: [SPARK-8297] [YARN] Scheduler backend is not n...

2015-07-15 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/7431#issuecomment-121779491 [Test build #37412 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/37412/console) for PR 7431 at commit

[GitHub] spark pull request: [SPARK-8297] [YARN] Scheduler backend is not n...

2015-07-15 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/7431#issuecomment-121779794 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-8297] [YARN] Scheduler backend is not n...

2015-07-15 Thread vanzin
Github user vanzin commented on the pull request: https://github.com/apache/spark/pull/7431#issuecomment-121759019 This is an updated version of #7243, which also works in yarn-client mode. /cc @mridulm @tgravescs I ran a bunch of tests on yarn client and cluster

[GitHub] spark pull request: [SPARK-8297] [YARN] Scheduler backend is not n...

2015-07-15 Thread mridulm
Github user mridulm commented on the pull request: https://github.com/apache/spark/pull/7431#issuecomment-121762507 Looks good to me ! If Tom does not come back with objections I am +1 on this. --- If your project is set up for it, you can reply to this email and have your reply

[GitHub] spark pull request: [SPARK-8297] [YARN] Scheduler backend is not n...

2015-07-15 Thread vanzin
GitHub user vanzin opened a pull request: https://github.com/apache/spark/pull/7431 [SPARK-8297] [YARN] Scheduler backend is not notified in case node fails in YARN This change adds code to notify the scheduler backend when a container dies in YARN. You can merge this pull

[GitHub] spark pull request: [SPARK-8297] [YARN] Scheduler backend is not n...

2015-07-15 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/7431#issuecomment-121759734 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-8297] [YARN] Scheduler backend is not n...

2015-07-15 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/7431#issuecomment-121759706 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-8297] [YARN] Scheduler backend is not n...

2015-07-15 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/7431#issuecomment-121760479 [Test build #37412 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/37412/consoleFull) for PR 7431 at commit

[GitHub] spark pull request: [SPARK-8297] [YARN] Scheduler backend is not n...

2015-07-07 Thread vanzin
Github user vanzin commented on the pull request: https://github.com/apache/spark/pull/7243#issuecomment-119271693 What about `yarn-client`? `ApplicationMaster` already has a way to talk to the driver regardless of the deploy mode (see `AMEndpoint`, and the equivalent on the driver

[GitHub] spark pull request: [SPARK-8297] [YARN] Scheduler backend is not n...

2015-07-07 Thread tgravescs
Github user tgravescs commented on the pull request: https://github.com/apache/spark/pull/7243#issuecomment-119271350 nice catch mridul. The logic looks fine other then the note from @vanzin. --- If your project is set up for it, you can reply to this email and have your reply

[GitHub] spark pull request: [SPARK-8297] [YARN] Scheduler backend is not n...

2015-07-07 Thread mridulm
Github user mridulm commented on the pull request: https://github.com/apache/spark/pull/7243#issuecomment-119090177 @tgravescs pls take a look, thanks. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does

[GitHub] spark pull request: [SPARK-8297] [YARN] Scheduler backend is not n...

2015-07-07 Thread mridulm
Github user mridulm commented on the pull request: https://github.com/apache/spark/pull/7243#issuecomment-119088417 The failures (in YarnClusterSuite) are unrelated to this change - and happen with a clean checkout as well. --- If your project is set up for it, you can reply to this

[GitHub] spark pull request: [SPARK-8297] [YARN] Scheduler backend is not n...

2015-07-07 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/7243#discussion_r34062845 --- Diff: yarn/src/main/scala/org/apache/spark/deploy/yarn/ApplicationMaster.scala --- @@ -218,11 +218,12 @@ private[spark] class ApplicationMaster(

[GitHub] spark pull request: [SPARK-8297] [YARN] Scheduler backend is not n...

2015-07-07 Thread vanzin
Github user vanzin commented on the pull request: https://github.com/apache/spark/pull/7243#issuecomment-119359707 The endpoint is for messages from core to yarn if I am not wrong ? If you look at `AMEndpoint`, it has a reference to the driver endpoint, so you should be able

[GitHub] spark pull request: [SPARK-8297] [YARN] Scheduler backend is not n...

2015-07-07 Thread mridulm
Github user mridulm commented on a diff in the pull request: https://github.com/apache/spark/pull/7243#discussion_r34098979 --- Diff: yarn/src/main/scala/org/apache/spark/deploy/yarn/ApplicationMaster.scala --- @@ -218,11 +218,12 @@ private[spark] class ApplicationMaster(

[GitHub] spark pull request: [SPARK-8297] [YARN] Scheduler backend is not n...

2015-07-07 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/7243#issuecomment-119364183 [Test build #36726 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/36726/consoleFull) for PR 7243 at commit

[GitHub] spark pull request: [SPARK-8297] [YARN] Scheduler backend is not n...

2015-07-07 Thread vanzin
Github user vanzin commented on the pull request: https://github.com/apache/spark/pull/7243#issuecomment-119371331 It's ok if you don't want to work on the full fix, I'm just wary of pushing a change to master that only fixes part of the problem, when the change to fix everything

[GitHub] spark pull request: [SPARK-8297] [YARN] Scheduler backend is not n...

2015-07-07 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/7243#discussion_r34097612 --- Diff: yarn/src/main/scala/org/apache/spark/deploy/yarn/ApplicationMaster.scala --- @@ -218,11 +218,12 @@ private[spark] class ApplicationMaster(

[GitHub] spark pull request: [SPARK-8297] [YARN] Scheduler backend is not n...

2015-07-07 Thread mridulm
Github user mridulm commented on the pull request: https://github.com/apache/spark/pull/7243#issuecomment-119359281 @vanzin The endpoint is for messages from core to yarn if I am not wrong ? This would flow the other way - from yarn to core. Or am I missing something ?

[GitHub] spark pull request: [SPARK-8297] [YARN] Scheduler backend is not n...

2015-07-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/7243#issuecomment-119363708 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-8297] [YARN] Scheduler backend is not n...

2015-07-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/7243#issuecomment-119363736 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

  1   2   >