[GitHub] spark pull request: [SPARK-10901] [YARN] spark.yarn.user.classpath...

2015-10-06 Thread vanzin
Github user vanzin commented on the pull request: https://github.com/apache/spark/pull/8959#issuecomment-145932373 Merging this, master and branch-1.5. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does

[GitHub] spark pull request: [SPARK-10901] [YARN] spark.yarn.user.classpath...

2015-10-06 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/8959 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark pull request: [SPARK-10901] [YARN] spark.yarn.user.classpath...

2015-10-05 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/8959#discussion_r41173657 --- Diff: yarn/src/main/scala/org/apache/spark/deploy/yarn/Client.scala --- @@ -1183,11 +1194,22 @@ object Client extends Logging { private def

[GitHub] spark pull request: [SPARK-10901] [YARN] spark.yarn.user.classpath...

2015-10-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8959#issuecomment-145609327 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-10901] [YARN] spark.yarn.user.classpath...

2015-10-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8959#issuecomment-145606571 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-10901] [YARN] spark.yarn.user.classpath...

2015-10-05 Thread tgravescs
Github user tgravescs commented on the pull request: https://github.com/apache/spark/pull/8959#issuecomment-145619279 sorry typo, running scalastyle manually and will post updated patch shortly. --- If your project is set up for it, you can reply to this email and have your reply

[GitHub] spark pull request: [SPARK-10901] [YARN] spark.yarn.user.classpath...

2015-10-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8959#issuecomment-145620685 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-10901] [YARN] spark.yarn.user.classpath...

2015-10-05 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/8959#issuecomment-145620682 [Test build #43245 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/43245/console) for PR 8959 at commit

[GitHub] spark pull request: [SPARK-10901] [YARN] spark.yarn.user.classpath...

2015-10-05 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/8959#issuecomment-145624942 [Test build #43246 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/43246/consoleFull) for PR 8959 at commit

[GitHub] spark pull request: [SPARK-10901] [YARN] spark.yarn.user.classpath...

2015-10-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8959#issuecomment-145606547 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-10901] [YARN] spark.yarn.user.classpath...

2015-10-05 Thread vanzin
Github user vanzin commented on the pull request: https://github.com/apache/spark/pull/8959#issuecomment-145609903 LGTM pending tests and a minor cleanup. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project

[GitHub] spark pull request: [SPARK-10901] [YARN] spark.yarn.user.classpath...

2015-10-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8959#issuecomment-145619563 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-10901] [YARN] spark.yarn.user.classpath...

2015-10-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8959#issuecomment-145619542 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-10901] [YARN] spark.yarn.user.classpath...

2015-10-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8959#issuecomment-145620687 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-10901] [YARN] spark.yarn.user.classpath...

2015-10-05 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/8959#issuecomment-145609320 [Test build #43242 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/43242/console) for PR 8959 at commit

[GitHub] spark pull request: [SPARK-10901] [YARN] spark.yarn.user.classpath...

2015-10-05 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/8959#discussion_r41173518 --- Diff: yarn/src/main/scala/org/apache/spark/deploy/yarn/Client.scala --- @@ -1183,11 +1194,22 @@ object Client extends Logging { private def

[GitHub] spark pull request: [SPARK-10901] [YARN] spark.yarn.user.classpath...

2015-10-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8959#issuecomment-145609329 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-10901] [YARN] spark.yarn.user.classpath...

2015-10-05 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/8959#issuecomment-145620136 [Test build #43245 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/43245/consoleFull) for PR 8959 at commit

[GitHub] spark pull request: [SPARK-10901] [YARN] spark.yarn.user.classpath...

2015-10-05 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/8959#issuecomment-145608874 [Test build #43242 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/43242/consoleFull) for PR 8959 at commit

[GitHub] spark pull request: [SPARK-10901] [YARN] spark.yarn.user.classpath...

2015-10-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8959#issuecomment-145623250 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-10901] [YARN] spark.yarn.user.classpath...

2015-10-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8959#issuecomment-145623218 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-10901] [YARN] spark.yarn.user.classpath...

2015-10-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8959#issuecomment-145671670 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-10901] [YARN] spark.yarn.user.classpath...

2015-10-05 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/8959#issuecomment-145671588 [Test build #43246 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/43246/console) for PR 8959 at commit

[GitHub] spark pull request: [SPARK-10901] [YARN] spark.yarn.user.classpath...

2015-10-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8959#issuecomment-145671673 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-10901] [YARN] spark.yarn.user.classpath...

2015-10-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8959#issuecomment-145674662 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-10901] [YARN] spark.yarn.user.classpath...

2015-10-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8959#issuecomment-145674686 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-10901] [YARN] spark.yarn.user.classpath...

2015-10-05 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/8959#issuecomment-145676066 [Test build #43251 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/43251/consoleFull) for PR 8959 at commit

[GitHub] spark pull request: [SPARK-10901] [YARN] spark.yarn.user.classpath...

2015-10-05 Thread tgravescs
Github user tgravescs commented on the pull request: https://github.com/apache/spark/pull/8959#issuecomment-145673897 Jenkins, test this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-10901] [YARN] spark.yarn.user.classpath...

2015-10-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8959#issuecomment-145706494 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-10901] [YARN] spark.yarn.user.classpath...

2015-10-05 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/8959#issuecomment-145706396 [Test build #43251 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/43251/console) for PR 8959 at commit

[GitHub] spark pull request: [SPARK-10901] [YARN] spark.yarn.user.classpath...

2015-10-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8959#issuecomment-145706493 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-10901] [YARN] spark.yarn.user.classpath...

2015-10-02 Thread vanzin
Github user vanzin commented on the pull request: https://github.com/apache/spark/pull/8959#issuecomment-145125393 Hmm, I think I see what you mean. Maybe there should be separate handling for app jar vs. other jars when `spark.yarn.user.classpath.first` is set. May this code

[GitHub] spark pull request: [SPARK-10901] [YARN] spark.yarn.user.classpath...

2015-10-02 Thread tgravescs
Github user tgravescs commented on the pull request: https://github.com/apache/spark/pull/8959#issuecomment-145123502 At first glance that sounds good, but on further looking it doesn't work because of the way addFileToClasspath is called. It passes them as the uri and not filename.

[GitHub] spark pull request: [SPARK-10901] [YARN] spark.yarn.user.classpath...

2015-10-01 Thread vanzin
Github user vanzin commented on the pull request: https://github.com/apache/spark/pull/8959#issuecomment-144818093 Hi @tgravescs , This fixes the problem but I think it's actually just masking a subtle bug elsewhere. In `getUserClasspath`, there's this code: val

[GitHub] spark pull request: [SPARK-10901] [YARN] spark.yarn.user.classpath...

2015-10-01 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8959#issuecomment-144810614 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-10901] [YARN] spark.yarn.user.classpath...

2015-10-01 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8959#issuecomment-144810576 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-10901] [YARN] spark.yarn.user.classpath...

2015-10-01 Thread tgravescs
Github user tgravescs commented on the pull request: https://github.com/apache/spark/pull/8959#issuecomment-144821230 where are you suggesting putting this .map so its clear? in getUserPath on mainUri? I'm not seeing how your map call fixes anything so I"m guessing I'm missing

[GitHub] spark pull request: [SPARK-10901] [YARN] spark.yarn.user.classpath...

2015-10-01 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/8959#issuecomment-144812185 [Test build #43157 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/43157/consoleFull) for PR 8959 at commit

[GitHub] spark pull request: [SPARK-10901] [YARN] spark.yarn.user.classpath...

2015-10-01 Thread tgravescs
GitHub user tgravescs opened a pull request: https://github.com/apache/spark/pull/8959 [SPARK-10901] [YARN] spark.yarn.user.classpath.first doesn't work This should go into 1.5.2 also. The issue is we were no longer adding the __app__.jar to the system classpath. You can

[GitHub] spark pull request: [SPARK-10901] [YARN] spark.yarn.user.classpath...

2015-10-01 Thread vanzin
Github user vanzin commented on the pull request: https://github.com/apache/spark/pull/8959#issuecomment-144825265 Sorry, my suggestion wasn't very clear and maybe not completely correct. Here's the possible values for `mainJar` when that method is called: -

[GitHub] spark pull request: [SPARK-10901] [YARN] spark.yarn.user.classpath...

2015-10-01 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/8959#issuecomment-144852722 [Test build #43157 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/43157/console) for PR 8959 at commit

[GitHub] spark pull request: [SPARK-10901] [YARN] spark.yarn.user.classpath...

2015-10-01 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8959#issuecomment-144852880 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-10901] [YARN] spark.yarn.user.classpath...

2015-10-01 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8959#issuecomment-144852878 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your