[GitHub] spark pull request: [SPARK-9782] [YARN] Support YARN application t...

2015-12-03 Thread shijinkui
Github user shijinkui commented on a diff in the pull request: https://github.com/apache/spark/pull/8072#discussion_r46532508 --- Diff: yarn/src/main/scala/org/apache/spark/deploy/yarn/Client.scala --- @@ -163,6 +163,23 @@ private[spark] class Client(

[GitHub] spark pull request: [SPARK-9782] [YARN] Support YARN application t...

2015-08-18 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/8072#discussion_r37324265 --- Diff: yarn/src/main/scala/org/apache/spark/deploy/yarn/Client.scala --- @@ -163,6 +163,23 @@ private[spark] class Client(

[GitHub] spark pull request: [SPARK-9782] [YARN] Support YARN application t...

2015-08-18 Thread dennishuo
Github user dennishuo commented on a diff in the pull request: https://github.com/apache/spark/pull/8072#discussion_r37323794 --- Diff: yarn/src/main/scala/org/apache/spark/deploy/yarn/Client.scala --- @@ -163,6 +163,23 @@ private[spark] class Client(

[GitHub] spark pull request: [SPARK-9782] [YARN] Support YARN application t...

2015-08-18 Thread dennishuo
Github user dennishuo commented on the pull request: https://github.com/apache/spark/pull/8072#issuecomment-132321367 Thanks everyone for the reviews! @sryza it sounds like after further clarification with @vanzin we're okay with the code as-is. Is there anything else I should look

[GitHub] spark pull request: [SPARK-9782] [YARN] Support YARN application t...

2015-08-18 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/8072 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark pull request: [SPARK-9782] [YARN] Support YARN application t...

2015-08-18 Thread tgravescs
Github user tgravescs commented on the pull request: https://github.com/apache/spark/pull/8072#issuecomment-132217285 thats fine. we can keep it tags and if another config with tags in it comes along be more specific on that. --- If your project is set up for it, you can reply to

[GitHub] spark pull request: [SPARK-9782] [YARN] Support YARN application t...

2015-08-18 Thread sryza
Github user sryza commented on the pull request: https://github.com/apache/spark/pull/8072#issuecomment-132214961 @tgravescs my thinking for just spark.yarn.tags was that it's redundant to include application, because configs are by definition per-application. We'd also be

[GitHub] spark pull request: [SPARK-9782] [YARN] Support YARN application t...

2015-08-18 Thread sryza
Github user sryza commented on the pull request: https://github.com/apache/spark/pull/8072#issuecomment-132218054 Cool, in that case @dennishuo mind making the change that @vanzin suggested and then I'll merge this? --- If your project is set up for it, you can reply to this email

[GitHub] spark pull request: [SPARK-9782] [YARN] Support YARN application t...

2015-08-18 Thread tgravescs
Github user tgravescs commented on the pull request: https://github.com/apache/spark/pull/8072#issuecomment-132213070 I think the name is ok, but it might be better if we have put application in it: spark.yarn.application.tags. thoughts? My thinking is that way if in the future

[GitHub] spark pull request: [SPARK-9782] [YARN] Support YARN application t...

2015-08-18 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/8072#discussion_r37316986 --- Diff: yarn/src/main/scala/org/apache/spark/deploy/yarn/Client.scala --- @@ -163,6 +163,23 @@ private[spark] class Client(

[GitHub] spark pull request: [SPARK-9782] [YARN] Support YARN application t...

2015-08-17 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/8072#discussion_r37214306 --- Diff: yarn/src/main/scala/org/apache/spark/deploy/yarn/Client.scala --- @@ -163,6 +163,23 @@ private[spark] class Client(

[GitHub] spark pull request: [SPARK-9782] [YARN] Support YARN application t...

2015-08-17 Thread vanzin
Github user vanzin commented on the pull request: https://github.com/apache/spark/pull/8072#issuecomment-131904066 Config name LGTM. Code too, just left a minor suggestion. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well.

[GitHub] spark pull request: [SPARK-9782] [YARN] Support YARN application t...

2015-08-16 Thread sryza
Github user sryza commented on the pull request: https://github.com/apache/spark/pull/8072#issuecomment-131679667 @tgravescs @vanzin @andrewor14 can I get one of you to sign off on my proposed property name `spark.yarn.tags`? Otherwise, this LGTM --- If your project is set

[GitHub] spark pull request: [SPARK-9782] [YARN] Support YARN application t...

2015-08-12 Thread sryza
Github user sryza commented on the pull request: https://github.com/apache/spark/pull/8072#issuecomment-130196650 jenkins, retest this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [SPARK-9782] [YARN] Support YARN application t...

2015-08-12 Thread sryza
Github user sryza commented on a diff in the pull request: https://github.com/apache/spark/pull/8072#discussion_r36833771 --- Diff: yarn/src/main/scala/org/apache/spark/deploy/yarn/Client.scala --- @@ -163,6 +163,23 @@ private[spark] class Client(

[GitHub] spark pull request: [SPARK-9782] [YARN] Support YARN application t...

2015-08-12 Thread dennishuo
Github user dennishuo commented on a diff in the pull request: https://github.com/apache/spark/pull/8072#discussion_r36892350 --- Diff: yarn/src/main/scala/org/apache/spark/deploy/yarn/Client.scala --- @@ -163,6 +163,23 @@ private[spark] class Client(

[GitHub] spark pull request: [SPARK-9782] [YARN] Support YARN application t...

2015-08-11 Thread dennishuo
Github user dennishuo commented on a diff in the pull request: https://github.com/apache/spark/pull/8072#discussion_r36814288 --- Diff: yarn/src/test/scala/org/apache/spark/deploy/yarn/ClientSuite.scala --- @@ -170,6 +173,39 @@ class ClientSuite extends SparkFunSuite with Matchers

[GitHub] spark pull request: [SPARK-9782] [YARN] Support YARN application t...

2015-08-11 Thread dennishuo
Github user dennishuo commented on a diff in the pull request: https://github.com/apache/spark/pull/8072#discussion_r36814036 --- Diff: yarn/src/test/scala/org/apache/spark/deploy/yarn/ClientSuite.scala --- @@ -170,6 +173,39 @@ class ClientSuite extends SparkFunSuite with Matchers

[GitHub] spark pull request: [SPARK-9782] [YARN] Support YARN application t...

2015-08-10 Thread dennishuo
GitHub user dennishuo opened a pull request: https://github.com/apache/spark/pull/8072 [SPARK-9782] [YARN] Support YARN application tags via SparkConf Add a new test case in yarn/ClientSuite which checks how the various SparkConf and ClientArguments propagate into the

[GitHub] spark pull request: [SPARK-9782] [YARN] Support YARN application t...

2015-08-10 Thread tgravescs
Github user tgravescs commented on the pull request: https://github.com/apache/spark/pull/8072#issuecomment-129545012 Jenkins, test this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-9782] [YARN] Support YARN application t...

2015-08-10 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8072#issuecomment-129546684 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-9782] [YARN] Support YARN application t...

2015-08-10 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8072#issuecomment-129545474 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-9782] [YARN] Support YARN application t...

2015-08-10 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8072#issuecomment-129545498 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-9782] [YARN] Support YARN application t...

2015-08-10 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8072#issuecomment-129543940 Can one of the admins verify this patch? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-9782] [YARN] Support YARN application t...

2015-08-10 Thread sryza
Github user sryza commented on a diff in the pull request: https://github.com/apache/spark/pull/8072#discussion_r36708697 --- Diff: yarn/src/main/scala/org/apache/spark/deploy/yarn/Client.scala --- @@ -163,6 +163,23 @@ private[spark] class Client(

[GitHub] spark pull request: [SPARK-9782] [YARN] Support YARN application t...

2015-08-10 Thread sryza
Github user sryza commented on a diff in the pull request: https://github.com/apache/spark/pull/8072#discussion_r36708684 --- Diff: docs/running-on-yarn.md --- @@ -320,6 +320,14 @@ If you need a reference to the proper location to put log files in the YARN so t /td /tr

[GitHub] spark pull request: [SPARK-9782] [YARN] Support YARN application t...

2015-08-10 Thread sryza
Github user sryza commented on a diff in the pull request: https://github.com/apache/spark/pull/8072#discussion_r36708654 --- Diff: yarn/src/main/scala/org/apache/spark/deploy/yarn/Client.scala --- @@ -163,6 +163,23 @@ private[spark] class Client(

[GitHub] spark pull request: [SPARK-9782] [YARN] Support YARN application t...

2015-08-10 Thread sryza
Github user sryza commented on a diff in the pull request: https://github.com/apache/spark/pull/8072#discussion_r36708634 --- Diff: docs/running-on-yarn.md --- @@ -320,6 +320,14 @@ If you need a reference to the proper location to put log files in the YARN so t /td /tr

[GitHub] spark pull request: [SPARK-9782] [YARN] Support YARN application t...

2015-08-10 Thread sryza
Github user sryza commented on a diff in the pull request: https://github.com/apache/spark/pull/8072#discussion_r36708965 --- Diff: yarn/src/main/scala/org/apache/spark/deploy/yarn/Client.scala --- @@ -163,6 +163,23 @@ private[spark] class Client(

[GitHub] spark pull request: [SPARK-9782] [YARN] Support YARN application t...

2015-08-10 Thread sryza
Github user sryza commented on a diff in the pull request: https://github.com/apache/spark/pull/8072#discussion_r36709001 --- Diff: yarn/src/main/scala/org/apache/spark/deploy/yarn/Client.scala --- @@ -163,6 +163,23 @@ private[spark] class Client(

[GitHub] spark pull request: [SPARK-9782] [YARN] Support YARN application t...

2015-08-10 Thread sryza
Github user sryza commented on a diff in the pull request: https://github.com/apache/spark/pull/8072#discussion_r36709027 --- Diff: yarn/src/test/scala/org/apache/spark/deploy/yarn/ClientSuite.scala --- @@ -170,6 +173,39 @@ class ClientSuite extends SparkFunSuite with Matchers

[GitHub] spark pull request: [SPARK-9782] [YARN] Support YARN application t...

2015-08-10 Thread dennishuo
Github user dennishuo commented on a diff in the pull request: https://github.com/apache/spark/pull/8072#discussion_r36709328 --- Diff: yarn/src/main/scala/org/apache/spark/deploy/yarn/Client.scala --- @@ -163,6 +163,23 @@ private[spark] class Client(

[GitHub] spark pull request: [SPARK-9782] [YARN] Support YARN application t...

2015-08-10 Thread dennishuo
Github user dennishuo commented on a diff in the pull request: https://github.com/apache/spark/pull/8072#discussion_r36709305 --- Diff: docs/running-on-yarn.md --- @@ -320,6 +320,14 @@ If you need a reference to the proper location to put log files in the YARN so t /td

[GitHub] spark pull request: [SPARK-9782] [YARN] Support YARN application t...

2015-08-10 Thread dennishuo
Github user dennishuo commented on a diff in the pull request: https://github.com/apache/spark/pull/8072#discussion_r36709312 --- Diff: yarn/src/main/scala/org/apache/spark/deploy/yarn/Client.scala --- @@ -163,6 +163,23 @@ private[spark] class Client(

[GitHub] spark pull request: [SPARK-9782] [YARN] Support YARN application t...

2015-08-10 Thread dennishuo
Github user dennishuo commented on a diff in the pull request: https://github.com/apache/spark/pull/8072#discussion_r36709327 --- Diff: yarn/src/main/scala/org/apache/spark/deploy/yarn/Client.scala --- @@ -163,6 +163,23 @@ private[spark] class Client(

[GitHub] spark pull request: [SPARK-9782] [YARN] Support YARN application t...

2015-08-10 Thread dennishuo
Github user dennishuo commented on a diff in the pull request: https://github.com/apache/spark/pull/8072#discussion_r36709324 --- Diff: docs/running-on-yarn.md --- @@ -320,6 +320,14 @@ If you need a reference to the proper location to put log files in the YARN so t /td

[GitHub] spark pull request: [SPARK-9782] [YARN] Support YARN application t...

2015-08-10 Thread dennishuo
Github user dennishuo commented on a diff in the pull request: https://github.com/apache/spark/pull/8072#discussion_r36709395 --- Diff: yarn/src/test/scala/org/apache/spark/deploy/yarn/ClientSuite.scala --- @@ -170,6 +173,39 @@ class ClientSuite extends SparkFunSuite with Matchers

[GitHub] spark pull request: [SPARK-9782] [YARN] Support YARN application t...

2015-08-10 Thread dennishuo
Github user dennishuo commented on a diff in the pull request: https://github.com/apache/spark/pull/8072#discussion_r36709333 --- Diff: yarn/src/main/scala/org/apache/spark/deploy/yarn/Client.scala --- @@ -163,6 +163,23 @@ private[spark] class Client(