[GitHub] incubator-spark pull request: SPARK-1053. Don't require SPARK_YARN...

2014-02-24 Thread pwendell
Github user pwendell commented on the pull request: https://github.com/apache/incubator-spark/pull/553#issuecomment-35954097 @tgravescs mind taking a look here as well? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If y

[GitHub] incubator-spark pull request: SPARK-1053. Don't require SPARK_YARN...

2014-02-12 Thread sryza
Github user sryza commented on the pull request: https://github.com/apache/incubator-spark/pull/553#issuecomment-34906646 @tgravescs I should have tried this - it looks like it actually works fine when SPARK_YARN_APP_JAR isn't specified. The client must be serving the jar in the typi

[GitHub] incubator-spark pull request: SPARK-1053. Don't require SPARK_YARN...

2014-02-12 Thread tgravescs
Github user tgravescs commented on the pull request: https://github.com/apache/incubator-spark/pull/553#issuecomment-34884178 So what does the user see in the yarn-client batch mode if SPARK_YARN_APP_JAR isn't specified?

[GitHub] incubator-spark pull request: SPARK-1053. Don't require SPARK_YARN...

2014-02-11 Thread colorant
Github user colorant commented on the pull request: https://github.com/apache/incubator-spark/pull/553#issuecomment-34831254 I agree that we do not need a hard check on it :) This patch looks good to me, I just wonder we might need a few more explanation to clarify why this one is not

[GitHub] incubator-spark pull request: SPARK-1053. Don't require SPARK_YARN...

2014-02-11 Thread sryza
Github user sryza commented on the pull request: https://github.com/apache/incubator-spark/pull/553#issuecomment-34830852 It's true that you may need it when running in yarn-client mode. And also true that you will not when running spark-shell. Because it depends, I think not requir

[GitHub] incubator-spark pull request: SPARK-1053. Don't require SPARK_YARN...

2014-02-11 Thread colorant
Github user colorant commented on the pull request: https://github.com/apache/incubator-spark/pull/553#issuecomment-34830448 @sryza I think this depends on what you mean to do with the yarn-client mode, for spark-shell, since it is already packaged in the assembly jar. you don't need