[GitHub] spark pull request: SPARK-1588 Re-add support for SPARK_YARN_USER_...
Github user mridulm closed the pull request at: https://github.com/apache/spark/pull/514 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] spark pull request: SPARK-1588 Re-add support for SPARK_YARN_USER_...
Github user mridulm commented on the pull request: https://github.com/apache/spark/pull/514#issuecomment-42525085 Closing this assuming it has been resolved through other PR's. Please let me know in case this is not the case. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] spark pull request: SPARK-1588 Re-add support for SPARK_YARN_USER_...
Github user pwendell commented on the pull request: https://github.com/apache/spark/pull/514#issuecomment-41361434 Hey @mridulm thanks for reporting this - I need to go and audit the spark-submit changes and make sure we properly backwards support all these options on YARN. I'm going to look at it this weekend. I think the issue with the test is currently unknown... however might be worth digging into --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] spark pull request: SPARK-1588 Re-add support for SPARK_YARN_USER_...
Github user pwendell commented on the pull request: https://github.com/apache/spark/pull/514#issuecomment-41361887 I made this JIRA for it. https://issues.apache.org/jira/browse/SPARK-1625 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] spark pull request: SPARK-1588 Re-add support for SPARK_YARN_USER_...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/514#issuecomment-41362725 Merged build finished. All automated tests passed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] spark pull request: SPARK-1588 Re-add support for SPARK_YARN_USER_...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/514#issuecomment-41362727 All automated tests passed. Refer to this link for build results: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/14479/ --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] spark pull request: SPARK-1588 Re-add support for SPARK_YARN_USER_...
Github user mridulm commented on the pull request: https://github.com/apache/spark/pull/514#issuecomment-41363418 Thanks patrick - looks like the tests passed now ! This was just a quick and dirty change based on what I did to unblock our immediate jobs - so might be suboptimal. Unfortunately, I am slightly swamped - so cant clean it up. Please do feel free to use this and make appropriate changes. Note that we used to propagate all SPARK_* env variables from Client to app master and so on - which got removed; which caused part of the problem (I selectively re-added only for these two variables, but it is possible other variables might be broken too ?). Additionally, ofcourse, dropping setting of the env variable entirely. I have not followed up the discussions on why this happened and for what reasons either ... --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] spark pull request: SPARK-1588 Re-add support for SPARK_YARN_USER_...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/514#issuecomment-41361028 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] spark pull request: SPARK-1588 Re-add support for SPARK_YARN_USER_...
GitHub user mridulm opened a pull request: https://github.com/apache/spark/pull/514 SPARK-1588 Re-add support for SPARK_YARN_USER_ENV and SPARK_JAVA_OPTS This is what I did to unblock our jobs - please feel free to modify or create a new PR based on this in case this wont suffice. Unfortunately, I wont be able to shepard this patch actively ... You can merge this pull request into a Git repository by running: $ git pull https://github.com/mridulm/spark yarn_fixes Alternatively you can review and apply these changes as the patch at: https://github.com/apache/spark/pull/514.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #514 commit 53e1a10d77c9066053470dd406d525ec167c6768 Author: Mridul Muralidharan mridul...@apache.org Date: 2014-04-23T20:09:53Z Re-add support for SPARK_YARN_USER_ENV and SPARK_JAVA_OPTS --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] spark pull request: SPARK-1588 Re-add support for SPARK_YARN_USER_...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/514#issuecomment-41209324 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] spark pull request: SPARK-1588 Re-add support for SPARK_YARN_USER_...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/514#issuecomment-41209345 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] spark pull request: SPARK-1588 Re-add support for SPARK_YARN_USER_...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/514#issuecomment-41213472 Refer to this link for build results: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/14391/ --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] spark pull request: SPARK-1588 Re-add support for SPARK_YARN_USER_...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/514#issuecomment-41213471 Merged build finished. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] spark pull request: SPARK-1588 Re-add support for SPARK_YARN_USER_...
Github user mridulm commented on the pull request: https://github.com/apache/spark/pull/514#issuecomment-41214164 Jenkins, retest this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] spark pull request: SPARK-1588 Re-add support for SPARK_YARN_USER_...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/514#issuecomment-41214422 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] spark pull request: SPARK-1588 Re-add support for SPARK_YARN_USER_...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/514#issuecomment-41214430 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] spark pull request: SPARK-1588 Re-add support for SPARK_YARN_USER_...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/514#issuecomment-41218912 Merged build finished. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] spark pull request: SPARK-1588 Re-add support for SPARK_YARN_USER_...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/514#issuecomment-41218916 Refer to this link for build results: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/14397/ --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] spark pull request: SPARK-1588 Re-add support for SPARK_YARN_USER_...
Github user mridulm commented on the pull request: https://github.com/apache/spark/pull/514#issuecomment-41219973 again unrelated failures imo ... CC @pwendell ... any idea why these tests keep failing ? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] spark pull request: SPARK-1588 Re-add support for SPARK_YARN_USER_...
Github user mridulm commented on the pull request: https://github.com/apache/spark/pull/514#issuecomment-41244177 ah ok, that sucks ... --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
