can you be more specific? What breaks existing jobs? If you are referring to my comment, SPARK_JAVA_OPTS still works but I think the intent is to move away from it.
Tom On Wednesday, April 23, 2014 8:07 AM, Mridul Muralidharan <mri...@gmail.com> wrote: This breaks all existing jobs which are not using spark-submit. The consensus was not to break compatibility unless there was an overriding reason to do so On Apr 23, 2014 6:32 PM, "Thomas Graves (JIRA)" <j...@apache.org> wrote: > > [ > https://issues.apache.org/jira/browse/SPARK-1576?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13978164#comment-13978164] > > Thomas Graves commented on SPARK-1576: > -------------------------------------- > > Is this meant for the driver or the executors? The spark-submit script > has a command line option for the driver: --driver-java-options. > I believe the intent of https://github.com/apache/spark/pull/299 was to > not expose SPARK_JAVA_OPTS to the user anymore. > > > Passing of JAVA_OPTS to YARN on command line > > -------------------------------------------- > > > > Key: SPARK-1576 > > URL: https://issues.apache.org/jira/browse/SPARK-1576 > > Project: Spark > > Issue Type: Improvement > > Affects Versions: 0.9.0, 1.0.0, 0.9.1 > > Reporter: Nishkam Ravi > > Fix For: 0.9.0, 1.0.0, 0.9.1 > > > > Attachments: SPARK-1576.patch > > > > > > JAVA_OPTS can be passed by using either env variables (i.e., > SPARK_JAVA_OPTS) or as config vars (after Patrick's recent change). It > would be good to allow the user to pass them on command line as well to > restrict scope to single application invocation. > > > > -- > This message was sent by Atlassian JIRA > (v6.2#6252) >
