[ https://issues.apache.org/jira/browse/SPARK-1350?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14239060#comment-14239060 ]
Aniket Bhatnagar commented on SPARK-1350: ----------------------------------------- I am using hadoop 2.5.0 (CDH). Agreed that it handles for windows. But the use case I am talking about is when SparkContext is created programmatically on a windows machine and is used to submit jobs on a yarn cluster running on Linux. As per above code, %JAVA_HOME%/bin/java will be generated as one of the commands by ClientBase and submitted to YARN cluster. This will obviously fail while YARN tries to execute the container as % is treated differently in linux. > YARN ContainerLaunchContext should use cluster's JAVA_HOME > ---------------------------------------------------------- > > Key: SPARK-1350 > URL: https://issues.apache.org/jira/browse/SPARK-1350 > Project: Spark > Issue Type: Bug > Components: YARN > Affects Versions: 0.9.0 > Reporter: Sandy Ryza > Assignee: Sandy Ryza > Fix For: 1.0.0 > > > {code} > var javaCommand = "java" > val javaHome = System.getenv("JAVA_HOME") > if ((javaHome != null && !javaHome.isEmpty()) || > env.isDefinedAt("JAVA_HOME")) { > javaCommand = Environment.JAVA_HOME.$() + "/bin/java" > } > {code} > Currently, if JAVA_HOME is specified on the client, it will be used instead > of the value given on the cluster. This makes it so that Java must be > installed in the same place on the client as on the cluster. > This is a possibly incompatible change that we should get in before 1.0. -- This message was sent by Atlassian JIRA (v6.3.4#6332) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org