Doug Haigh commented on YARN-2625:

To be honest, I never knew about that option - but that does not get me 
anything more than what I can read from the yarn-site.xml file (although I like 
not having to have those files around). 

It still has the two problems described above because

1) If the {{yarn.application.classpath}} value is not specified, I still have 
no way to know the default classpath
2) If the {{yarn.application.classpath}} value has environment variables in it, 
they still need to be resolved somehow.

If the value returned by that URL was the *resolved* classpath, that would be 
the work.

> Problems with CLASSPATH in Job Submission REST API
> --------------------------------------------------
>                 Key: YARN-2625
>                 URL: https://issues.apache.org/jira/browse/YARN-2625
>             Project: Hadoop YARN
>          Issue Type: Bug
>          Components: api
>    Affects Versions: 2.5.1
>            Reporter: Doug Haigh
> There are a couple of issues I have found specifying the CLASSPATH 
> environment variable using the REST API.
> 1) In the Java client, the CLASSPATH environment is usually made up of either 
> the value of the yarn.application.classpath in yarn-site.xml value or the 
> default YARN classpath value as defined by 
> consumers have no method of telling the resource manager to use the default 
> unless they hardcode the default value themselves. If the default ever 
> changes, the code would need to change. 
> 2) If any environment variables are used in the CLASSPATH environment 'value' 
> field, they are evaluated when the values are NULL resulting in bad values in 
> the CLASSPATH. For example, if I had hardcoded the CLASSPATH value to the 
> default of "$HADOOP_CONF_DIR, $HADOOP_COMMON_HOME/share/hadoop/common/*, 
> $HADOOP_COMMON_HOME/share/hadoop/common/lib/*, 
> $HADOOP_HDFS_HOME/share/hadoop/hdfs/*, 
> $HADOOP_HDFS_HOME/share/hadoop/hdfs/lib/*, 
> $HADOOP_YARN_HOME/share/hadoop/yarn/*, 
> $HADOOP_YARN_HOME/share/hadoop/yarn/lib/*" the classpath passed to the 
> application master is 
> ":/share/hadoop/common/*:/share/hadoop/common/lib/*:/share/hadoop/hdfs/*:/share/hadoop/hdfs/lib/*:/share/hadoop/yarn/*:/share/hadoop/yarn/lib/*"
> These two problems require REST API consumers to always have the fully 
> resolved path defined in the yarn.application.classpath value. If the 
> property is missing or contains environment varaibles, the application 
> created by the REST API will fail due to the CLASSPATH being incorrect.

This message was sent by Atlassian JIRA

Reply via email to