Siddharth Seth created YARN-2464:

             Summary: Provide Hadoop as a local resource (on HDFS) which can be 
used by other projcets
                 Key: YARN-2464
             Project: Hadoop YARN
          Issue Type: Improvement
            Reporter: Siddharth Seth

DEFAULT_YARN_APPLICATION_CLASSPATH are used by YARN projects to setup their AM 
/ task classpaths if they have a dependency on Hadoop libraries.

It'll be useful to provide similar access to a Hadoop tarball (Hadoop libs, 
native libraries) etc, which could be used instead - for applications which do 
not want to rely upon Hadoop versions from a cluster node. This would also 
require functionality to update the classpath/env for the apps based on the 
structure of the tar.

As an example, MR has support for a full tar (for rolling upgrades). Similarly, 
Tez ships hadoop libraries along with it's build. I'm not sure about the Spark 
/ Storm / HBase model for this - but using a common copy instead of everyone 
localizing Hadoop libraries would be useful.

This message was sent by Atlassian JIRA

Reply via email to