André Kelpe created TEZ-2563:
--------------------------------
Summary: unable to pass environment/classpath to AM via TezClient
Key: TEZ-2563
URL: https://issues.apache.org/jira/browse/TEZ-2563
Project: Apache Tez
Issue Type: Bug
Affects Versions: 0.6.1, 0.7.0
Reporter: André Kelpe
In Cascading we allow our users to submit applications as jar files. These jar
files can contain custom Input- and Output-Formats, that are only in the
application jar and not installed cluster side. We now run into the problem
that the ApplicationMaster is unable to find the InputFormat during split
calculation, if it is packaged in a hadoop style jar. (fat-jars seem to work).
On the Dag level we are able to pass an environment along with the
LocalResources to be used so that we can tell YARN, what to put on the
classpath. There is no obvious way to do the same for the TezClient so that we
can fix the ClassNotFoundException.
P.S.:
I tried to "hack" around this problem by setting
TezConfiguration.TEZ_AM_LAUNCH_ENV, but that does not work, since I have to be
able to resolve all paths and cannot just pass the templates along.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
