[
https://issues.apache.org/jira/browse/TEZ-2563?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14590831#comment-14590831
]
Hitesh Shah commented on TEZ-2563:
----------------------------------
[~fs111] Can you upload the launch_container.sh for the ApplicationMaster? It
would exist in one of the local-dirs configured for the YARN Nodemanager on the
node where the AM was launched.
tez.am.launch.env seems to have been implement to only set the environment if
there is no existing value.
You can try setting "tez.cluster.additional.classpath.prefix" to get the
classpath updated. By default, $PWD and $PWD/* are always in the classpath so
normal jars would work if they have been added to LocalResources as type FILE.
On the other hand, fat-jars at times do not work depending on whether they are
unbundled on localization. Taking a look at launch_container.sh would help to
see what is going on. Furthermore, you can also look at the container's working
dir to see how the localized bits are laid out.
You will need to set "yarn.nodemanager.delete.debug-delay-sec" to a large value
and restart all nodemanagers to be able to find the above files. ( Dirs get
deleted on container completion unless debug delay is configured )
> unable to pass environment/classpath to AM via TezClient
> --------------------------------------------------------
>
> Key: TEZ-2563
> URL: https://issues.apache.org/jira/browse/TEZ-2563
> Project: Apache Tez
> Issue Type: Bug
> Affects Versions: 0.7.0, 0.6.1
> Reporter: André Kelpe
>
> In Cascading we allow our users to submit applications as jar files. These
> jar files can contain custom Input- and Output-Formats, that are only in the
> application jar and not installed cluster side. We now run into the problem
> that the ApplicationMaster is unable to find the InputFormat during split
> calculation, if it is packaged in a hadoop style jar. (fat-jars seem to
> work).
> On the Dag level we are able to pass an environment along with the
> LocalResources to be used so that we can tell YARN, what to put on the
> classpath. There is no obvious way to do the same for the TezClient so that
> we can fix the ClassNotFoundException.
> P.S.:
> I tried to "hack" around this problem by setting
> TezConfiguration.TEZ_AM_LAUNCH_ENV, but that does not work, since I have to
> be able to resolve all paths and cannot just pass the templates along.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
