[
https://issues.apache.org/jira/browse/TWILL-117?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14598561#comment-14598561
]
ASF GitHub Bot commented on TWILL-117:
--------------------------------------
GitHub user gsps1 opened a pull request:
https://github.com/apache/incubator-twill/pull/49
[Twill-117] adding hadoop classpath to the AM Container launch
we add hadoop_classpath to the AM container classpath, so address the issue
mentioned here, https://issues.apache.org/jira/browse/TWILL-117
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/gsps1/incubator-twill
feature/TWILL-117-Add-Hadoop-Classpath-AMLaunch
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/incubator-twill/pull/49.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #49
----
commit 6170a3b90cb5aeb71fed8b68d53fa84c91c99cec
Author: shankar <[email protected]>
Date: 2015-06-17T18:36:25Z
allow adding hadoop classpath to the AM launcher
----
> MapR extension
> ---------------
>
> Key: TWILL-117
> URL: https://issues.apache.org/jira/browse/TWILL-117
> Project: Apache Twill
> Issue Type: New Feature
> Components: yarn
> Affects Versions: 0.5.0-incubating
> Reporter: Tongqing Qiu
> Priority: Minor
> Fix For: 0.6.0-incubating
>
>
> I made some extensions to make twill working for mapr. Besides of what I
> included in github, there are some other issues. Currently, I have to include
> maprfs library in yarn-example pom file, then related jars will be bundled
> and deployed into yarn. These libraries, however, are included in the
> HADOOP_CLASS, I don't know why it is not been loading when twill/yarn
> application runs. So if we don't are not using bundle jar application while
> let twill to scan dependent classes (like CDAP startup services did), the
> jars are missing. The way to make it work right now is a hacking:
> 1. add new MapRFileSystem() in the code of TwillContainerMain and
> Hadoop21YarnAppClient class, so that the dependency will be included, even
> though it does nothing. BTW, I think it works for other hadoop distribution,
> because there two classes already depends the hdfs related stuff and jars are
> packaged into container.jar and application jar (which I think is not
> necessary since very yarn node should already contain these jars)
> 2. Add twillRunner.setJVMOptions("-Djava.library.path=/opt/mapr/lib") when
> run the twill application.
> I think it is really not the right way to do, but not sure how to avoid that,
> just let yarn/twill know to include HADOOP_CLASSPATH when run the application?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
