[
https://issues.apache.org/jira/browse/YARN-7766?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16336246#comment-16336246
]
Jian He commented on YARN-7766:
-------------------------------
I'm wondering if below makes sense to be a followup item.
Currently, if user doesn't supply location when run yarn app -enableFastLaunch,
the jars will be put under this location
{code}
hdfs:///yarn-services/<versionId>/service-dep.tar.gz
{code}
Since API server is embedded in RM, should RM look for this location too if
"yarn.service.framework.path" is not specified ?
And if "yarn.service.framework.path" is not specified and still the file
doesn't exist at above default location, I think RM can try to upload the jars
to above default location instead, currently RM is uploading the jars to the
location defined by below code. This folder is per app and also inconsistent
with CLI location.
{code}
protected Path addJarResource(String serviceName,
Map<String, LocalResource> localResources)
throws IOException, SliderException {
Path libPath = fs.buildClusterDirPath(serviceName);
{code}
By doing this, the next time a submission request comes, RM doesn't need to
upload the jars again.
> Introduce a new config property for YARN Service dependency tarball location
> ----------------------------------------------------------------------------
>
> Key: YARN-7766
> URL: https://issues.apache.org/jira/browse/YARN-7766
> Project: Hadoop YARN
> Issue Type: Bug
> Components: applications, client, yarn-native-services
> Reporter: Gour Saha
> Assignee: Gour Saha
> Priority: Major
> Fix For: 3.1.0
>
> Attachments: YARN-7766.001.patch, YARN-7766.002.patch,
> YARN-7766.003.patch, YARN-7766.004.patch
>
>
> Introduce a new config property (something like _yarn.service.framework.path_
> in-line with _mapreduce.application.framework.path_) for YARN Service
> dependency tarball location. This will provide flexibility to the
> user/cluster-admin to upload the dependency tarball to a location of their
> choice. If this config property is not set, YARN Service client will default
> to uploading all dependency jars from the client-host's classpath for every
> service launch request (as it does today).
> Also, accept an optional destination HDFS location for *-enableFastLaunch*
> command, to specify the location where user/cluster-admin wants to upload the
> tarball. If not specified, let's default it to the location we use today. The
> cluster-admin still needs to set _yarn.service.framework.path_ to this
> default location otherwise it will not be used. So the command-line will
> become something like this -
> {code:java}
> yarn app -enableFastLaunch [<Destination Folder>]{code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
