[
https://issues.apache.org/jira/browse/YARN-7799?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16455986#comment-16455986
]
Gour Saha commented on YARN-7799:
---------------------------------
[~billie.rinaldi] Few points -
h5. ServiceClient.java
{code}
System.out.println(String.format(
"Dependency libs are already uploaded to %s.", dependencyLibTarGzip
.toUri()));
{code}
Seems like this is going to be printed for every call to launch/start an app.
Is there some way we can suppress this and print only when enableFastLaunch is
called?
h5. Configurations.md
{code}
|yarn.service.framework.path | HDFS URI of the service AM dependency tarball.
When no file exists at this location, AM dependencies will be uploaded each
time a service is started or launched. If unspecified, value will be assumed to
be /yarn-services/$\{hadoop.version\}/service-dep.tar.gz.|
{code}
With this patch, I think this statement won't be valid anymore - "AM
dependencies will be uploaded each time a service is started or launched". It
will be uploaded only for the first service launch, right?
> YARN Service dependency follow up work
> --------------------------------------
>
> Key: YARN-7799
> URL: https://issues.apache.org/jira/browse/YARN-7799
> Project: Hadoop YARN
> Issue Type: Bug
> Components: client, resourcemanager
> Reporter: Gour Saha
> Assignee: Billie Rinaldi
> Priority: Critical
> Attachments: YARN-7799.1.patch
>
>
> As per [~jianhe] these are some followup items that make sense to do after
> YARN-7766. Quoting Jian's comment below -
> Currently, if user doesn't supply location when run yarn app
> -enableFastLaunch, the jars will be put under this location
> {code}
> hdfs:///yarn-services/<versionId>/service-dep.tar.gz
> {code}
> Since API server is embedded in RM, should RM look for this location too if
> "yarn.service.framework.path" is not specified ?
> And if "yarn.service.framework.path" is not specified and still the file
> doesn't exist at above default location, I think RM can try to upload the
> jars to above default location instead, currently RM is uploading the jars to
> the location defined by below code. This folder is per app and also
> inconsistent with CLI location.
> {code}
> protected Path addJarResource(String serviceName,
> Map<String, LocalResource> localResources)
> throws IOException, SliderException {
> Path libPath = fs.buildClusterDirPath(serviceName);
> {code}
> By doing this, the next time a submission request comes, RM doesn't need to
> upload the jars again.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
