[
https://issues.apache.org/jira/browse/YARN-2185?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16852859#comment-16852859
]
Prashant Gour commented on YARN-2185:
-------------------------------------
In this ticket there are changes made in RunJar.java where in a new method
public static void unJar(InputStream inputStream, File toDir, Pattern
unpackRegex) is added. This method uses JarInputStream to unpack a jar file
provided as an archive file. Now, JarInputStream ignores "META-INF/MANIFEST.MF"
specifically, present in jar.
The above method is being called from FSDownload.java in case the archive is a
jar file.
In our framework, when an application is launched on YARN, it uses field named
"Export-Package" in MANIFEST.MF to load appropriate classes. As the Manifest
file is no longer available in jar file, none of the classes is being loaded
from the archived jar. As a consequence, our YARN applications are failing. The
same used to work perfectly for us prior to Hadoop 3.1. Can some please re-look
the changes in this ticket as they are backward incompatible or suggest
otherwise.
> Use pipes when localizing archives
> ----------------------------------
>
> Key: YARN-2185
> URL: https://issues.apache.org/jira/browse/YARN-2185
> Project: Hadoop YARN
> Issue Type: Improvement
> Components: nodemanager
> Affects Versions: 2.4.0
> Reporter: Jason Lowe
> Assignee: Miklos Szegedi
> Priority: Major
> Fix For: 3.1.0
>
> Attachments: YARN-2185.000.patch, YARN-2185.001.patch,
> YARN-2185.002.patch, YARN-2185.003.patch, YARN-2185.004.patch,
> YARN-2185.005.patch, YARN-2185.006.patch, YARN-2185.007.patch,
> YARN-2185.008.patch, YARN-2185.009.patch, YARN-2185.010.patch,
> YARN-2185.011.patch, YARN-2185.012.patch, YARN-2185.012.patch,
> YARN-2185.013.patch, YARN-2185.014.patch
>
>
> Currently the nodemanager downloads an archive to a local file, unpacks it,
> and then removes it. It would be more efficient to stream the data as it's
> being unpacked to avoid both the extra disk space requirements and the
> additional disk activity from storing the archive.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
