[
https://issues.apache.org/jira/browse/YARN-2185?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16331335#comment-16331335
]
Robert Kanter commented on YARN-2185:
-------------------------------------
Here's some comments:
# The do-while loop in {{RunJar#unJar}} can be made into a for-loop, which is
probably more clear and avoids the infinite loop and break. {{for(JarEntry
entry = jar.getNextJarEntry(); entry == null; entry = jar.getNextJarEntry())}}
#- Similarly with {{FileUtil#unZip}}
# In {{FSDownload}}, it has {{downloadAndUnpack}} catch {{Exception}} to wrap
and throw it in a {{RuntimeException}}, which is then caught and thrown as a
{{YarnException}} in {{verifyAndCopy}}. Why not simply have
{{downloadAndUnpack}} throw a {{YarnException}} directly instead of a
{{RuntimeException}}, cutting out one level of wrapping? {{downloadAndUnpack}}
is a new and {{private}} method, so there's no worry about needing to make it
not declare a throwable.
# In the untar command, shouldn't we use {{&&}} instead of {{;}}? With {{;}},
it'll continue even if the first command fails. It looks like you already
agreed with [~jlowe] to make this change? We should also fix the existing
{{unTarUsingTar}} method too.
# There's a number of utility functions where you added a second version that
operates on streams instead of files. We should rename the relevant arguments
for clarity, like you did in {{unTarUsingJava}} ({{inFile}} -->
{{inputStream}}).
> Use pipes when localizing archives
> ----------------------------------
>
> Key: YARN-2185
> URL: https://issues.apache.org/jira/browse/YARN-2185
> Project: Hadoop YARN
> Issue Type: Improvement
> Components: nodemanager
> Affects Versions: 2.4.0
> Reporter: Jason Lowe
> Assignee: Miklos Szegedi
> Priority: Major
> Attachments: YARN-2185.000.patch, YARN-2185.001.patch,
> YARN-2185.002.patch, YARN-2185.003.patch, YARN-2185.004.patch,
> YARN-2185.005.patch, YARN-2185.006.patch
>
>
> Currently the nodemanager downloads an archive to a local file, unpacks it,
> and then removes it. It would be more efficient to stream the data as it's
> being unpacked to avoid both the extra disk space requirements and the
> additional disk activity from storing the archive.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
