[
https://issues.apache.org/jira/browse/YARN-2185?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16338279#comment-16338279
]
Miklos Szegedi commented on YARN-2185:
--------------------------------------
Thank you, [~jlowe]. I updated the patch.
{quote}makeShellPath is a pre-existing, public function,...
{quote}
If we want to be conservative with this jira I would be more conservative with
security rather than compatibility. I used both Windows and Linux but I have
never seen a legitimate path with ' or ". That said, I would rather throw an
exception in these cases rather than trying to be backward compatible for
existing malware. :) I changed the patch to have a new makeSecureShellPath
function, and I ignore Windows, since it is dead code right now, and it is not
so trivial to avoid code injection.
{quote}Attempting to get the futures from the executor could result in an
ExecutionException...
{quote}
I caught the only IOException thrown there but anyone can change the code
later, so I refactored the it a little bit to protect this scenario. I also log
now on the fly, to avoid OOM errors. Let me know, what you think.
> Use pipes when localizing archives
> ----------------------------------
>
> Key: YARN-2185
> URL: https://issues.apache.org/jira/browse/YARN-2185
> Project: Hadoop YARN
> Issue Type: Improvement
> Components: nodemanager
> Affects Versions: 2.4.0
> Reporter: Jason Lowe
> Assignee: Miklos Szegedi
> Priority: Major
> Attachments: YARN-2185.000.patch, YARN-2185.001.patch,
> YARN-2185.002.patch, YARN-2185.003.patch, YARN-2185.004.patch,
> YARN-2185.005.patch, YARN-2185.006.patch, YARN-2185.007.patch,
> YARN-2185.008.patch, YARN-2185.009.patch, YARN-2185.010.patch
>
>
> Currently the nodemanager downloads an archive to a local file, unpacks it,
> and then removes it. It would be more efficient to stream the data as it's
> being unpacked to avoid both the extra disk space requirements and the
> additional disk activity from storing the archive.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
