[
https://issues.apache.org/jira/browse/HADOOP-15528?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16531802#comment-16531802
]
Giovanni Matteo Fumarola commented on HADOOP-15528:
---------------------------------------------------
This Jira is changing the way a container starts.
In the old semantic, we create the directories and all the symlink *AFTER* the
container starts.
In this new semantic, we create everything *BEFORE* the container starts. We
will have more control over failures due to Symlink and better performance.
I run this patch in Windows, and I saw a good 35% reduction in latency for
symlink and 7000 IO ops less than before.
In future, we can add more retry logic over failures and possible avoiding to
start a container when we are not able to recover from a failure.
cc. [~subru] , [~curino] , [~leftnoteasy] , [~sunilg]
> Deprecate ContainerLaunch#link by using FileUtil#SymLink
> --------------------------------------------------------
>
> Key: HADOOP-15528
> URL: https://issues.apache.org/jira/browse/HADOOP-15528
> Project: Hadoop Common
> Issue Type: Sub-task
> Reporter: Giovanni Matteo Fumarola
> Assignee: Giovanni Matteo Fumarola
> Priority: Major
> Attachments: HADOOP-15528-HADOOP-15461.v1.patch,
> HADOOP-15528-HADOOP-15461.v2.patch, HADOOP-15528-HADOOP-15461.v3.patch
>
>
> {{ContainerLaunch}} currently uses its own utility to create links (including
> winutils).
> This should be deprecated and rely on {{FileUtil#SymLink}} which is already
> multi-platform and pure Java.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
