[
https://issues.apache.org/jira/browse/YARN-71?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13599622#comment-13599622
]
Siddharth Seth commented on YARN-71:
------------------------------------
Thanks for the patches Xuan. Couple of comments on the latest one (71-6.txt)
- The ResourceLocalizationService is already creating a FileContxt instance -
that can be used.
- The rename/exception code is the same across the 3 top level dirs and can
move into a function.
- The patch needs some formatting fixes (missing spaces after "+", "," etc)
- Haven't looked at the unit test yet, but I don't think it belongs in
TestNodeStatusUpdater.
In terms of the approach of moving dirs and scheduling them for background
deletion - this seems reasonable to me. Once concern I have though is that this
could cause some load on the NMs after they're restarted (especially on systems
with a large dist cache and disks). This effectively ends up affecting
performance. Anyone else have thoughts on this ? Maybe background / inline
deletion should be configurable ?
> Ensure/confirm that the NodeManager cleans up local-dirs on restart
> -------------------------------------------------------------------
>
> Key: YARN-71
> URL: https://issues.apache.org/jira/browse/YARN-71
> Project: Hadoop YARN
> Issue Type: Bug
> Components: nodemanager
> Reporter: Vinod Kumar Vavilapalli
> Assignee: Xuan Gong
> Priority: Critical
> Attachments: YARN-71.1.patch, YARN-71.2.patch, YARN-71.3.patch,
> YARN.71.4.patch, YARN-71.5.patch, YARN-71.6.patch
>
>
> We have to make sure that NodeManagers cleanup their local files on restart.
> It may already be working like that in which case we should have tests
> validating this.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
