[
https://issues.apache.org/jira/browse/MAPREDUCE-1302?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12791259#action_12791259
]
Ravi Gummadi commented on MAPREDUCE-1302:
-----------------------------------------
deletion of contents of work dir in TaskTracker.SetupWorkDir() should also be
done asynchronously(as in MAPREDUCE-1213, by first moving to toBeDeleted and
then removing by a separate thread(could be CleanupQueue)) sothat the next task
in the same jvm(in case of jvm reuse) can start immediately. Currently this
deletion is done inline, which can take a long time based on the number of
files existing in work dir.
> TrackerDistributedCacheManager can delete file asynchronously
> -------------------------------------------------------------
>
> Key: MAPREDUCE-1302
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-1302
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Reporter: Zheng Shao
> Assignee: Zheng Shao
> Attachments: MAPREDUCE-1302.0.patch
>
>
> With the help of AsyncDiskService from MAPREDUCE-1213, we should be able to
> delete files from distributed cache asynchronously.
> That will help make task initialization faster, because task initialization
> calls the code that localizes files into the cache and may delete some other
> files.
> The deletion can slow down the task initialization speed.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
