[
https://issues.apache.org/jira/browse/MAPREDUCE-1568?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12861017#action_12861017
]
Scott Chen commented on MAPREDUCE-1568:
---------------------------------------
Amareshwari,
The reason we made this change is because we saw that even with
MRAsyncDiskService, the task which performs the deletion can still get 600
second timeout.
I agree with your point. The deletion of distributed cache should not be in
Task's space. I will work on it and upload the patch soon. Thanks for your help.
> TrackerDistributedCacheManager should do deleteLocalPath asynchronously
> -----------------------------------------------------------------------
>
> Key: MAPREDUCE-1568
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-1568
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Affects Versions: 0.22.0
> Reporter: Scott Chen
> Assignee: Scott Chen
> Fix For: 0.22.0
>
> Attachments: MAPREDUCE-1568.txt
>
>
> TrackerDistributedCacheManager.deleteCache() has been improved:
> MAPREDUCE-1302 makes TrackerDistributedCacheManager rename the caches in the
> main thread and then delete them in the background
> MAPREDUCE-1098 avoids global locking while do the renaming (renaming lots of
> directories can also takes a long time)
> But the deleteLocalCache is still in the main thread of TaskRunner.run(). So
> it will still slow down the task which triggers the deletion (originally this
> will blocks all tasks, but it is fixed by MAPREDUCE-1098). Other tasks do not
> wait for the deletion. The task which triggers the deletion should not wait
> for this either. TrackerDistributedCacheManager should do deleteLocalPath()
> asynchronously.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
