[
https://issues.apache.org/jira/browse/MAPREDUCE-1140?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12779954#action_12779954
]
V.Karthikeyan commented on MAPREDUCE-1140:
------------------------------------------
Test Scenario:
Configure local.cache.size to 4GB.
Ran Job1 with cache files file1 and file2 - Job succeeded.
Ran Job2 with cache files file3 and file1. When file3 is getting localized,
removed file3 from dfs - Job2 failed.
Ran Job3 with cache files file1, file1(again) and file4. file4 is huge (say
5GB), larger than local.cache.size.
Without the patch, but the bug could be reproduced and verified by validating
the logs(file1,file2,file3 got deleted if the cache is full).
Test Scenario: (With the patch)
Configure local.cache.size to 4GB.
Ran Job1 with cache files file1 and file2 - Job succeeded.
Ran Job2 with cache files file3 and file1. When file3 is getting localized,
removed file3 from dfs - Job2 failed.
Ran Job3 with cache files file1, file1(again) and file4. file4 is huge (say
5GB), larger than local.cache.size.
Job3 succeeded, and it is also validated using the logs (file2 and file3 got
deleted while, file1 never got deleted even the cache is not enough to localize
file4. ).
> Per cache-file refcount can become negative when tasks release
> distributed-cache files
> --------------------------------------------------------------------------------------
>
> Key: MAPREDUCE-1140
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-1140
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: tasktracker
> Affects Versions: 0.20.2, 0.21.0, 0.22.0
> Reporter: Vinod K V
> Assignee: Amareshwari Sriramadasu
> Attachments: patch-1140-1.txt, patch-1140-2.txt,
> patch-1140-ydist.txt, patch-1140-ydist.txt, patch-1140.txt
>
>
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
