[
https://issues.apache.org/jira/browse/MAPREDUCE-1225?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12858757#action_12858757
]
Amareshwari Sriramadasu commented on MAPREDUCE-1225:
----------------------------------------------------
Comments on the patch :
* Calling the method checkCacheStatusValidity even for the cache that has not
inited does not solve the problem. checkCacheStatusValidity does the check by
calling isExistsAndFresh(). isExistsAndFresh does two checks :
## dfsFileStamp != confFileStamp
## dfsFileStamp != lcacheStatus.mtime
Here, the second check is not valid for the file that is not localized. So,
isExistsAndFresh() always returns false after the patch. We should refactor the
first check into a different method and call only that for cache that is not
yet inited. Does that make sense?
* Can you add a unit test in TestTrackerDistributedCacheManager which fails
without the patch and passes after the patch ?
> TT successfully localizes a task even though the corresponding cache-file has
> already changed on DFS.
> -----------------------------------------------------------------------------------------------------
>
> Key: MAPREDUCE-1225
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-1225
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: tasktracker
> Affects Versions: 0.21.0, 0.22.0
> Reporter: Vinod K V
> Fix For: 0.21.0
>
> Attachments: MAPREDUCE-1225.patch, MAPREDUCE-1225.patch
>
>
> This happens with the first task of a job being localized on this TT. TT
> doesn't check if the file on DFS is fresh according to the timestamps set in
> job-conf during submission. After the first task incorrectly gets localized,
> all further tasks fail on this TT as expected.
> Found this issue while trying to improve test-case for MAPREUCE-913.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
