[
https://issues.apache.org/jira/browse/MAPREDUCE-1288?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12860880#action_12860880
]
Amareshwari Sriramadasu commented on MAPREDUCE-1288:
----------------------------------------------------
bq. Why should an old job fail because of what is, essentially, an external
event?
The current behavior is that Job will fail if a distributed cache file (in use)
gets modified on the DFS. Even if the task is localized on a new tracker we
should fail the task, is being done through MAPREDUCE-1225.
Allen, What should be the behavior for the following use-case?
For the case : "some tasks have downloaded a version-0 file and ran
successfully; some other tasks cannot find version-0 and they can find only
version-1 file, and they use version-1file and run successfully", final Job
output would be undefined right? The output generated from the tasks which used
version-0 file would be entirely different from output generated from the tasks
which used version-1 file.
So, I think the current behavior to fail the job, if a file added to
distributed-cache gets changed after job submission is correct.
> DistributedCache localizes only once per cache URI
> --------------------------------------------------
>
> Key: MAPREDUCE-1288
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-1288
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: security, tasktracker
> Affects Versions: 0.21.0
> Reporter: Devaraj Das
> Priority: Blocker
> Fix For: 0.21.0
>
>
> As part of the file localization the distributed cache localizer creates a
> copy of the file in the corresponding user's private directory. The
> localization in DistributedCache assumes the key as the URI of the cachefile
> and if it already exists in the map, the localization is not done again. This
> means that another user cannot access the same distributed cache file. We
> should change the key to include the username so that localization is done
> for every user.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
