[
https://issues.apache.org/jira/browse/HADOOP-1612?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Arun C Murthy updated HADOOP-1612:
----------------------------------
Assignee: Arun C Murthy
bq. It might be a race condition: does the job complete before the files in the
output directory are promoted?
No. The way it works is that a task isn't declared as a *success* until
promotion completes successfully (to guard against failure to move files etc.)
and hence the job is declared a *success* only after tasks promote their files
successfully.
I'll take this one up... and I'd appreciate any more information you have to
help me fix this. For e.g. do you have multiple outputs per task i.e. do you
open a hdfs file from within your map/reduce functions and write to it etc.?
Thanks!
> listing of an output directory shortly after job completion fails
> -----------------------------------------------------------------
>
> Key: HADOOP-1612
> URL: https://issues.apache.org/jira/browse/HADOOP-1612
> Project: Hadoop
> Issue Type: Bug
> Components: mapred
> Affects Versions: 0.14.0
> Reporter: Christian Kunz
> Assignee: Arun C Murthy
> Priority: Blocker
> Fix For: 0.14.0
>
>
> Sometimes, after a job finishes, and another application wants to rename dfs
> files created by that job, listing of the output directory containing the
> newly created files fails. File creation and directory listing is done via
> libhdfs, but it is unlikely that this makes any difference, therefore, I add
> this to the mapred component.
> It might be a race condition: does the job complete before the files in the
> output directory are promoted?
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
