[
https://issues.apache.org/jira/browse/MAPREDUCE-6512?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14988166#comment-14988166
]
Jason Lowe commented on MAPREDUCE-6512:
---------------------------------------
I would love to fix this, since it would help jobs fail when they collide in
the output directory, but I'm not sure we can. I fear there may be custom
output formats or other setups where they are implicitly (or explicitly)
relying on the automatic parent directory creation to create the output
directory itself. If the output directory was being created by this behavior
and we stop doing it then we'll start failing jobs that used to work before.
> FileOutputCommitter tasks unconditionally create parent directories
> -------------------------------------------------------------------
>
> Key: MAPREDUCE-6512
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6512
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Reporter: Chang Li
> Assignee: Chang Li
> Attachments: MAPREDUCE-6512.2.patch, MAPREDUCE-6512.2.patch,
> MAPREDUCE-6512.patch, MAPREDUCE-6512.patch
>
>
> If the output directory is deleted then subsequent tasks should fail. Instead
> they blindly create the missing parent directories, leading the job to be
> "succesful" despite potentially missing almost all of the output. Task
> attempts should fail if the parent app attempt directory is missing when they
> go to create their task attempt directory.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
