[
https://issues.apache.org/jira/browse/FLINK-1572?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14348807#comment-14348807
]
Robert Metzger commented on FLINK-1572:
---------------------------------------
How about removing the files of a failed job by default (seems to be Hadoop's
behavior as well:
http://hadoop.apache.org/docs/r1.2.1/mapred_tutorial.html#OutputCommitter) and
adding a configuration option to disable behavior?
> Output directories are created before input paths are checked
> -------------------------------------------------------------
>
> Key: FLINK-1572
> URL: https://issues.apache.org/jira/browse/FLINK-1572
> Project: Flink
> Issue Type: Improvement
> Components: JobManager
> Affects Versions: 0.9
> Reporter: Robert Metzger
> Priority: Minor
>
> Flink is first creating the output directories for a job before creating the
> input splits.
> If a job's input directories are wrong, the system will have created output
> directories for a failed job.
> It would be much better if the system is creating the output directories "on
> demand" before data is actually written.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
