[
https://issues.apache.org/jira/browse/FLINK-1268?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17321779#comment-17321779
]
Flink Jira Bot commented on FLINK-1268:
---------------------------------------
This issue and all of its Sub-Tasks have not been updated for 180 days. So, it
has been labeled "stale-minor". If you are still affected by this bug or are
still interested in this issue, please give an update and remove the label. In
7 days the issue will be closed automatically.
> FileOutputFormat with overwrite does not clear local output directories
> -----------------------------------------------------------------------
>
> Key: FLINK-1268
> URL: https://issues.apache.org/jira/browse/FLINK-1268
> Project: Flink
> Issue Type: Bug
> Components: API / DataSet
> Reporter: Till Rohrmann
> Priority: Minor
> Labels: stale-minor
>
> I noticed that the FileOutputFormat does not clear the output directories if
> it writes to local disk. This has the consequence that previous partitions
> are still contained in the directory if one decreases the DOP between
> subsequent runs. If one reads the data from this directory, then more
> partitions will be read in than were actually written. This can lead to a
> wrong user code behaviour which is hard to debug. I'm aware that in case of a
> distributed execution the TaskManagers or the Tasks have to be responsible
> for the cleanup and if multiple Tasks are running on a TaskManager, then the
> cleanup has to be coordinated.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
