[
https://issues.apache.org/jira/browse/BEAM-57?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15634012#comment-15634012
]
ASF GitHub Bot commented on BEAM-57:
------------------------------------
Github user asfgit closed the pull request at:
https://github.com/apache/incubator-beam/pull/1050
> FileBasedSink should write intermediate output to a different folder
> --------------------------------------------------------------------
>
> Key: BEAM-57
> URL: https://issues.apache.org/jira/browse/BEAM-57
> Project: Beam
> Issue Type: New Feature
> Components: sdk-java-core
> Reporter: Daniel Halperin
> Assignee: Eugene Kirpichov
> Priority: Minor
>
> Currently, FileBasedSink writes intermediate files and final output files to
> the same folder. Since deletion of intermediate files is prone to eventual
> consistency issues with GCS,S3,other systems and users may use globs in
> subsequent pipelines, intermediate files should be written to a different
> folder.
> This may be a subdirectory or a pipeline-defined temp directory (with
> probably some additional subdirectories for this source).
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
