[ 
https://issues.apache.org/jira/browse/BEAM-894?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15713239#comment-15713239
 ] 

Pei He commented on BEAM-894:
-----------------------------

Closed without the fix for the following reason:
@Teardown doesn't guarantee temp files are always removed, and the benefits are 
not worth the extra requirement on Sink writer: "additional abstract method 
abort()". 

> Using @Teardown to remove temp files from failed bundles in Write.WriteBundles
> ------------------------------------------------------------------------------
>
>                 Key: BEAM-894
>                 URL: https://issues.apache.org/jira/browse/BEAM-894
>             Project: Beam
>          Issue Type: New Feature
>          Components: sdk-java-core
>            Reporter: Pei He
>            Assignee: Pei He
>             Fix For: Not applicable
>
>
> FileBasedSink lefts temp files behind for failed bundles, and it forces 
> finalize() to depend on pattern match.
> However, pattern matching is not always reliable for eventual consistency 
> file system, such as GCS.
> Given we now have DoFn.TearDown, we can improve FileBasedSink (and in general 
> Write transform) to remove temp files/resources early when DoFn bundles fail.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Reply via email to