Pei He commented on BEAM-894:

Closed without the fix for the following reason:
@Teardown doesn't guarantee temp files are always removed, and the benefits are 
not worth the extra requirement on Sink writer: "additional abstract method 

> Using @Teardown to remove temp files from failed bundles in Write.WriteBundles
> ------------------------------------------------------------------------------
>                 Key: BEAM-894
>                 URL: https://issues.apache.org/jira/browse/BEAM-894
>             Project: Beam
>          Issue Type: New Feature
>          Components: sdk-java-core
>            Reporter: Pei He
>            Assignee: Pei He
>             Fix For: Not applicable
> FileBasedSink lefts temp files behind for failed bundles, and it forces 
> finalize() to depend on pattern match.
> However, pattern matching is not always reliable for eventual consistency 
> file system, such as GCS.
> Given we now have DoFn.TearDown, we can improve FileBasedSink (and in general 
> Write transform) to remove temp files/resources early when DoFn bundles fail.

This message was sent by Atlassian JIRA

Reply via email to