Abacn commented on issue #39370: URL: https://github.com/apache/beam/issues/39370#issuecomment-5097570818
Reify here was intentional, as it introduces a fusion break to make sure all files gets written to the final destination when succeeded, or none gets written when there is error in upstream steps causing pipeline failure, leaving incomplete results. I understand there is a feature request sidestepping two stage write for performance reason. This could be added as an option (to write directly to final destination) but please do not change existing behavior as its a breaking change -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
