pabloem commented on pull request #15576: URL: https://github.com/apache/beam/pull/15576#issuecomment-933959656
sorry about the delay on this. Currently we're removing files at an erroneous time. I think we need to delete orphaned files in a subsequent step (after we're sure that there won't be retries). But for now, and since Beam 2.34.0 is around the corner, I think we can move forward with this, because we want to make sure no data is dropped - I will review this by tomorrow and merge it if it looks good -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
