reuvenlax commented on pull request #13558: URL: https://github.com/apache/beam/pull/13558#issuecomment-780811179
Do other file systems have these retention policies? If not, is it safe to just continue overwriting in the HDFS case? On Wed, Feb 17, 2021 at 10:32 AM Pablo <[email protected]> wrote: > No. This is to avoid overwriting files that were successfully written > previously, but are part of a bundle that is being retried. > > We do this because some users are reporting issues where pipelines > experience errors on retries for GCS buckets with specific retention > policies (files cannot be deleted, and thus cannot be overwritten) > > — > You are receiving this because your review was requested. > Reply to this email directly, view it on GitHub > <https://github.com/apache/beam/pull/13558#issuecomment-780759786>, or > unsubscribe > <https://github.com/notifications/unsubscribe-auth/AFAYJVIHTFPPZXPZ4BN6TJ3S7QDSRANCNFSM4U5FVBKA> > . > ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected]
