gaborgsomogyi commented on PR #27423: URL: https://github.com/apache/flink/pull/27423#issuecomment-3870494838
> You're absolutely right. MinIO itself doesn't create a default file. it's just passive storage. However, in our environment (with a webhook), when a PutObject action is detected, a default.txt file is created. I strongly believe that Flink is responsible to delete the files/directories what it's created. I can be convinced if we would talk about Flink generated garbage/not needed files deletion and we can justify that never ever delete shared things. This can be done by testing all 3 RestoreMode cases (CLAIM, NO_CLAIM, LEGACY) and listing all cases which call this delete recursive code together with the justification why it's safe. It's important to highlight that I basically tend to feel that recursive delete is a good direction but: - External system created random files is not business justification to take accidental delete risk - Big tech companies are having TB scale states and accidental deletes have serious consequences so we need exact proofs - My current not proven understanding is that the actual codebase is depending on the false (non-recursive behavior) and works as a safety belt. Maybe caller side(s) must be changed. All in all I can look back when more proof is available here. Again. I'm not nitpicking, state cleanup is required and good direction but we must give exact proofs. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
