liming30 commented on PR #1823: URL: https://github.com/apache/incubator-paimon/pull/1823#issuecomment-1681963695
> Overall, I have a question. Will the current deletions be strictly in order? For example, deleting the datafiles first and then the manifestfiles will ensure the reentrancy of the expiration cleanup and avoid leaving zombie files due to failover. @JingsongLi It can only guarantee the order of deleting `dataFiles` first, and then deleting `manifests`. But there is no guarantee that zombie files will not be generated, because we always `commitEarliestHint` first to ensure that the expired snapshot is invisible, and we do not wait for the deletion of `dataFiles`/`manifests` to end. In case of failover, these files may become zombie files. > 2. Maybe: make the entire expire asynchronous and completely decouple from the commit, without blocking the normal process of the commit. Do you mean we can introduce an `expire` operator? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
