mxm commented on issue #14090: URL: https://github.com/apache/iceberg/issues/14090#issuecomment-3324732610
>(9) is where the flaw lies, because every committable (WriteResult) updates the snapshot properties with the the max checkpoint id. We can fix this issue by staging all WriteResults for a given TableKey and only then commit. This will also work across branches because the max checkpoint id is already maintained per branch. Even if we fail in the process, we will only attempt to commit the WriteResults for a given TableKey which haven't been committed previously. There is one false assumption here. For regular append WriteResults, we can follow the above approach, but whenever delete files are present in the next WriteResult, we need to create a table snapshot for the current WriteResult, because delete files are applied before appending new data. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
