tanmayrauth commented on issue #2339:
URL: https://github.com/apache/iceberg-go/issues/2339#issuecomment-6475120235
I'm +1 on the direction. One thing I think the design has to cover
explicitly:
PostCommit has to see the replanned update. Transaction.Commit runs
PostCommit over the staged meta.updates with t.tbl as preTable. The rebuilt
updates never leave doCommit. removeSnapshotsUpdate.PostCommit deletes each
listed snapshot's manifest list without checking it against postTable; only
data and stats files get that check. Today the pins keep staged and committed
IDs identical, so this is safe. With replanning, if a peer's branch makes us
keep snapshot S, a stale PostCommit deletes S's manifest list while S is still
live. I think doCommit needs to return the final updates and the metadata they
were planned against, and PostCommit should run on those, as Java's cleanup
does with the base it re-reads in internalApply.
To confirm: the rebuild hook would also regenerate the expiry's
AssertRefSnapshotID requirements, right? Non-commit-branch assertions are
validated as-is after a refresh, so dropping the pins alone wouldn't let a
moved branch through.
@laskoviymishka @zeroshade, any thoughts on the approach?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
