vishnuprakaz commented on PR #17630:
URL: https://github.com/apache/iceberg/pull/17630#issuecomment-5284839153
Thanks for taking a look @mxm!
Actually no incident behind it,
I was following the conversion work and was reading through the operators to
understand how a cycle works. I ended up wondering what happens when a cycle
fails partway through. There's a retry test for that, but the failure comes
before the deletes resolve, so I couldn't tell what happens when it comes after.
So I went back through the series to see whether it had come up before, and
found #17038. Two cycles ended up running on the same uncommitted snapshot
there, and one committed its marker without the DV. Same result as what I was
worried about, just a different cause so I wanted to see whether one failing
cycle could do it alone. That's what the test in the PR checks.
so I don't have numbers on how often cycles fail. My guess is occasional
rather than frequent a transient read error, or a checkpoint landing mid cycle
on a restart.
Agreed on the buffering,I think the planner needs the pending snapshot
either way to tell the index whether to apply or discard.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
