RussellSpitzer commented on issue #3466:
URL: https://github.com/apache/iceberg/issues/3466#issuecomment-1104404998
We were just discussing this today, the message is a bit confusing but is
the correct response. I think we should probably change it to something like
"Cannot apply RewriteManifests result since manifests being replaced have
already been removed in the current snapshot"
because what is happening actually the same as an other validation error we
do during the commit phase. For example, say we run two optimize metadata
operations simultaneously
```
Optimize 1: Rewrites Manfiests A and B into C' : Manifests Deleted [A,
B], Manifests Added [C']
Optimize 2: Rewrites Manifests A and B into C'' : Manifests Deleted [A,
B], Manifests Added [C'']
```
If 1 finishes first the state of our table is
```
Manifests [C']
```
If we try to apply Optimize 2 we see that [A and B] are already deleted and
not present, if we added [C''] our table would look like
```
Manifests [C', C'']
```
And end up with duplicate records! So the correct response is to cancel the
2nd optimize oepration
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
