manuzhang opened a new issue, #19026:
URL: https://github.com/apache/iceberg/issues/19026

   ### Apache Iceberg version
   
   main (development)
   
   ### Please describe the bug 🐞
   
   When `RewriteManifests` explicitly replaces a manifest staged earlier in the 
same transaction, a concurrent commit can cause the transaction to retry and 
publish the replacement with metadata from the original attempt.
   
   1. Stage an append of file A with `Transaction.newFastAppend()`.
   2. Read the staged manifest and write a replacement that copies its entries 
as `EXISTING`. In the same transaction, use `RewriteManifests` to delete the 
staged manifest and add the replacement.
   3. Append file B outside the transaction, then call `commitTransaction()`.
   
   On retry, the staged manifest keeps its path but can inherit a new sequence 
number and first row ID from the new manifest list. `validateDeletedManifests` 
checks presence by path, so the caller-supplied replacement can still be 
committed with the original values.
   
   The regression test reports a stale data sequence number for file A in 
format v2 and overlapping row-ID ranges for files A and B in formats v3 and v4. 
A stale data sequence number can also affect which equality deletes apply.
   
   The commit should preserve the sequence numbers and row IDs assigned on 
retry, or reject the replacement if its source manifest's inherited metadata 
changed.
   
   - Failing regression test in `TestRewriteManifests`. 
[manuzhang/iceberg#214](https://github.com/manuzhang/iceberg/pull/214)
   - Related transaction replay validation in Iceberg C++. 
[apache/iceberg-cpp#1104](https://github.com/apache/iceberg-cpp/pull/1104)
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]


---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

  • [I] Core: RewriteManifests c... manuzhang (via GitHub) via issues

Reply via email to