badalprasadsingh opened a new issue, #2606:
URL: https://github.com/apache/iceberg-go/issues/2606

   ### Apache Iceberg version
   
   main (development)
   
   ### Please describe the bug 🐞
   
   After a partition spec change, a v2 mor Delete of a row in a data file 
written under the old spec writes its pos-delete file under the new spec. The 
delete file keeps the data file's partition values (with `null` for the new 
field) but gets the new spec ID. After that, every scan fails:
   
   ```
   invalid metadata: position delete file 
.../data/cat=a/00000-0-c8f9...-00001.parquet uses partition spec 1, but data 
file .../data/cat=a/00000-0-7d13...-00001.parquet uses partition spec 0
   ```
   
   I mean: with `write.delete.mode=merge-on-read`:
   
   1. v2 table `id long, cat string, region string`, partitioned by 
`identity(cat)`
   2. append `(1, a, x)` and `(2, a, x)`
   3. `UpdateSpec().AddIdentity("region")`
   4. `Delete(id = 1)`: commits
   5. `Scan()`: fails with the error above
   
   On v0.7.0 the same steps write the same wrong delete file, and the scan 
silently skips it. It returns `[1 2]` instead of `[2]`, so the delete is lost.
   
   The spec says a pos-delete file applies to a data file only when "the data 
file's partition (both spec and partition values) is equal" to the delete 
file's partition. v3 tables are not affected, because the dv path already 
writes each dv with its data file's spec


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]


---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Reply via email to