badalprasadsingh opened a new issue, #2606:
URL: https://github.com/apache/iceberg-go/issues/2606
### Apache Iceberg version
main (development)
### Please describe the bug 🐞
After a partition spec change, a v2 mor Delete of a row in a data file
written under the old spec writes its pos-delete file under the new spec. The
delete file keeps the data file's partition values (with `null` for the new
field) but gets the new spec ID. After that, every scan fails:
```
invalid metadata: position delete file
.../data/cat=a/00000-0-c8f9...-00001.parquet uses partition spec 1, but data
file .../data/cat=a/00000-0-7d13...-00001.parquet uses partition spec 0
```
I mean: with `write.delete.mode=merge-on-read`:
1. v2 table `id long, cat string, region string`, partitioned by
`identity(cat)`
2. append `(1, a, x)` and `(2, a, x)`
3. `UpdateSpec().AddIdentity("region")`
4. `Delete(id = 1)`: commits
5. `Scan()`: fails with the error above
On v0.7.0 the same steps write the same wrong delete file, and the scan
silently skips it. It returns `[1 2]` instead of `[2]`, so the delete is lost.
The spec says a pos-delete file applies to a data file only when "the data
file's partition (both spec and partition values) is equal" to the delete
file's partition. v3 tables are not affected, because the dv path already
writes each dv with its data file's spec
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]