codeAnqiang-ma opened a new issue, #1815:
URL: https://github.com/apache/iceberg-go/issues/1815
### Apache Iceberg version
main (development)
### Please describe the bug 🐞
`snapshotProducer.accumulateSummaryDelta` resolves partition specs two
different ways depending on the branch. Added files go through
`sp.spec(int(df.SpecID()))`, which is a by-ID lookup on top of
`MetadataBuilder.GetSpecByID`. Removed data files, delete files and deletion
vectors instead index the spec slice directly, `specs[df.SpecID()]`, in three
places.
Spec ID is an identifier, not a slice position, and the two come apart on
ordinary tables:
- `MetadataBuilder.RemovePartitionSpecs` filters the slice and never
renumbers the survivors, so removing a non-default spec that sits before the
default one leaves `specs[i].ID() != i`.
- Specs loaded from existing metadata keep the order of the
`partition-specs` JSON array (`slices.Clone`, no sort). The spec format
identifies each entry by `spec-id` and does not promise ascending order, so
metadata written by another engine can also break the `index == id` assumption.
Java is unaffected because it keys specs by ID in a `Map<Integer,
PartitionSpec>`.
`specs[df.SpecID()]` is evaluated as a call argument, so it fails before
`removeFile` ever runs. On such a table, any `Delete` / `Overwrite` /
`RewriteFiles` commit that removes files panics with `index out of range`, or —
when the ID happens to stay within bounds but points at the wrong entry —
silently records the changed-partition summary under a partition path derived
from the wrong spec.
**Reproduction**, using only public metadata updates: evolve to spec ids
0/1/2, make 2 the default, then `RemovePartitionSpecs([]int{1})` so `specs ==
[id0, id2]` (len 2). Now feed the *same* spec-2 data file to both branches of
the function:
| path | result |
| --- | --- |
| add | `partitions.id_trunc=0`, no error |
| remove | `panic: runtime error: index out of range [2] with length 2` |
Same file, same spec, same function — the only variable is which branch it
goes down.
```
panic: runtime error: index out of range [2] with length 2
github.com/apache/iceberg-go/table.(*snapshotProducer).accumulateSummaryDelta
table/snapshot_producers.go:1023
```
Expected: removed files resolve their spec by ID, exactly as added files
already do.
### Contribution
Working on a PR for the fix (regression test included).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
