Shruthirdy opened a new issue, #2452:
URL: https://github.com/apache/iceberg-go/issues/2452
### Apache Iceberg version
main (development)
### Please describe the bug 🐞
If a table's metadata has a `last-column-id` lower than a field ID that's
already in use, adding a column with `UpdateSchema` panics instead of returning
an error.
`UpdateSchema` sets its ID counter from `last-column-id`, so the new column
gets an ID that already exists. `Apply()` then passes the schema to
`iceberg.NewSchema`, which panics on duplicate IDs. `Apply()` and `Commit()`
both usually return an `error`, so callers expect a failure to come back as an
error, not a panic.
Parsing accepts corrupted metadata, so with a corrupted metadata the table
loads normally, and the crash only happens on the first schema update.
### Repro
Take metadata where the highest field ID is 3 (for example
`ExampleTableMetadataV2`, which has fields `x`=1, `y`=2, `z`=3) and change
`"last-column-id": 3` to `"last-column-id": 2`. Then:
```go
meta, err := table.ParseMetadataString(metadataJSON) // parses fine, err ==
nil
tbl := table.New(table.Identifier{"db", "t"}, meta, "", nil, nil)
_, err = tbl.NewTransaction().
UpdateSchema(true, false).
AddColumn([]string{"w"}, iceberg.PrimitiveTypes.Int32, "", false, nil).
Apply() // panics (same goes with Commit())
```
### Expected behavior
`Apply()` / `Commit()` return an error saying `last-column-id` is invalid.
### Actual behavior
```
panic: invalid schema: multiple fields for id 3: z and w
```
### Additional context
- `MetadataBuilder` already has `validateLastColumnID()`, which reports this
exact case as `ErrInvalidMetadata`, so one option is to run that check in
`NewUpdateSchema`.
- I found this while testing `UpdateSchema` with deliberately corrupted
metadata.
I already have a fix for this and would be happy to open a PR for this.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
