rahulsmahadev opened a new issue, #4373:
URL: https://github.com/apache/iceberg-python/issues/4373
### Apache Iceberg version
main (development)
### Please describe the bug
Renaming a struct column that contains an identifier field fails at commit:
```python
schema = Schema(
NestedField(1, "user", StructType(NestedField(2, "id", LongType(),
required=True)), required=True),
identifier_field_ids=[2],
)
tbl = catalog.create_table("db.t", schema=schema)
with tbl.update_schema() as update:
update.rename_column("user", "account")
# ValueError: Cannot find identifier field user.id. In case of deletion,
update the identifier fields first.
```
The rename keeps every field ID, and the spec stores identifier fields by
ID, so the identifier should follow the rename. `UpdateSchema` keeps the
identifier fields as dotted names while the update is staged, and
`rename_column` only updates a name that equals the renamed column itself, so a
renamed parent struct leaves `user.id` behind. Java's
`SchemaUpdate.renameColumn` has the same exact-name check.
I tried a few fixes and would like a direction before opening a PR, because
the names-while-staging design makes each one trade something:
1. Rewrite the names nested under the renamed column in `rename_column`.
Small, but staged names can collide, for example when two structs swap names,
or when a case-insensitive update meets columns that differ only in case. Some
of these updates work today.
2. Keep identifier fields by field ID while staging. Renames then just work,
but `set_identifier_fields` needs a rule for which schema its names refer to
(the starting schema like the other `UpdateSchema` methods, or the final one
like Java), and either rule changes the result of some updates that work today.
3. Keep today's name lookup, and fall back to field IDs recorded at rename
time only when a name no longer resolves. Updates that work today give the same
result, but the fallback needs care so that it never picks a different field,
which takes a few dozen lines.
Would you prefer 2, or 3 to stay compatible?
### Willingness to contribute
- [x] I can contribute a fix for this bug independently
This issue was written by Isaac.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]