rahulsmahadev opened a new pull request, #4391:
URL: https://github.com/apache/iceberg-python/pull/4391
# Rationale for this change
`rename_field` gives the wrong result when the new name belongs to a void
partition field. A void field keeps its name after `remove_field` on a v1
table. On a v2 table, an explicit void field (`add_field` with `VoidTransform`)
keeps its name too. Renaming a live field onto that name should give the live
field the requested name and move the void field out of the way, as Java does.
Instead, the live field is renamed to `<old name>_<void field id>` and the
requested name is dropped:
```python
# v1 table: identity(x) (field id 1000), bucket[16](y) named y_shard (field
id 1001)
table.update_spec().remove_field("y_shard").commit()
table.update_spec().rename_field("x", "y_shard").commit()
# before: [(1000, "x_1001", identity), (1001, "y_shard", void)]
# after: [(1000, "y_shard", identity), (1001, "y_shard_1001", void)]
```
The cause is the recursive call in `UpdateSpec.rename_field`. For a void
target it called `rename_field(name, name + "_" + field_id)`. That renames the
live field and returns, so the requested rename is never recorded. The fix
renames the void field to `<new name>_<field id>` and then records the
requested rename. This is the same sequence as
`BaseUpdatePartitionSpec.renameField` in Java. Related: #4307 handles removing
and adding a field with the same name in one update; this change handles
renaming onto an existing void field's name. `add_field` also calls
`rename_field`, so nested void-name collisions use the corrected sequence.
Field ids, source ids and transforms do not change, only the names do.
## Are these changes tested?
Yes. Five tests in `tests/table/test_init.py`:
- `test_update_spec_rename_onto_removed_partition_field_name` (runs for each
`catalog` fixture variant): a v1 `remove_field`, then a rename onto the removed
name. It reloads the table from the catalog and checks names, field ids, source
ids and transforms.
- `test_update_spec_rename_onto_explicit_void_partition_field_name` (v2
table): an explicit void field, then a rename onto its name.
- `test_update_spec_rename_to_unused_partition_field_name`: a control that
renames onto an unused name, which must keep working as before.
- `test_update_spec_rename_onto_void_rejects_removal_in_same_update`: checks
both operation orders on v1 and v2.
- `test_update_spec_cascades_void_name_collisions`: checks nested void-name
collisions through both `rename_field` and `add_field`.
Without this change, the two rename tests fail with the live field named
`x_1001`, the removal test fails because nothing is raised, and the cascade
test fails on the resulting names. All five tests pass with this change, and
the unused-name control passes on both.
## Are there any user-facing changes?
Yes: renaming onto the name of a void partition field now gives that name to
the renamed field, and the void field becomes `<name>_<field id>`. Removing the
existing void field and then renaming onto its name in the same update now
raises `Cannot delete and rename partition field <name>`, and the reverse order
raises `Cannot rename and delete field <name>`. Both checks match Java; before,
both sequences committed the wrong name for the live field.
This pull request and its description were written by Isaac.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]