mukul-mpac opened a new issue, #2786:
URL: https://github.com/apache/iceberg-python/issues/2786
## Bug Report
### Description
When using `update_schema().update_column()` to update the `required`
property of list elements, the field type updates successfully but the
`required` property does not change.
### Environment
- **PyIceberg version:** 0.10.0
- **Python version:** 3.12.12
- **Platform:** macOS
### Root Cause
Located in `pyiceberg/table/update/schema.py:112` in the
`_ApplyChanges.list()` method:
```python
return ListType(element_id=list_type.element_id, element=element_type,
element_required=list_type.element_required) # Always uses
original value
```
The method always uses the original `element_required` value instead of
checking if the element field has been updated in `self._updates`.
### Minimal Reproduction
```python
from pyiceberg.types import ListType, StringType
# Create table with list field where elements are optional
(element_required=False)
with iceberg_table.update_schema() as update_schema:
update_schema.add_column(
path="test_list_primitive",
field_type=ListType(element_type=StringType(),
element_required=False),
)
# Try to update element to be required
with iceberg_table.update_schema(allow_incompatible_changes=True) as
update_schema:
update_schema.update_column(
path=("test_list_primitive", "element"),
required=True, # This doesn't work
)
# Check schema - element_required is still False
field = iceberg_table.schema().find_field("test_list_primitive")
print(field.field_type.element_required) # Still False, should be True
```
### Expected Behavior
The `element_required` property should be updated to `True`.
### Actual Behavior
The `element_required` property remains `False` (unchanged).
### Proposed Fix
```python
def list(self, list_type: ListType) -> ListType:
element_type = self._visit_type(list_type.element_type)
element_required = list_type.element_required
# Check if element field has been updated
if update := self._updates.get(list_type.element_id):
element_required = update.required
return ListType(
element_id=list_type.element_id,
element=element_type,
element_required=element_required
)
```
### Workaround
Drop and recreate the list field:
```python
# Step 1: Drop
with iceberg_table.update_schema(allow_incompatible_changes=True) as
update_schema:
update_schema.delete_column("test_list_primitive")
# Step 2: Recreate with element_required=True
with iceberg_table.update_schema(allow_incompatible_changes=True) as
update_schema:
update_schema.add_column(
path="test_list_primitive",
field_type=ListType(element_type=StringType(),
element_required=True),
)
```
### Additional Notes
A similar issue likely exists for `MapType.value_required` in the `map()`
method of the same class.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
