123digits opened a new issue, #4066:
URL: https://github.com/apache/iceberg-python/issues/4066
### Apache Iceberg version
0.12.0 (latest release)
### Please describe the bug 🐞
`RestCatalog.update_namespace_properties` raises a pydantic
`ValidationError` when the server's response omits the `missing` field, even
though the update was applied. The REST spec makes `missing` optional, but
pyiceberg's response model requires it. The model is unchanged on `main`.
**The spec** (`open-api/rest-catalog-open-api.yaml`):
```yaml
UpdateNamespacePropertiesResponse:
type: object
required:
- updated
- removed
properties:
...
missing:
type: array
items:
type: string
description:
List of properties requested for removal that were not found
in the namespace's properties. Represents a partial success response.
Server's do not need to implement this.
nullable: true
```
**pyiceberg** (`pyiceberg/catalog/rest/__init__.py`, unchanged on `main`):
```python
class UpdateNamespacePropertiesResponse(IcebergBaseModel):
removed: list[str] = Field()
updated: list[str] = Field()
missing: list[str] = Field() # required, but optional per the spec
```
**Servers that omit it.**
[Lakekeeper](https://github.com/lakekeeper/lakekeeper) leaves `missing` out
when nothing was missing (`skip_serializing_if = "Option::is_none"`), as does
the `UpdateNamespacePropertiesResponse` type in `iceberg-rust`
(`crates/catalog/rest/src/types.rs`). So with Lakekeeper, any removal where
every requested key exists fails in pyiceberg.
**Reproduction** (Lakekeeper 0.13.4, pyiceberg 0.12.0):
```python
from pyiceberg.catalog.rest import RestCatalog
catalog = RestCatalog("lakekeeper", uri="http://localhost:8181/catalog";,
warehouse="demo")
catalog.create_namespace(("probe",), {"a": "1", "b": "2"})
catalog.update_namespace_properties(("probe",), removals={"a", "b"})
```
The server answers `200 {"updated":[],"removed":["a","b"]}` and applies the
change, but pyiceberg raises:
```
pydantic_core._pydantic_core.ValidationError: 1 validation error for
UpdateNamespacePropertiesResponse
missing
Field required [type=missing, input_value={'updated': [], 'removed': ['a',
'b']}, input_type=dict]
```
`load_namespace_properties(("probe",))` afterwards shows `a` and `b` are
gone. When at least one requested key is absent, the server includes
`"missing": [...]` and the call succeeds.
**Expected:** the call returns a `PropertiesUpdateSummary` with `missing=[]`
(or `None`).
**Suggested fix:** make the field optional, defaulting to empty, so
`PropertiesUpdateSummary` stays unchanged for callers:
```python
missing: list[str] = Field(default_factory=list)
```
### Willingness to contribute
- [ ] I can contribute a fix for this bug independently
- [ ] I would be willing to contribute a fix for this bug with guidance from
the Iceberg community
- [x] I cannot contribute a fix for this bug at this time
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
