amogh-jahagirdar commented on code in PR #411:
URL: https://github.com/apache/iceberg-python/pull/411#discussion_r1485409963
##########
pyiceberg/table/metadata.py:
##########
@@ -260,8 +260,10 @@ def set_v2_compatible_defaults(cls, data: Dict[str, Any])
-> Dict[str, Any]:
The TableMetadata with the defaults applied.
"""
# When the schema doesn't have an ID
- if data.get("schema") and "schema_id" not in data["schema"]:
- data["schema"]["schema_id"] = DEFAULT_SCHEMA_ID
+ schema = data.get("schema")
+ if schema and isinstance(schema, dict):
+ if "schema_id" not in data["schema"]:
+ data["schema"]["schema_id"] = DEFAULT_SCHEMA_ID
Review Comment:
When I explicitly pass in a schema it's no longer a dictionary. Then this
validator fails since data["schema"] is an acutal Schema and cannot be
referenced. I'm not really familiar with Pydantic but my understanding of this
validator is that it's meant for just converting the schema metadata a common
representation that can be shared with V2 metadata. I think that should only
happen when reading the actual JSON file.
I'm still working through another issue for partition specs in V1 (this
error is thrown in the updated test):
```
pydantic_core._pydantic_core.ValidationError: 2 validation errors for
TableMetadataV1
partition_spec.0
Input should be a valid dictionary [type=dict_type, input_value=('spec_id',
0), input_type=tuple]
For further information visit https://errors.pydantic.dev/2.6/v/dict_type
partition_spec.1
Input should be a valid dictionary [type=dict_type, input_value=('fields',
()), input_type=tuple]
For further information visit https://errors.pydantic.dev/2.6/v/dict_type
```
I'm not sure why this happens for V1 metadata explicitly constructed but V2
works fine.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org
For additional commands, e-mail: issues-h...@iceberg.apache.org
