geruh commented on code in PR #2367:
URL: https://github.com/apache/iceberg-python/pull/2367#discussion_r2575143308
##########
pyiceberg/partitioning.py:
##########
@@ -222,11 +223,14 @@ def partition_type(self, schema: Schema) -> StructType:
:return: A StructType that represents the PartitionSpec, with a
NestedField for each PartitionField.
"""
nested_fields = []
+ schema_ids = index_by_id(schema)
for field in self.fields:
- source_type = schema.find_type(field.source_id)
- result_type = field.transform.result_type(source_type)
- required = schema.find_field(field.source_id).required
- nested_fields.append(NestedField(field.field_id, field.name,
result_type, required=required))
+ if field.source_id in schema_ids:
+ source_field = schema.find_field(field.source_id)
+ result_type =
field.transform.result_type(source_field.field_type)
+ nested_fields.append(NestedField(field.field_id, field.name,
result_type, required=source_field.required))
+ else:
Review Comment:
I think this is fine, if I understand the concern correctly. The worry is
that we're allowing unknown type for non void transforms when the source field
is missing.
On the Java side, when we're reading partition specs from the metadata, we
are using the allow missing fields equal to true, which skips the validation if
the source is
missing([here](https://github.com/apache/iceberg/blob/main/core/src/main/java/org/apache/iceberg/PartitionSpecParser.java#L71)).
This validation is only hit when we're constructing new partition specs.
When we're reading, it's fine.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
