JonasJ-ap commented on code in PR #6997:
URL: https://github.com/apache/iceberg/pull/6997#discussion_r1174077806
##########
python/pyiceberg/io/pyarrow.py:
##########
@@ -358,14 +368,17 @@ def field(self, field: NestedField, field_result:
pa.DataType) -> pa.Field:
name=field.name,
type=field_result,
nullable=field.optional,
- metadata={"doc": field.doc, "id": str(field.field_id)} if
field.doc else {},
+ metadata={FIELD_DOC: field.doc, FIELD_ID: str(field.field_id)} if
field.doc else {FIELD_ID: str(field.field_id)},
)
- def list(self, _: ListType, element_result: pa.DataType) -> pa.DataType:
- return pa.list_(value_type=element_result)
+ def list(self, list_type: ListType, element_result: pa.DataType) ->
pa.DataType:
+ element_field = self.field(list_type.element_field, element_result)
+ return pa.list_(value_type=element_field)
Review Comment:
I think this is also a bug in pyarrow, similar to the above one.
Based on examples in this upstream issue: [[Python] MapType comparison
ignores nullable flag of item_field · Issue #33574 · apache/arrow
(github.com)](https://github.com/apache/arrow/issues/33574) `pa.map_` accepts
`pa.field` as its key_value and map_value. I also confirmed this by looking at
the underlying C++ constructor
https://github.com/apache/arrow/blob/388f3a88c64750b3db77c5ca044ceae750e8db85/cpp/src/arrow/type.h#L1006-L1014
In terms of this PR, I think this test indicates the `pa.map_` can accept
`pa.field` as input
https://github.com/apache/iceberg/pull/6997/files#diff-320245c4e67f5814708bcf8aea113471b5d2e51e0e56b3eaae1a3bc1dfcbbfe7R303-R357
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org
For additional commands, e-mail: issues-h...@iceberg.apache.org
