CTTY commented on code in PR #1511:
URL: https://github.com/apache/iceberg-rust/pull/1511#discussion_r2219808529
##########
crates/iceberg/src/arrow/value.rs:
##########
@@ -440,10 +440,12 @@ impl PartnerAccessor<ArrayRef> for ArrowArrayAccessor {
Ok(schema_partner)
}
+ // todo generate field_pos in datafusion instead of passing to here
Review Comment:
This is a temporary hack to an issue that I don't know how exactly to fix
for now: the `RecordBatch` from Datafusion won't have
`PARQUET_FIELD_ID_META_KEY` in its schema's metadata, causing the schema
visiting to fail
[here](https://github.com/apache/iceberg-rust/blob/fbc3716c7eac6bba6f1902610407e82e925a83ba/crates/iceberg/src/arrow/schema.rs#L244).
I'm thinking maybe we can bound the schema in datafusion via name mapping,
but have not got the chance to explore more
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
