dwsmith1983 commented on code in PR #6116:
URL: https://github.com/apache/datafusion-comet/pull/6116#discussion_r4078313858
##########
native/core/src/parquet/parquet_support.rs:
##########
@@ -414,6 +414,39 @@ fn field_id(field: &arrow::datatypes::Field) ->
Option<i32> {
.and_then(|v| v.parse::<i32>().ok())
}
+/// True when a field in `fields`, at any nesting depth, carries a Parquet
field id. Spark's
+/// `containsFieldIds` walks the whole file schema the same way, and
`ParquetUtils.hasFieldIds`
+/// walks the read schema. The root-only `schema_has_field_ids` in the schema
adapter stays as the
+/// gate for id matching, which only ever renames root fields.
+pub(crate) fn any_nested_field_has_id(fields: &Fields) -> bool {
Review Comment:
> Could you note in the doc comment that the element and key/value checks
are there for the physical schema?
Done in ba0d8d7c2. The comment now says the element and key, value and
entries checks serve the file-side walk that mirrors `containsFieldIds` over
the raw message type, that on the requested side only struct fields carry an
id, and that the requested-side answer is computed once at plan time.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
