kosiew commented on code in PR #24130:
URL: https://github.com/apache/datafusion/pull/24130#discussion_r3783106990
##########
datafusion/datasource-parquet/src/nested_schema_pruning.rs:
##########
@@ -128,6 +128,35 @@ pub(crate) fn clip_for_cast(
Some((kept, pruned_type))
}
+/// Rebuilds `physical` for a sorted, deduplicated subset of its leaf offsets.
+///
+/// This is the second half of merging several nested accesses to one root:
+/// callers union the offsets required by each access, then use this function
+/// to derive the Arrow type the Parquet reader emits for that union. Partial
+/// projection is supported through the same struct, list, and large-list
+/// shapes as [`clip_for_cast`]. Any partial selection below another wrapper
+/// returns `None`, preserving the total-fallback property of cast clipping.
+pub(crate) fn type_for_leaf_subset(
Review Comment:
Could we add a small unit test for the case where the union covers every
leaf? `type_for_leaf_subset` intentionally returns `None` there so the caller
falls back to the full root. It would be nice to pin that behavior explicitly
and avoid a future change treating a full union as an unsupported partial
projection.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
