bryanck commented on code in PR #18336:
URL: https://github.com/apache/iceberg/pull/18336#discussion_r4156446648
##########
parquet/src/main/java/org/apache/iceberg/parquet/PruneColumns.java:
##########
@@ -91,9 +91,17 @@ public Type struct(StructType expected, GroupType struct,
List<Type> fields) {
Type field = fields.get(i);
Integer fieldId = getId(originalField);
if (fieldId != null && selectedIds.contains(fieldId)) {
- filteredFields.add(originalField);
+ // the field id is in selectedIds both when the struct itself is
projected (field ==
+ // originalField) and when it is only a path to a deeper projected
field (field is pruned);
+ // adding the pruned field keeps deep projections from widening back
to the full struct
Review Comment:
I feel it is OK to leave this somewhat long comment, given the behavior
isn't obvious
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
