rdblue commented on a change in pull request #2953:
URL: https://github.com/apache/iceberg/pull/2953#discussion_r707622367
##########
File path: core/src/main/java/org/apache/iceberg/avro/PruneColumns.java
##########
@@ -81,13 +81,18 @@ public Schema record(Schema record, List<String> names,
List<Schema> fields) {
Schema fieldSchema = fields.get(field.pos());
// All primitives are selected by selecting the field, but map and list
- // types can be selected by projecting the keys, values, or elements.
+ // types can be selected by projecting the keys, values, or elements.
Empty
+ // Structs can be selected by selecting the record itself instead of
it's children.
// This creates two conditions where the field should be selected: if the
// id is selected or if the result of the field is non-null. The only
// case where the converted field is non-null is when a map or list is
// selected by lower IDs.
if (selectedIds.contains(fieldId)) {
- filteredFields.add(copyField(field, field.schema(), fieldId));
+ if (fieldSchema != null) {
+ filteredFields.add(copyField(field, fieldSchema, fieldId));
Review comment:
I'm not sure that I understand the reason for this change. Is this
implementing the same change as the previous PR, but in the Avro `PruneColumns`?
It looks like if a struct field is selected and a sub-field is selected,
then the selection for the struct isn't a full selection. But if a sub-field is
not selected then the selection for the struct is a full selection. That
doesn't make sense to me.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
