rdblue commented on a change in pull request #2953:
URL: https://github.com/apache/iceberg/pull/2953#discussion_r707644068
##########
File path: core/src/main/java/org/apache/iceberg/avro/PruneColumns.java
##########
@@ -81,13 +81,18 @@ public Schema record(Schema record, List<String> names,
List<Schema> fields) {
Schema fieldSchema = fields.get(field.pos());
// All primitives are selected by selecting the field, but map and list
- // types can be selected by projecting the keys, values, or elements.
+ // types can be selected by projecting the keys, values, or elements.
Empty
+ // Structs can be selected by selecting the record itself instead of
it's children.
// This creates two conditions where the field should be selected: if the
// id is selected or if the result of the field is non-null. The only
// case where the converted field is non-null is when a map or list is
// selected by lower IDs.
if (selectedIds.contains(fieldId)) {
- filteredFields.add(copyField(field, field.schema(), fieldId));
+ if (fieldSchema != null) {
+ filteredFields.add(copyField(field, fieldSchema, fieldId));
Review comment:
As I'm thinking about this more, I think that the behavior in this class
should always match `project`. I doubt there's a case where we want select
behavior, right? In that case, shouldn't the `else` case check whether the type
is a record and create an empty record?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
