rdblue commented on a change in pull request #2953:
URL: https://github.com/apache/iceberg/pull/2953#discussion_r711792732
##########
File path: core/src/main/java/org/apache/iceberg/avro/PruneColumns.java
##########
@@ -81,13 +83,22 @@ public Schema record(Schema record, List<String> names,
List<Schema> fields) {
Schema fieldSchema = fields.get(field.pos());
// All primitives are selected by selecting the field, but map and list
- // types can be selected by projecting the keys, values, or elements.
+ // types can be selected by projecting the keys, values, or elements.
Empty
+ // Structs can be selected by selecting the record itself instead of its
children.
// This creates two conditions where the field should be selected: if the
// id is selected or if the result of the field is non-null. The only
// case where the converted field is non-null is when a map or list is
// selected by lower IDs.
if (selectedIds.contains(fieldId)) {
- filteredFields.add(copyField(field, field.schema(), fieldId));
+ if (fieldSchema != null) {
+ filteredFields.add(copyField(field, fieldSchema, fieldId));
+ } else {
+ if (isRecord(field.schema())) {
+ filteredFields.add(copyField(field, makeEmptyCopy(field.schema()),
fieldId));
+ } else {
+ filteredFields.add(copyField(field, field.schema(), fieldId));
Review comment:
Okay, so in this case the field is a map, list, or primitive. Then we
just follow the old behavior. Looks good to me since the Iceberg schema
selection would fail.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
