RussellSpitzer commented on a change in pull request #1744:
URL: https://github.com/apache/iceberg/pull/1744#discussion_r520707087
##########
File path: core/src/main/java/org/apache/iceberg/avro/PruneColumns.java
##########
@@ -95,7 +95,7 @@ public Schema record(Schema record, List<String> names,
List<Schema> fields) {
if (hasChange) {
return copyRecord(record, filteredFields);
- } else if (filteredFields.size() == record.getFields().size()) {
+ } else if (record.getFields().size() != 0 && filteredFields.size() ==
record.getFields().size()) {
Review comment:
I tried to figure out if there was a simpler way to do this and I ended
up modifying BuildAvroProjection. In my mind the real problem here was we were
telling BuildAvroProjection what columns we wanted, but our pruned schema
really only showed us what columns we needed that have data. This only left
leaf empty-struct nodes. So during BuildAvroProjection we check for these
empty-structs being requested in the expected schema and just add them back.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
