rdblue commented on a change in pull request #1566:
URL: https://github.com/apache/iceberg/pull/1566#discussion_r516903215
##########
File path: parquet/src/main/java/org/apache/iceberg/parquet/ReadConf.java
##########
@@ -85,34 +84,28 @@
this.projection = ParquetSchemaUtil.pruneColumnsFallback(fileSchema,
expectedSchema);
}
+ // ParquetFileReader has filters(stats, dictionary and future bloomfilter)
in the constructor,
+ // so getRowGroups returns filtered row groups
this.rowGroups = reader.getRowGroups();
- this.shouldSkip = new boolean[rowGroups.size()];
// Fetch all row groups starting positions to compute the row offsets of
the filtered row groups
Map<Long, Long> offsetToStartPos = generateOffsetToStartPos();
this.startRowPositions = new long[rowGroups.size()];
- ParquetMetricsRowGroupFilter statsFilter = null;
- ParquetDictionaryRowGroupFilter dictFilter = null;
- if (filter != null) {
- statsFilter = new ParquetMetricsRowGroupFilter(expectedSchema, filter,
caseSensitive);
- dictFilter = new ParquetDictionaryRowGroupFilter(expectedSchema, filter,
caseSensitive);
- }
Review comment:
This PR removes support for pushing down filters like `startsWith` and
`in`?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
