clairemcginty commented on code in PR #2941:
URL: https://github.com/apache/parquet-java/pull/2941#discussion_r1666001736
##########
parquet-column/src/main/java/org/apache/parquet/internal/column/columnindex/ColumnIndexBuilder.java:
##########
@@ -371,7 +371,11 @@ public <T extends Comparable<T>> PrimitiveIterator.OfInt
visit(NotIn<T> notIn) {
@Override
public <T extends Comparable<T>> PrimitiveIterator.OfInt visit(Contains<T>
contains) {
- return contains.filter(this, IndexIterator::intersection,
IndexIterator::union);
+ return contains.filter(
Review Comment:
My initial idea was to implement `IndexIterator::subtract` for
`not(contains())` -- for example, `not(contains(eq("bar")))` would return
something like `IndexIterator.all(getPageCount()).subtract(visit(eq("bar"))`.
But we can't rely on the individual predicate implementations to return exact
row ranges -- for example, `NotIn` [always returns
IndexIterator.all](https://github.com/apache/parquet-java/blob/c26624fd8fc196f3d65edb0bcb0460a03cd163fd/parquet-column/src/main/java/org/apache/parquet/internal/column/columnindex/ColumnIndexBuilder.java#L369).
So I don't think it's possible to safely subtract any row ranges here.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
