anoopj commented on code in PR #3416:
URL: https://github.com/apache/iceberg-rust/pull/3416#discussion_r4253338308
##########
crates/iceberg/src/arrow/reader/row_filter.rs:
##########
@@ -975,6 +975,88 @@ mod tests {
)
}
+ /// A `FileScanTask` built or deserialized outside `TableScan` carries an
+ /// un-normalized predicate, so a raw `Not(...)` can reach the reader.
Driving
+ /// it through a full read with row-group filtering and row selection
enabled
+ /// exercises both the row-group and page-index evaluators, which negate
NOT
Review Comment:
Added `test_not_predicate_read_through_page_index`, to test this.
▎ but with a single row group and default writer props the row-group stage
prunes first and the page-index path almost certainly never runs
The default Arrow writer does emit a column index, so the page-index path
runs whenever a row group survives (the new test exercises exactly that).
Agreed it wasn't isolated in the original test, which is why the new one turns
row-group filtering off.
▎ NOT(a AND b)/NOT(a OR b), NOT(x IS NULL), NOT(StartsWith), and a NOT
arriving via the equality-delete combination
This PR only adds the single `rewrite_not()` call at the pipeline boundary;
it doesn't change rewrite_not. The per-shape behavior lives in `rewrite_not`
and is already unit-tested, so testing each shape end to end here would be at
the wrong layer and duplicate it. The equality-delete combination is the one
that actually touches reader code (the predicate is AND-ed with the delete
predicate before the rewrite), so I can add that one if you'd like.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
