mbutrovich commented on code in PR #7658:
URL: https://github.com/apache/datafusion-comet/pull/7658#discussion_r4634244718
##########
native/spark-expr/benches/check_overflow.rs:
##########
@@ -85,6 +97,41 @@ fn criterion_benchmark(c: &mut Criterion) {
c.bench_function("check_overflow: ansi no overflow", |b| {
b.iter(|| black_box(ansi.evaluate(black_box(&no_overflow)).unwrap()))
});
+ c.bench_function("check_overflow: ansi no overflow, with nulls", |b| {
+ b.iter(||
black_box(ansi.evaluate(black_box(&no_overflow_nulls)).unwrap()))
+ });
+
+ // One overflow in the last row, so the scan reads the whole batch before
the slow path runs.
+ // `sparse overflow` and `dense overflow` overflow at row 0 and stop in
the first chunk.
Review Comment:
The scan no longer works in chunks, so "stop in the first chunk" describes
the earlier fold. With the `position` loop, these two shapes stop at row 0.
```suggestion
// One overflow in the last row, so the scan reads the whole batch
before the slow path runs.
// `sparse overflow` and `dense overflow` overflow at row 0, where the
scan stops.
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]