EeshanBembi commented on PR #17554:
URL: https://github.com/apache/datafusion/pull/17554#issuecomment-3343047310
> @EeshanBembi, I am curious about the performance impact of enabling
overflow checking by default. Could you add criterion benchmarks?
Hi Andy! I've implemented the criterion benchmarks you requested. Here are
the performance results:
Overflow Checking Performance Impact:
- Small numbers: 9-14% overhead (~25-30ns per operation)
- Medium numbers: ~12% overhead
- Close to overflow: ~9% overhead
- Actual overflow cases: 36% faster (early error detection vs wrapping)
Benchmark Details:
- Added comprehensive criterion benchmarks in
datafusion/physical-expr/benches/binary_op.rs
- Tests scenarios: small numbers, medium numbers, close-to-overflow, and
guaranteed overflow
- Uses feature comparison methodology (WITH vs WITHOUT overflow checking)
- Run with: cargo bench --bench binary_op -- feature_comparison
Key Findings:
- Overhead is minimal and acceptable for SQL-standard compliance
- Smart optimizations (branch-free detection, SIMD) keep impact low
- Error cases are actually faster due to early exit vs expensive wrapping
- Real-world impact: ~25-30ms overhead per 1M operations
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
