unikdahal commented on issue #6588: URL: https://github.com/apache/datafusion-comet/issues/6588#issuecomment-5983718140
## Updated benchmark results I ran a more controlled benchmark with runtime filtering enabled and disabled from otherwise identical builds, over the same generated Iceberg tables. The dataset contains **16M rows across 16 files**, tested in three layouts: sorted, sorted with position deletes, and randomly ordered. Each result below is based on **18 warm runs across 6 rounds in separate JVMs**, with execution order rotated between variants. All variants returned identical results for every query. | Query | Main | Runtime filters ON | Speedup vs main (95% CI) | Reader bytes vs main | | --- | ---: | ---: | ---: | ---: | | Join, sorted | 956 ms | 107 ms | 8.92x (7.98-10.99) | 0.3% | | Join, position deletes | 988 ms | 113 ms | 8.75x (8.11-9.80) | 0.3% | | Join, unsorted | 955 ms | 342 ms | 2.79x (2.61-2.92) | 38.5% | | TopK, sorted | 991 ms | 142 ms | 6.97x (6.43-8.43) | 5.8% | | TopK, position deletes | 1,029 ms | 163 ms | 6.30x (5.97-7.24) | 6.5% | | TopK, unsorted | 994 ms | 296 ms | 3.36x (3.12-3.51) | 30.2% | | MIN, sorted | 99 ms | 66 ms | 1.51x (1.24-1.67) | 9.3% | | MIN, unsorted | 110 ms | 82 ms | 1.34x (1.23-1.61) | 21.7% | | No runtime producer (control) | 906 ms | 917 ms | 0.99x (0.96-1.02) | 100% | A few qualifications are worth calling out: - The runtime-filter-OFF build tracks `main`, suggesting that the dependency and plumbing changes themselves are neutral. - The control query is effectively unchanged. Scans without a runtime-filter producer do not gain pruning, and reader bytes remain at 100%. - The largest gains are on physically ordered data. The unsorted cases still improve, but substantially less, which is consistent with pruning only where file/row-group statistics allow it. - These are warm local-disk timings, so the elapsed-time numbers are workload-specific. Reader-byte reduction is the more storage-independent measure of avoided scan work. - The MIN queries are very short (~100 ms) and noisier (CV 14-18%), so I would put more weight on the byte reduction than the exact latency improvement there. The speedups are larger than I initially expected, particularly for the sorted join and TopK cases. However, the neutral control, the runtime-filter-OFF comparison against `main`, and the smaller gains on randomly ordered data all line up with the expected behavior of runtime pruning. I'm treating these results as validation that the runtime-predicate path is working as intended, rather than as a claim that arbitrary workloads should expect 6-9x speedups. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
