hhhizzz commented on PR #10852: URL: https://github.com/apache/arrow-rs/pull/10852#issuecomment-5435965297
I'm not sure how to run whole bench in this PR, So I run them on my dev server, here's the main results: ### End-to-end benchmark results I ran three cyclic-order rounds against Arrow main using the same DataFusion integration harness. The first iteration of each query was excluded, and the table reports the cross-round median of steady-state query times. | Suite | Main OFF | Main ON | Candidate ON | Candidate vs Main ON | Query geomean | |---|---:|---:|---:|---:|---:| | ClickBench | 41.394 s | 35.932 s | 36.280 s | +0.97% | -4.76% | | TPC-H SF10 | 9.549 s | 8.265 s | 7.992 s | -3.31% | -5.02% | | TPC-DS SF10 | 73.218 s | 46.051 s | 44.450 s | -3.48% | -4.12% | Selected queries: | Suite | Query | Main OFF | Main ON | Candidate ON | Candidate vs Main ON | Candidate vs OFF | |---|---:|---:|---:|---:|---:|---:| | ClickBench | Q25 | 180.4 ms | 283.5 ms | 172.8 ms | **-39.0%** | **-4.2%** | | ClickBench | Q12 | 552.0 ms | 671.4 ms | 567.9 ms | **-15.4%** | +2.9% | | ClickBench | Q23 | 7,738.4 ms | 857.3 ms | 853.4 ms | -0.5% | **-89.0%** | | ClickBench | Q21 | 834.5 ms | 1,592.3 ms | 1,570.2 ms | -1.4% | +88.2% | | TPC-H | Q6 | 122.7 ms | 324.0 ms | 239.0 ms | **-26.2%** | +94.8% | | TPC-H | Q18 | 3,136.8 ms | 871.1 ms | 859.1 ms | -1.4% | **-72.6%** | | TPC-DS | Q9 | 279.3 ms | 1,198.3 ms | 468.6 ms | **-60.9%** | +67.8% | | TPC-DS | Q72 | 31,044.1 ms | 1,376.0 ms | 1,255.3 ms | **-8.8%** | **-96.0%** | | TPC-DS | Q14 | 1,777.4 ms | 1,622.4 ms | 1,801.8 ms | +11.1% | +1.4% | | TPC-DS | Q50 | 280.9 ms | 500.2 ms | 553.4 ms | +10.6% | +97.1% | The candidate fixes the target Q25 regression, preserves the major pushdown wins such as ClickBench Q23, TPC-H Q18, and TPC-DS Q72, and reduces several other Main-ON regressions. It is not a pure monotonic improvement: regressions remain on queries such as ClickBench Q21 and TPC-DS Q14/Q50, and pushdown is still not universally faster than OFF. All query row counts matched across every arm and round. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
