simonvandel commented on PR #9346:
URL:
https://github.com/apache/arrow-datafusion/pull/9346#issuecomment-1969097864
> I re-ran benchmarks in my machine. The results are below
Sorry for the noise, I must have somehow messed up. I also get green numbers
now.
I wanted to merge main into this PR to get a direct comparison, but there
are some conflicts.
So this is only comparing the PR head with the main head.
Baseline: b220f03fffda22c70f03fa84e244cf04f0e6644c
This PR commit: a8fac85429f519ea5ae258a2a6425eaa9ab333c8
I guess it would be good to do a final run when this PR merges cleanly with
main.
<details>
<summary>Results</summary>
```
logical_select_one_from_700
time: [571.99 µs 573.71 µs 575.87 µs]
change: [-2.3965% -1.9565% -1.5211%] (p = 0.00 <
0.05)
Performance has improved.
Found 10 outliers among 100 measurements (10.00%)
1 (1.00%) low mild
4 (4.00%) high mild
5 (5.00%) high severe
physical_select_one_from_700
time: [4.0660 ms 4.0824 ms 4.1013 ms]
change: [-2.8819% -1.9879% -1.1440%] (p = 0.00 <
0.05)
Performance has improved.
Found 5 outliers among 100 measurements (5.00%)
4 (4.00%) high mild
1 (1.00%) high severe
logical_trivial_join_low_numbered_columns
time: [591.08 µs 593.13 µs 595.76 µs]
change: [-6.5247% -2.7512% -0.3801%] (p = 0.09 >
0.05)
No change in performance detected.
Found 8 outliers among 100 measurements (8.00%)
2 (2.00%) high mild
6 (6.00%) high severe
logical_trivial_join_high_numbered_columns
time: [633.10 µs 634.01 µs 634.98 µs]
change: [-3.4749% -2.6612% -1.9606%] (p = 0.00 <
0.05)
Performance has improved.
Found 4 outliers among 100 measurements (4.00%)
1 (1.00%) low mild
1 (1.00%) high mild
2 (2.00%) high severe
Benchmarking logical_aggregate_with_join: Warming up for 3.0000 s
Warning: Unable to complete 100 samples in 5.0s. You may wish to increase
target time to 5.5s, enable flat sampling, or reduce sample count to 60.
logical_aggregate_with_join
time: [1.0910 ms 1.0924 ms 1.0939 ms]
change: [-3.0224% -2.1454% -1.3335%] (p = 0.00 <
0.05)
Performance has improved.
Found 6 outliers among 100 measurements (6.00%)
2 (2.00%) high mild
4 (4.00%) high severe
physical_plan_tpch_q1 time: [7.8388 ms 7.8798 ms 7.9215 ms]
change: [-5.4862% -4.6508% -3.8017%] (p = 0.00 <
0.05)
Performance has improved.
physical_plan_tpch_q2 time: [11.599 ms 11.660 ms 11.732 ms]
change: [-29.491% -28.931% -28.370%] (p = 0.00 <
0.05)
Performance has improved.
Found 1 outliers among 100 measurements (1.00%)
1 (1.00%) high severe
physical_plan_tpch_q3 time: [3.8482 ms 3.8557 ms 3.8645 ms]
change: [-22.953% -22.571% -22.194%] (p = 0.00 <
0.05)
Performance has improved.
Found 4 outliers among 100 measurements (4.00%)
1 (1.00%) high mild
3 (3.00%) high severe
physical_plan_tpch_q4 time: [2.9394 ms 2.9436 ms 2.9481 ms]
change: [-13.579% -13.302% -13.037%] (p = 0.00 <
0.05)
Performance has improved.
Found 5 outliers among 100 measurements (5.00%)
3 (3.00%) high mild
2 (2.00%) high severe
physical_plan_tpch_q5 time: [5.7565 ms 5.7768 ms 5.7990 ms]
change: [-58.072% -57.831% -57.580%] (p = 0.00 <
0.05)
Performance has improved.
Found 8 outliers among 100 measurements (8.00%)
4 (4.00%) high mild
4 (4.00%) high severe
Benchmarking physical_plan_tpch_q6: Warming up for 3.0000 s
Warning: Unable to complete 100 samples in 5.0s. You may wish to increase
target time to 9.3s, enable flat sampling, or reduce sample count to 50.
physical_plan_tpch_q6 time: [1.8326 ms 1.8348 ms 1.8373 ms]
change: [-4.3461% -3.8854% -3.3835%] (p = 0.00 <
0.05)
Performance has improved.
Found 8 outliers among 100 measurements (8.00%)
3 (3.00%) high mild
5 (5.00%) high severe
physical_plan_tpch_q7 time: [8.1415 ms 8.1731 ms 8.2058 ms]
change: [-45.628% -45.283% -44.930%] (p = 0.00 <
0.05)
Performance has improved.
physical_plan_tpch_q8 time: [11.600 ms 11.647 ms 11.697 ms]
change: [-72.389% -72.217% -72.048%] (p = 0.00 <
0.05)
Performance has improved.
Found 1 outliers among 100 measurements (1.00%)
1 (1.00%) high mild
physical_plan_tpch_q9 time: [8.7531 ms 8.7829 ms 8.8135 ms]
change: [-47.833% -47.538% -47.232%] (p = 0.00 <
0.05)
Performance has improved.
physical_plan_tpch_q10 time: [5.9565 ms 5.9761 ms 5.9972 ms]
change: [-31.327% -30.949% -30.571%] (p = 0.00 <
0.05)
Performance has improved.
Found 6 outliers among 100 measurements (6.00%)
5 (5.00%) high mild
1 (1.00%) high severe
physical_plan_tpch_q11 time: [4.7788 ms 4.7900 ms 4.8030 ms]
change: [-23.407% -20.882% -18.549%] (p = 0.00 <
0.05)
Performance has improved.
Found 7 outliers among 100 measurements (7.00%)
3 (3.00%) high mild
4 (4.00%) high severe
physical_plan_tpch_q12 time: [3.8070 ms 3.8132 ms 3.8200 ms]
change: [-13.715% -13.448% -13.212%] (p = 0.00 <
0.05)
Performance has improved.
Found 6 outliers among 100 measurements (6.00%)
4 (4.00%) high mild
2 (2.00%) high severe
physical_plan_tpch_q13 time: [2.4866 ms 2.4904 ms 2.4944 ms]
change: [-21.541% -21.256% -20.973%] (p = 0.00 <
0.05)
Performance has improved.
Found 2 outliers among 100 measurements (2.00%)
2 (2.00%) high mild
physical_plan_tpch_q14 time: [3.2541 ms 3.2599 ms 3.2664 ms]
change: [-2.6579% -2.3979% -2.1429%] (p = 0.00 <
0.05)
Performance has improved.
Found 7 outliers among 100 measurements (7.00%)
5 (5.00%) high mild
2 (2.00%) high severe
physical_plan_tpch_q16 time: [4.9112 ms 4.9275 ms 4.9454 ms]
change: [-18.165% -17.709% -17.242%] (p = 0.00 <
0.05)
Performance has improved.
Found 12 outliers among 100 measurements (12.00%)
6 (6.00%) high mild
6 (6.00%) high severe
physical_plan_tpch_q17 time: [4.7187 ms 4.7405 ms 4.7678 ms]
change: [-6.2345% -5.6606% -5.0112%] (p = 0.00 <
0.05)
Performance has improved.
Found 9 outliers among 100 measurements (9.00%)
4 (4.00%) high mild
5 (5.00%) high severe
physical_plan_tpch_q18 time: [5.1662 ms 5.1835 ms 5.2024 ms]
change: [-10.562% -10.086% -9.5753%] (p = 0.00 <
0.05)
Performance has improved.
Found 4 outliers among 100 measurements (4.00%)
2 (2.00%) high mild
2 (2.00%) high severe
physical_plan_tpch_q19 time: [9.6546 ms 9.6977 ms 9.7416 ms]
change: [-3.6676% -3.0457% -2.4012%] (p = 0.00 <
0.05)
Performance has improved.
physical_plan_tpch_q20 time: [6.1156 ms 6.1346 ms 6.1554 ms]
change: [-12.986% -12.532% -12.057%] (p = 0.00 <
0.05)
Performance has improved.
Found 3 outliers among 100 measurements (3.00%)
2 (2.00%) high mild
1 (1.00%) high severe
physical_plan_tpch_q21 time: [9.0939 ms 9.1482 ms 9.2128 ms]
change: [-34.273% -33.805% -33.250%] (p = 0.00 <
0.05)
Performance has improved.
Found 2 outliers among 100 measurements (2.00%)
2 (2.00%) high severe
physical_plan_tpch_q22 time: [4.3380 ms 4.3443 ms 4.3512 ms]
change: [-14.344% -14.025% -13.729%] (p = 0.00 <
0.05)
Performance has improved.
Found 3 outliers among 100 measurements (3.00%)
2 (2.00%) high mild
1 (1.00%) high severe
physical_plan_tpch_all time: [12.851 ms 12.870 ms 12.890 ms]
change: [-0.5789% -0.3165% -0.0670%] (p = 0.02 <
0.05)
Change within noise threshold.
Found 2 outliers among 100 measurements (2.00%)
1 (1.00%) high mild
1 (1.00%) high severe
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscr...@arrow.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
