haohuaijin commented on PR #9595:
URL:
https://github.com/apache/arrow-datafusion/pull/9595#issuecomment-2003583033
this is a benchmark result; the baseline is
cf0f8eececd37f593b811320f89c0edd00fd3945 in the main.
```
logical_select_one_from_700
time: [636.48 µs 637.79 µs 639.20 µs]
change: [+17.618% +18.087% +18.592%] (p = 0.00 <
0.05)
Performance has regressed.
Found 9 outliers among 100 measurements (9.00%)
1 (1.00%) low severe
2 (2.00%) low mild
5 (5.00%) high mild
1 (1.00%) high severe
physical_select_one_from_700
time: [3.7188 ms 3.7238 ms 3.7295 ms]
change: [+24.347% +24.897% +25.417%] (p = 0.00 <
0.05)
Performance has regressed.
Found 5 outliers among 100 measurements (5.00%)
4 (4.00%) high mild
1 (1.00%) high severe
logical_select_all_from_1000
time: [23.270 ms 23.296 ms 23.324 ms]
change: [-72.737% -72.690% -72.643%] (p = 0.00 <
0.05)
Performance has improved.
Found 8 outliers among 100 measurements (8.00%)
6 (6.00%) high mild
2 (2.00%) high severe
Benchmarking physical_select_all_from_1000: Warming up for 3.0000 s
Warning: Unable to complete 100 samples in 5.0s. You may wish to increase
target time to 16.1s, or reduce sample count to 30.
physical_select_all_from_1000
time: [161.09 ms 161.35 ms 161.63 ms]
change: [-74.231% -74.190% -74.138%] (p = 0.00 <
0.05)
Performance has improved.
Found 16 outliers among 100 measurements (16.00%)
6 (6.00%) high mild
10 (10.00%) high severe
logical_trivial_join_low_numbered_columns
time: [602.80 µs 603.92 µs 605.05 µs]
change: [+6.0809% +6.6720% +7.2644%] (p = 0.00 <
0.05)
Performance has regressed.
Found 6 outliers among 100 measurements (6.00%)
2 (2.00%) low mild
2 (2.00%) high mild
2 (2.00%) high severe
logical_trivial_join_high_numbered_columns
time: [623.12 µs 624.41 µs 625.75 µs]
change: [+2.5065% +2.9476% +3.3981%] (p = 0.00 <
0.05)
Performance has regressed.
Found 2 outliers among 100 measurements (2.00%)
1 (1.00%) high mild
1 (1.00%) high severe
Benchmarking logical_aggregate_with_join: Warming up for 3.0000 s
Warning: Unable to complete 100 samples in 5.0s. You may wish to increase
target time to 5.7s, enable flat sampling, or reduce sample count to 60.
logical_aggregate_with_join
time: [1.1193 ms 1.1213 ms 1.1235 ms]
change: [+15.657% +16.111% +16.610%] (p = 0.00 <
0.05)
Performance has regressed.
Found 11 outliers among 100 measurements (11.00%)
2 (2.00%) low mild
2 (2.00%) high mild
7 (7.00%) high severe
physical_plan_tpch_q1 time: [6.7090 ms 6.7185 ms 6.7288 ms]
change: [-4.6107% -4.3279% -4.0398%] (p = 0.00 <
0.05)
Performance has improved.
Found 4 outliers among 100 measurements (4.00%)
3 (3.00%) high mild
1 (1.00%) high severe
physical_plan_tpch_q2 time: [10.510 ms 10.526 ms 10.544 ms]
change: [-0.3679% -0.1707% +0.0234%] (p = 0.10 >
0.05)
No change in performance detected.
Found 7 outliers among 100 measurements (7.00%)
4 (4.00%) high mild
3 (3.00%) high severe
physical_plan_tpch_q3 time: [3.5859 ms 3.5923 ms 3.5996 ms]
change: [+4.4629% +4.7337% +5.0124%] (p = 0.00 <
0.05)
Performance has regressed.
Found 6 outliers among 100 measurements (6.00%)
5 (5.00%) high mild
1 (1.00%) high severe
physical_plan_tpch_q4 time: [2.4776 ms 2.4814 ms 2.4856 ms]
change: [-6.8969% -6.6831% -6.4669%] (p = 0.00 <
0.05)
Performance has improved.
Found 7 outliers among 100 measurements (7.00%)
5 (5.00%) high mild
2 (2.00%) high severe
physical_plan_tpch_q5 time: [5.4360 ms 5.4433 ms 5.4511 ms]
change: [+7.1262% +7.4714% +7.7929%] (p = 0.00 <
0.05)
Performance has regressed.
Found 7 outliers among 100 measurements (7.00%)
4 (4.00%) high mild
3 (3.00%) high severe
Benchmarking physical_plan_tpch_q6: Warming up for 3.0000 s
Warning: Unable to complete 100 samples in 5.0s. You may wish to increase
target time to 8.5s, enable flat sampling, or reduce sample count to 50.
physical_plan_tpch_q6 time: [1.6593 ms 1.6620 ms 1.6651 ms]
change: [+3.6049% +4.0420% +4.4263%] (p = 0.00 <
0.05)
Performance has regressed.
Found 15 outliers among 100 measurements (15.00%)
10 (10.00%) high mild
5 (5.00%) high severe
physical_plan_tpch_q7 time: [7.1784 ms 7.1881 ms 7.1990 ms]
change: [-0.9565% -0.7731% -0.5913%] (p = 0.00 <
0.05)
Change within noise threshold.
Found 9 outliers among 100 measurements (9.00%)
5 (5.00%) high mild
4 (4.00%) high severe
physical_plan_tpch_q8 time: [9.3676 ms 9.3803 ms 9.3946 ms]
change: [-11.944% -11.733% -11.519%] (p = 0.00 <
0.05)
Performance has improved.
Found 7 outliers among 100 measurements (7.00%)
2 (2.00%) high mild
5 (5.00%) high severe
physical_plan_tpch_q9 time: [7.0398 ms 7.0497 ms 7.0603 ms]
change: [-12.724% -12.537% -12.338%] (p = 0.00 <
0.05)
Performance has improved.
Found 4 outliers among 100 measurements (4.00%)
3 (3.00%) high mild
1 (1.00%) high severe
physical_plan_tpch_q10 time: [5.3599 ms 5.3822 ms 5.4061 ms]
change: [+0.7193% +1.2464% +1.7604%] (p = 0.00 <
0.05)
Change within noise threshold.
Found 6 outliers among 100 measurements (6.00%)
6 (6.00%) high mild
physical_plan_tpch_q11 time: [4.3129 ms 4.3200 ms 4.3277 ms]
change: [+6.8734% +7.1393% +7.4163%] (p = 0.00 <
0.05)
Performance has regressed.
Found 3 outliers among 100 measurements (3.00%)
1 (1.00%) high mild
2 (2.00%) high severe
physical_plan_tpch_q12 time: [3.5347 ms 3.5404 ms 3.5463 ms]
change: [+3.9650% +4.1949% +4.4171%] (p = 0.00 <
0.05)
Performance has regressed.
Found 5 outliers among 100 measurements (5.00%)
3 (3.00%) high mild
2 (2.00%) high severe
physical_plan_tpch_q13 time: [2.2836 ms 2.2871 ms 2.2909 ms]
change: [+7.2793% +7.5424% +7.7873%] (p = 0.00 <
0.05)
Performance has regressed.
Found 5 outliers among 100 measurements (5.00%)
3 (3.00%) high mild
2 (2.00%) high severe
physical_plan_tpch_q14 time: [2.9526 ms 2.9590 ms 2.9660 ms]
change: [-0.6401% +0.3243% +1.2630%] (p = 0.51 >
0.05)
No change in performance detected.
Found 15 outliers among 100 measurements (15.00%)
9 (9.00%) high mild
6 (6.00%) high severe
physical_plan_tpch_q16 time: [4.4232 ms 4.4304 ms 4.4386 ms]
change: [+3.9770% +4.2441% +4.5300%] (p = 0.00 <
0.05)
Performance has regressed.
Found 7 outliers among 100 measurements (7.00%)
3 (3.00%) high mild
4 (4.00%) high severe
physical_plan_tpch_q17 time: [4.1021 ms 4.1085 ms 4.1160 ms]
change: [+7.2418% +7.4904% +7.7389%] (p = 0.00 <
0.05)
Performance has regressed.
Found 6 outliers among 100 measurements (6.00%)
5 (5.00%) high mild
1 (1.00%) high severe
physical_plan_tpch_q18 time: [4.8702 ms 4.9019 ms 4.9339 ms]
change: [+8.7271% +9.5424% +10.331%] (p = 0.00 <
0.05)
Performance has regressed.
physical_plan_tpch_q19 time: [9.1227 ms 9.1335 ms 9.1454 ms]
change: [+2.0674% +2.2535% +2.4382%] (p = 0.00 <
0.05)
Performance has regressed.
Found 8 outliers among 100 measurements (8.00%)
5 (5.00%) high mild
3 (3.00%) high severe
physical_plan_tpch_q20 time: [5.5458 ms 5.5523 ms 5.5593 ms]
change: [+7.5830% +7.8068% +8.0257%] (p = 0.00 <
0.05)
Performance has regressed.
Found 4 outliers among 100 measurements (4.00%)
2 (2.00%) high mild
2 (2.00%) high severe
physical_plan_tpch_q21 time: [8.2274 ms 8.2406 ms 8.2548 ms]
change: [+3.6119% +3.8229% +4.0440%] (p = 0.00 <
0.05)
Performance has regressed.
Found 6 outliers among 100 measurements (6.00%)
6 (6.00%) high mild
physical_plan_tpch_q22 time: [3.7159 ms 3.7217 ms 3.7282 ms]
change: [+1.3475% +1.5751% +1.8234%] (p = 0.00 <
0.05)
Performance has regressed.
Found 6 outliers among 100 measurements (6.00%)
2 (2.00%) high mild
4 (4.00%) high severe
Benchmarking physical_plan_tpch_all: Warming up for 3.0000 s
Warning: Unable to complete 100 samples in 5.0s. You may wish to increase
target time to 11.4s, or reduce sample count to 40.
physical_plan_tpch_all time: [112.98 ms 113.14 ms 113.32 ms]
change: [-0.6799% -0.1096% +0.3016%] (p = 0.72 >
0.05)
No change in performance detected.
Found 11 outliers among 100 measurements (11.00%)
8 (8.00%) high mild
3 (3.00%) high severe
logical_plan_tpch_all time: [12.974 ms 13.015 ms 13.056 ms]
change: [+1.8106% +2.1746% +2.5372%] (p = 0.00 <
0.05)
Performance has regressed.
Found 1 outliers among 100 measurements (1.00%)
1 (1.00%) high mild
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
