comphead commented on PR #9824:
URL:
https://github.com/apache/arrow-datafusion/pull/9824#issuecomment-2030071719
Latest `Arc<String>` consistently improves 5-7%, in very good cases even 14%
```
logical_select_one_from_700
time: [576.23 µs 577.70 µs 579.05 µs]
change: [-6.5082% -5.9265% -5.3353%] (p = 0.00 <
0.05)
Performance has improved.
Found 9 outliers among 100 measurements (9.00%)
1 (1.00%) low severe
4 (4.00%) low mild
1 (1.00%) high mild
3 (3.00%) high severe
physical_select_one_from_700
time: [2.1965 ms 2.2016 ms 2.2071 ms]
change: [-14.414% -14.055% -13.708%] (p = 0.00 <
0.05)
Performance has improved.
Found 9 outliers among 100 measurements (9.00%)
4 (4.00%) high mild
5 (5.00%) high severe
Benchmarking logical_select_all_from_1000: Warming up for 3.0000 s
Warning: Unable to complete 100 samples in 5.0s. You may wish to increase
target time to 6.6s, or reduce sample count to 70.
logical_select_all_from_1000
time: [65.417 ms 65.721 ms 66.037 ms]
change: [-3.9689% -3.3293% -2.7003%] (p = 0.00 <
0.05)
Performance has improved.
Found 2 outliers among 100 measurements (2.00%)
2 (2.00%) high mild
Benchmarking physical_select_all_from_1000: Warming up for 3.0000 s
Warning: Unable to complete 100 samples in 5.0s. You may wish to increase
target time to 47.2s, or reduce sample count to 10.
physical_select_all_from_1000
time: [469.29 ms 470.48 ms 471.69 ms]
change: [-4.8974% -4.5601% -4.1916%] (p = 0.00 <
0.05)
Performance has improved.
Found 2 outliers among 100 measurements (2.00%)
2 (2.00%) high mild
logical_trivial_join_low_numbered_columns
time: [606.50 µs 608.04 µs 609.58 µs]
change: [-4.6959% -4.1899% -3.6424%] (p = 0.00 <
0.05)
Performance has improved.
Found 6 outliers among 100 measurements (6.00%)
1 (1.00%) low severe
3 (3.00%) low mild
1 (1.00%) high mild
1 (1.00%) high severe
logical_trivial_join_high_numbered_columns
time: [634.81 µs 637.24 µs 639.53 µs]
change: [-4.7751% -4.3011% -3.8219%] (p = 0.00 <
0.05)
Performance has improved.
Found 11 outliers among 100 measurements (11.00%)
7 (7.00%) low mild
1 (1.00%) high mild
3 (3.00%) high severe
logical_aggregate_with_join
time: [833.85 µs 835.68 µs 837.53 µs]
change: [-7.1013% -6.7186% -6.3543%] (p = 0.00 <
0.05)
Performance has improved.
Found 3 outliers among 100 measurements (3.00%)
1 (1.00%) low severe
1 (1.00%) low mild
1 (1.00%) high mild
physical_plan_tpch_q1 time: [3.5415 ms 3.5513 ms 3.5618 ms]
change: [-7.6268% -7.2103% -6.8134%] (p = 0.00 <
0.05)
Performance has improved.
Found 7 outliers among 100 measurements (7.00%)
5 (5.00%) high mild
2 (2.00%) high severe
physical_plan_tpch_q2 time: [5.5515 ms 5.5835 ms 5.6185 ms]
change: [-11.115% -10.514% -9.8843%] (p = 0.00 <
0.05)
Performance has improved.
Found 5 outliers among 100 measurements (5.00%)
4 (4.00%) high mild
1 (1.00%) high severe
Benchmarking physical_plan_tpch_q3: Warming up for 3.0000 s
Warning: Unable to complete 100 samples in 5.0s. You may wish to increase
target time to 9.9s, enable flat sampling, or reduce sample count to 50.
physical_plan_tpch_q3 time: [2.0002 ms 2.0202 ms 2.0393 ms]
change: [-6.7269% -5.8744% -5.0422%] (p = 0.00 <
0.05)
Performance has improved.
Found 1 outliers among 100 measurements (1.00%)
1 (1.00%) high mild
Benchmarking physical_plan_tpch_q4: Warming up for 3.0000 s
Warning: Unable to complete 100 samples in 5.0s. You may wish to increase
target time to 8.5s, enable flat sampling, or reduce sample count to 50.
physical_plan_tpch_q4 time: [1.6361 ms 1.6525 ms 1.6692 ms]
change: [-7.7611% -6.8830% -6.0578%] (p = 0.00 <
0.05)
Performance has improved.
Found 3 outliers among 100 measurements (3.00%)
3 (3.00%) high mild
physical_plan_tpch_q5 time: [2.8525 ms 2.8742 ms 2.8964 ms]
change: [-8.1109% -7.3859% -6.6532%] (p = 0.00 <
0.05)
Performance has improved.
Benchmarking physical_plan_tpch_q6: Warming up for 3.0000 s
Warning: Unable to complete 100 samples in 5.0s. You may wish to increase
target time to 6.1s, enable flat sampling, or reduce sample count to 60.
physical_plan_tpch_q6 time: [1.1371 ms 1.1439 ms 1.1512 ms]
change: [-4.8222% -4.1028% -3.3634%] (p = 0.00 <
0.05)
Performance has improved.
Found 3 outliers among 100 measurements (3.00%)
3 (3.00%) high mild
physical_plan_tpch_q7 time: [4.0255 ms 4.0757 ms 4.1341 ms]
change: [-8.3332% -7.0531% -5.6627%] (p = 0.00 <
0.05)
Performance has improved.
Found 5 outliers among 100 measurements (5.00%)
3 (3.00%) high mild
2 (2.00%) high severe
physical_plan_tpch_q8 time: [5.6720 ms 5.6937 ms 5.7166 ms]
change: [-12.538% -12.035% -11.493%] (p = 0.00 <
0.05)
Performance has improved.
Found 3 outliers among 100 measurements (3.00%)
3 (3.00%) high mild
physical_plan_tpch_q9 time: [4.2870 ms 4.2995 ms 4.3133 ms]
change: [-11.457% -11.051% -10.665%] (p = 0.00 <
0.05)
Performance has improved.
Found 6 outliers among 100 measurements (6.00%)
4 (4.00%) high mild
2 (2.00%) high severe
physical_plan_tpch_q10 time: [2.8187 ms 2.8270 ms 2.8359 ms]
change: [-10.879% -10.380% -9.9175%] (p = 0.00 <
0.05)
Performance has improved.
Found 6 outliers among 100 measurements (6.00%)
5 (5.00%) high mild
1 (1.00%) high severe
physical_plan_tpch_q11 time: [2.2546 ms 2.2614 ms 2.2689 ms]
change: [-9.0121% -8.6145% -8.1883%] (p = 0.00 <
0.05)
Performance has improved.
Found 3 outliers among 100 measurements (3.00%)
2 (2.00%) high mild
1 (1.00%) high severe
physical_plan_tpch_q12 time: [2.0119 ms 2.0176 ms 2.0239 ms]
change: [-8.0297% -7.6041% -7.1867%] (p = 0.00 <
0.05)
Performance has improved.
Found 3 outliers among 100 measurements (3.00%)
1 (1.00%) high mild
2 (2.00%) high severe
Benchmarking physical_plan_tpch_q13: Warming up for 3.0000 s
Warning: Unable to complete 100 samples in 5.0s. You may wish to increase
target time to 6.4s, enable flat sampling, or reduce sample count to 60.
physical_plan_tpch_q13 time: [1.2584 ms 1.2620 ms 1.2663 ms]
change: [-7.9586% -7.4495% -6.9731%] (p = 0.00 <
0.05)
Performance has improved.
Found 8 outliers among 100 measurements (8.00%)
2 (2.00%) low mild
4 (4.00%) high mild
2 (2.00%) high severe
Benchmarking physical_plan_tpch_q14: Warming up for 3.0000 s
Warning: Unable to complete 100 samples in 5.0s. You may wish to increase
target time to 8.3s, enable flat sampling, or reduce sample count to 50.
physical_plan_tpch_q14 time: [1.6386 ms 1.6432 ms 1.6482 ms]
change: [-17.798% -15.212% -12.713%] (p = 0.00 <
0.05)
Performance has improved.
Found 4 outliers among 100 measurements (4.00%)
1 (1.00%) low mild
1 (1.00%) high mild
2 (2.00%) high severe
physical_plan_tpch_q16 time: [2.3289 ms 2.3359 ms 2.3439 ms]
change: [-14.684% -14.050% -13.412%] (p = 0.00 <
0.05)
Performance has improved.
Found 4 outliers among 100 measurements (4.00%)
3 (3.00%) high mild
1 (1.00%) high severe
physical_plan_tpch_q17 time: [2.1194 ms 2.1255 ms 2.1323 ms]
change: [-9.7062% -9.3504% -8.9638%] (p = 0.00 <
0.05)
Performance has improved.
Found 6 outliers among 100 measurements (6.00%)
4 (4.00%) high mild
2 (2.00%) high severe
physical_plan_tpch_q18 time: [2.3301 ms 2.3373 ms 2.3448 ms]
change: [-11.888% -11.391% -10.910%] (p = 0.00 <
0.05)
Performance has improved.
Found 3 outliers among 100 measurements (3.00%)
3 (3.00%) high mild
physical_plan_tpch_q19 time: [5.2007 ms 5.2191 ms 5.2392 ms]
change: [-9.2573% -8.7779% -8.2815%] (p = 0.00 <
0.05)
Performance has improved.
Found 4 outliers among 100 measurements (4.00%)
4 (4.00%) high mild
physical_plan_tpch_q20 time: [2.8280 ms 2.8368 ms 2.8466 ms]
change: [-9.9140% -9.4954% -9.0702%] (p = 0.00 <
0.05)
Performance has improved.
Found 1 outliers among 100 measurements (1.00%)
1 (1.00%) high severe
physical_plan_tpch_q21 time: [4.1423 ms 4.1551 ms 4.1689 ms]
change: [-12.567% -12.169% -11.735%] (p = 0.00 <
0.05)
Performance has improved.
Found 7 outliers among 100 measurements (7.00%)
5 (5.00%) high mild
2 (2.00%) high severe
physical_plan_tpch_q22 time: [2.0965 ms 2.1030 ms 2.1100 ms]
change: [-8.2668% -7.7013% -7.1833%] (p = 0.00 <
0.05)
Performance has improved.
Found 4 outliers among 100 measurements (4.00%)
4 (4.00%) high mild
Benchmarking physical_plan_tpch_all: Warming up for 3.0000 s
Warning: Unable to complete 100 samples in 5.0s. You may wish to increase
target time to 6.5s, or reduce sample count to 70.
physical_plan_tpch_all time: [64.117 ms 64.298 ms 64.491 ms]
change: [-12.073% -11.663% -11.246%] (p = 0.00 <
0.05)
Performance has improved.
Found 4 outliers among 100 measurements (4.00%)
4 (4.00%) high mild
logical_plan_tpch_all time: [12.591 ms 12.638 ms 12.686 ms]
change: [-2.1208% -1.5156% -0.8282%] (p = 0.00 <
0.05)
Change within noise threshold.
Found 4 outliers among 100 measurements (4.00%)
1 (1.00%) low mild
3 (3.00%) high mild
```
@alamb @jayzhan211 which way you guys prefer?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
