tustvold commented on PR #4470:
URL: https://github.com/apache/arrow-rs/pull/4470#issuecomment-1614758771
There is a fair bit of noise, but I'm happy this doesn't drastically regress
the performance anymore
```
convert_columns 4096 u64(0)
time: [17.309 µs 17.317 µs 17.328 µs]
change: [+0.3653% +1.2777% +2.3487%] (p = 0.02 <
0.05)
Change within noise threshold.
Found 17 outliers among 100 measurements (17.00%)
1 (1.00%) low mild
1 (1.00%) high mild
15 (15.00%) high severe
convert_columns_prepared 4096 u64(0)
time: [17.161 µs 17.166 µs 17.173 µs]
change: [-15.951% -15.847% -15.746%] (p = 0.00 <
0.05)
Performance has improved.
Found 8 outliers among 100 measurements (8.00%)
3 (3.00%) high mild
5 (5.00%) high severe
convert_rows 4096 u64(0)
time: [43.738 µs 43.749 µs 43.762 µs]
change: [-9.0306% -8.7858% -8.6404%] (p = 0.00 <
0.05)
Performance has improved.
Found 9 outliers among 100 measurements (9.00%)
1 (1.00%) low mild
5 (5.00%) high mild
3 (3.00%) high severe
convert_columns 4096 i64(0)
time: [17.304 µs 17.311 µs 17.318 µs]
change: [-1.6188% -1.4897% -1.3000%] (p = 0.00 <
0.05)
Performance has improved.
Found 15 outliers among 100 measurements (15.00%)
4 (4.00%) low mild
3 (3.00%) high mild
8 (8.00%) high severe
convert_columns_prepared 4096 i64(0)
time: [19.085 µs 19.316 µs 19.514 µs]
change: [+8.4826% +9.4567% +10.488%] (p = 0.00 <
0.05)
Performance has regressed.
Found 17 outliers among 100 measurements (17.00%)
17 (17.00%) low severe
convert_rows 4096 i64(0)
time: [45.207 µs 45.222 µs 45.242 µs]
change: [-6.3163% -6.1969% -5.9861%] (p = 0.00 <
0.05)
Performance has improved.
Found 8 outliers among 100 measurements (8.00%)
4 (4.00%) high mild
4 (4.00%) high severe
convert_columns 4096 string(10, 0)
time: [63.672 µs 63.687 µs 63.706 µs]
change: [+0.4749% +0.7652% +1.0325%] (p = 0.00 <
0.05)
Change within noise threshold.
Found 12 outliers among 100 measurements (12.00%)
7 (7.00%) high mild
5 (5.00%) high severe
convert_columns_prepared 4096 string(10, 0)
time: [63.474 µs 63.490 µs 63.505 µs]
change: [+0.4334% +0.6488% +0.7677%] (p = 0.00 <
0.05)
Change within noise threshold.
Found 3 outliers among 100 measurements (3.00%)
3 (3.00%) high mild
convert_rows 4096 string(10, 0)
time: [62.774 µs 62.790 µs 62.810 µs]
change: [-3.8972% -3.8456% -3.7968%] (p = 0.00 <
0.05)
Performance has improved.
Found 10 outliers among 100 measurements (10.00%)
1 (1.00%) low mild
2 (2.00%) high mild
7 (7.00%) high severe
convert_columns 4096 string(30, 0)
time: [63.683 µs 63.701 µs 63.719 µs]
change: [+0.8005% +1.1281% +1.5777%] (p = 0.00 <
0.05)
Change within noise threshold.
Found 7 outliers among 100 measurements (7.00%)
5 (5.00%) high mild
2 (2.00%) high severe
convert_columns_prepared 4096 string(30, 0)
time: [63.637 µs 63.662 µs 63.688 µs]
change: [+0.5443% +0.8511% +1.1496%] (p = 0.00 <
0.05)
Change within noise threshold.
Found 1 outliers among 100 measurements (1.00%)
1 (1.00%) high severe
convert_rows 4096 string(30, 0)
time: [63.752 µs 64.440 µs 65.319 µs]
change: [+3.5137% +5.0297% +6.3617%] (p = 0.00 <
0.05)
Performance has regressed.
convert_columns 4096 string(100, 0)
time: [85.907 µs 85.961 µs 86.021 µs]
change: [+4.2815% +4.6011% +4.9283%] (p = 0.00 <
0.05)
Performance has regressed.
Found 1 outliers among 100 measurements (1.00%)
1 (1.00%) high severe
convert_columns_prepared 4096 string(100, 0)
time: [85.606 µs 85.624 µs 85.645 µs]
change: [+3.9518% +4.4749% +4.8798%] (p = 0.00 <
0.05)
Performance has regressed.
Found 9 outliers among 100 measurements (9.00%)
4 (4.00%) high mild
5 (5.00%) high severe
convert_rows 4096 string(100, 0)
time: [95.406 µs 95.517 µs 95.639 µs]
change: [+0.3985% +0.6151% +0.8626%] (p = 0.00 <
0.05)
Change within noise threshold.
Found 6 outliers among 100 measurements (6.00%)
3 (3.00%) low severe
3 (3.00%) high severe
convert_columns 4096 string(100, 0.5)
time: [102.21 µs 102.26 µs 102.33 µs]
change: [+4.1579% +4.3407% +4.5809%] (p = 0.00 <
0.05)
Performance has regressed.
Found 7 outliers among 100 measurements (7.00%)
5 (5.00%) high mild
2 (2.00%) high severe
convert_columns_prepared 4096 string(100, 0.5)
time: [102.05 µs 102.10 µs 102.15 µs]
change: [+4.2080% +4.3656% +4.6188%] (p = 0.00 <
0.05)
Performance has regressed.
Found 8 outliers among 100 measurements (8.00%)
1 (1.00%) low mild
2 (2.00%) high mild
5 (5.00%) high severe
convert_rows 4096 string(100, 0.5)
time: [98.614 µs 98.660 µs 98.713 µs]
change: [-2.6106% -2.2896% -1.9902%] (p = 0.00 <
0.05)
Performance has improved.
Found 6 outliers among 100 measurements (6.00%)
1 (1.00%) low mild
2 (2.00%) high mild
3 (3.00%) high severe
convert_columns 4096 string_dictionary(10, 0)
time: [903.72 µs 903.96 µs 904.27 µs]
change: [+1.6355% +1.8429% +2.0210%] (p = 0.00 <
0.05)
Performance has regressed.
Found 6 outliers among 100 measurements (6.00%)
3 (3.00%) high mild
3 (3.00%) high severe
convert_columns_prepared 4096 string_dictionary(10, 0)
time: [165.34 µs 165.39 µs 165.47 µs]
change: [+0.0240% +0.2124% +0.3262%] (p = 0.00 <
0.05)
Change within noise threshold.
Found 7 outliers among 100 measurements (7.00%)
1 (1.00%) low severe
2 (2.00%) high mild
4 (4.00%) high severe
convert_rows 4096 string_dictionary(10, 0)
time: [267.33 µs 267.39 µs 267.49 µs]
change: [-1.6229% -1.5810% -1.5314%] (p = 0.00 <
0.05)
Performance has improved.
Found 11 outliers among 100 measurements (11.00%)
2 (2.00%) low mild
4 (4.00%) high mild
5 (5.00%) high severe
convert_columns 4096 string_dictionary(30, 0)
time: [918.56 µs 918.83 µs 919.11 µs]
change: [+0.5134% +0.6866% +0.8530%] (p = 0.00 <
0.05)
Change within noise threshold.
Found 12 outliers among 100 measurements (12.00%)
1 (1.00%) low mild
7 (7.00%) high mild
4 (4.00%) high severe
convert_columns_prepared 4096 string_dictionary(30, 0)
time: [169.24 µs 169.28 µs 169.33 µs]
change: [-0.5830% -0.2718% -0.0845%] (p = 0.02 <
0.05)
Change within noise threshold.
Found 13 outliers among 100 measurements (13.00%)
2 (2.00%) low mild
6 (6.00%) high mild
5 (5.00%) high severe
convert_rows 4096 string_dictionary(30, 0)
time: [269.76 µs 269.81 µs 269.88 µs]
change: [-2.4372% -2.2900% -2.0986%] (p = 0.00 <
0.05)
Performance has improved.
Found 16 outliers among 100 measurements (16.00%)
5 (5.00%) low mild
5 (5.00%) high mild
6 (6.00%) high severe
convert_columns 4096 string_dictionary(100, 0)
time: [884.64 µs 884.82 µs 885.04 µs]
change: [+0.1187% +0.2086% +0.2745%] (p = 0.00 <
0.05)
Change within noise threshold.
Found 13 outliers among 100 measurements (13.00%)
2 (2.00%) low mild
5 (5.00%) high mild
6 (6.00%) high severe
convert_columns_prepared 4096 string_dictionary(100, 0)
time: [221.10 µs 221.15 µs 221.22 µs]
change: [-0.1477% -0.0124% +0.1862%] (p = 0.91 >
0.05)
No change in performance detected.
Found 12 outliers among 100 measurements (12.00%)
1 (1.00%) low mild
7 (7.00%) high mild
4 (4.00%) high severe
convert_rows 4096 string_dictionary(100, 0)
time: [303.68 µs 303.79 µs 303.90 µs]
change: [-0.5015% -0.2210% +0.0415%] (p = 0.07 >
0.05)
No change in performance detected.
Found 8 outliers among 100 measurements (8.00%)
4 (4.00%) high mild
4 (4.00%) high severe
convert_columns 4096 string_dictionary_non_preserving(100, 0)
time: [165.64 µs 165.69 µs 165.74 µs]
change: [+3.0833% +3.3515% +3.6023%] (p = 0.00 <
0.05)
Performance has regressed.
Found 3 outliers among 100 measurements (3.00%)
2 (2.00%) high mild
1 (1.00%) high severe
convert_columns_prepared 4096 string_dictionary_non_preserving(100, 0)
time: [162.89 µs 162.94 µs 162.99 µs]
change: [+2.7897% +3.0292% +3.2802%] (p = 0.00 <
0.05)
Performance has regressed.
Found 7 outliers among 100 measurements (7.00%)
1 (1.00%) low severe
3 (3.00%) high mild
3 (3.00%) high severe
convert_rows 4096 string_dictionary_non_preserving(100, 0)
time: [94.879 µs 94.969 µs 95.058 µs]
change: [-0.8412% -0.6973% -0.4752%] (p = 0.00 <
0.05)
Change within noise threshold.
Found 9 outliers among 100 measurements (9.00%)
4 (4.00%) low mild
2 (2.00%) high mild
3 (3.00%) high severe
convert_columns 4096 string_dictionary(100, 0.5)
time: [423.63 µs 424.32 µs 425.71 µs]
change: [-7.9284% -6.7005% -5.4051%] (p = 0.00 <
0.05)
Performance has improved.
Found 8 outliers among 100 measurements (8.00%)
4 (4.00%) high mild
4 (4.00%) high severe
convert_columns_prepared 4096 string_dictionary(100, 0.5)
time: [142.48 µs 142.55 µs 142.62 µs]
change: [+2.3471% +2.6818% +3.0239%] (p = 0.00 <
0.05)
Performance has regressed.
Found 11 outliers among 100 measurements (11.00%)
4 (4.00%) high mild
7 (7.00%) high severe
convert_rows 4096 string_dictionary(100, 0.5)
time: [156.65 µs 156.70 µs 156.76 µs]
change: [-3.3369% -3.0405% -2.7526%] (p = 0.00 <
0.05)
Performance has improved.
Found 6 outliers among 100 measurements (6.00%)
4 (4.00%) high mild
2 (2.00%) high severe
convert_columns 4096 string_dictionary_non_preserving(100, 0.5)
time: [121.10 µs 121.17 µs 121.25 µs]
change: [-4.4204% -4.1771% -3.9828%] (p = 0.00 <
0.05)
Performance has improved.
Found 8 outliers among 100 measurements (8.00%)
5 (5.00%) high mild
3 (3.00%) high severe
convert_columns_prepared 4096 string_dictionary_non_preserving(100, 0.5)
time: [119.18 µs 119.20 µs 119.23 µs]
change: [-4.9679% -4.9138% -4.8618%] (p = 0.00 <
0.05)
Performance has improved.
Found 7 outliers among 100 measurements (7.00%)
4 (4.00%) high mild
3 (3.00%) high severe
convert_rows 4096 string_dictionary_non_preserving(100, 0.5)
time: [98.298 µs 98.328 µs 98.359 µs]
change: [-2.6643% -2.3476% -2.0359%] (p = 0.00 <
0.05)
Performance has improved.
Found 3 outliers among 100 measurements (3.00%)
1 (1.00%) high mild
2 (2.00%) high severe
convert_columns 4096 string(20, 0.5), string(30, 0), string(100, 0), i64(0)
time: [254.95 µs 255.19 µs 255.46 µs]
change: [-10.552% -10.378% -10.145%] (p = 0.00 <
0.05)
Performance has improved.
Found 5 outliers among 100 measurements (5.00%)
3 (3.00%) high mild
2 (2.00%) high severe
convert_columns_prepared 4096 string(20, 0.5), string(30, 0), string(100,
0), i64(0)
time: [254.65 µs 255.05 µs 255.52 µs]
change: [-9.5911% -9.3850% -9.1608%] (p = 0.00 <
0.05)
Performance has improved.
Found 2 outliers among 100 measurements (2.00%)
2 (2.00%) high mild
convert_rows 4096 string(20, 0.5), string(30, 0), string(100, 0), i64(0)
time: [258.38 µs 258.45 µs 258.52 µs]
change: [+2.3731% +2.6820% +2.9723%] (p = 0.00 <
0.05)
Performance has regressed.
Found 6 outliers among 100 measurements (6.00%)
3 (3.00%) high mild
3 (3.00%) high severe
convert_columns 4096 4096 string_dictionary(20, 0.5), string_dictionary(30,
0), string_dictionary(10...
time: [453.39 µs 454.62 µs 455.71 µs]
change: [+2.1490% +2.7530% +3.4015%] (p = 0.00 <
0.05)
Performance has regressed.
Found 3 outliers among 100 measurements (3.00%)
3 (3.00%) low mild
convert_columns_prepared 4096 4096 string_dictionary(20, 0.5),
string_dictionary(30, 0), string_dict...
time: [416.87 µs 417.58 µs 418.29 µs]
change: [-4.1962% -3.7800% -3.3997%] (p = 0.00 <
0.05)
Performance has improved.
Found 2 outliers among 100 measurements (2.00%)
2 (2.00%) high mild
convert_rows 4096 4096 string_dictionary(20, 0.5), string_dictionary(30, 0),
string_dictionary(100, ...
time: [256.80 µs 257.17 µs 257.66 µs]
change: [-0.5142% +0.8155% +1.7484%] (p = 0.19 >
0.05)
No change in performance detected.
Found 13 outliers among 100 measurements (13.00%)
7 (7.00%) low mild
2 (2.00%) high mild
4 (4.00%) high severe
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
