vrongmeal commented on PR #4795:
URL: https://github.com/apache/arrow-rs/pull/4795#issuecomment-1715647497
Cleaned up the PR. I made some minor refactors. Here we are with the final
benchmarks (not much different from the previous ones):
```
4096 i32_small(0) - 128 time: [129.48 µs 129.69 µs 129.93 µs]
change: [-0.1022% +0.2640% +0.6342%] (p = 0.16 >
0.05)
No change in performance detected.
Found 26 outliers among 100 measurements (26.00%)
10 (10.00%) low severe
5 (5.00%) low mild
7 (7.00%) high mild
4 (4.00%) high severe
4096 i32_small(0) - 1024
time: [102.46 µs 102.58 µs 102.73 µs]
change: [+0.9391% +1.0842% +1.2562%] (p = 0.00 <
0.05)
Change within noise threshold.
Found 6 outliers among 100 measurements (6.00%)
2 (2.00%) high mild
4 (4.00%) high severe
4096 i32_small(0) - 4096
time: [100.04 µs 100.18 µs 100.32 µs]
change: [+1.5302% +1.6691% +1.8415%] (p = 0.00 <
0.05)
Performance has regressed.
Found 1 outliers among 100 measurements (1.00%)
1 (1.00%) high severe
4096 i32(0) - 128 time: [206.07 µs 206.22 µs 206.36 µs]
change: [+0.2724% +0.3939% +0.5160%] (p = 0.00 <
0.05)
Change within noise threshold.
Found 2 outliers among 100 measurements (2.00%)
2 (2.00%) high severe
4096 i32(0) - 1024 time: [180.19 µs 180.31 µs 180.44 µs]
change: [+0.3165% +0.4330% +0.5506%] (p = 0.00 <
0.05)
Change within noise threshold.
Found 4 outliers among 100 measurements (4.00%)
3 (3.00%) high mild
1 (1.00%) high severe
4096 i32(0) - 4096 time: [186.55 µs 186.78 µs 187.02 µs]
change: [+0.8841% +1.1445% +1.3982%] (p = 0.00 <
0.05)
Change within noise threshold.
Found 2 outliers among 100 measurements (2.00%)
1 (1.00%) high mild
1 (1.00%) high severe
4096 u64_small(0) - 128 time: [129.82 µs 130.01 µs 130.21 µs]
change: [-0.2892% -0.1184% +0.0488%] (p = 0.17 >
0.05)
No change in performance detected.
4096 u64_small(0) - 1024
time: [103.63 µs 103.69 µs 103.76 µs]
change: [+0.2414% +0.3326% +0.4239%] (p = 0.00 <
0.05)
Change within noise threshold.
Found 7 outliers among 100 measurements (7.00%)
1 (1.00%) low mild
4 (4.00%) high mild
2 (2.00%) high severe
4096 u64_small(0) - 4096
time: [101.59 µs 101.64 µs 101.69 µs]
change: [+0.8516% +0.9603% +1.0727%] (p = 0.00 <
0.05)
Change within noise threshold.
Found 2 outliers among 100 measurements (2.00%)
1 (1.00%) low mild
1 (1.00%) high severe
4096 u64(0) - 128 time: [221.43 µs 221.57 µs 221.72 µs]
change: [-1.7539% +0.3171% +1.5285%] (p = 0.82 >
0.05)
No change in performance detected.
Found 4 outliers among 100 measurements (4.00%)
3 (3.00%) high mild
1 (1.00%) high severe
4096 u64(0) - 1024 time: [194.33 µs 194.44 µs 194.55 µs]
change: [+1.6837% +1.8242% +1.9828%] (p = 0.00 <
0.05)
Performance has regressed.
Found 10 outliers among 100 measurements (10.00%)
6 (6.00%) high mild
4 (4.00%) high severe
4096 u64(0) - 4096 time: [202.89 µs 204.05 µs 205.06 µs]
change: [-1.0604% -0.5223% -0.0185%] (p = 0.06 >
0.05)
No change in performance detected.
4096 i64_small(0) - 128 time: [147.68 µs 147.85 µs 148.02 µs]
change: [-1.2601% -1.0891% -0.9241%] (p = 0.00 <
0.05)
Change within noise threshold.
Found 1 outliers among 100 measurements (1.00%)
1 (1.00%) high mild
4096 i64_small(0) - 1024
time: [122.26 µs 122.33 µs 122.39 µs]
change: [-0.3205% -0.2296% -0.1350%] (p = 0.00 <
0.05)
Change within noise threshold.
Found 2 outliers among 100 measurements (2.00%)
1 (1.00%) high mild
1 (1.00%) high severe
4096 i64_small(0) - 4096
time: [119.52 µs 119.58 µs 119.65 µs]
change: [-0.3122% -0.2236% -0.1442%] (p = 0.00 <
0.05)
Change within noise threshold.
Found 5 outliers among 100 measurements (5.00%)
2 (2.00%) high mild
3 (3.00%) high severe
4096 i64(0) - 128 time: [252.20 µs 252.36 µs 252.52 µs]
change: [-0.4413% -0.2934% -0.1557%] (p = 0.00 <
0.05)
Change within noise threshold.
Found 3 outliers among 100 measurements (3.00%)
2 (2.00%) high mild
1 (1.00%) high severe
4096 i64(0) - 1024 time: [226.34 µs 226.43 µs 226.53 µs]
change: [+0.2713% +0.3469% +0.4166%] (p = 0.00 <
0.05)
Change within noise threshold.
Found 10 outliers among 100 measurements (10.00%)
9 (9.00%) high mild
1 (1.00%) high severe
4096 i64(0) - 4096 time: [239.07 µs 239.31 µs 239.56 µs]
change: [-1.4105% -1.2061% -1.0047%] (p = 0.00 <
0.05)
Performance has improved.
Found 5 outliers among 100 measurements (5.00%)
3 (3.00%) high mild
2 (2.00%) high severe
4096 f32_small(0) - 128 time: [188.32 µs 188.39 µs 188.46 µs]
change: [-1.2398% -1.0624% -0.8853%] (p = 0.00 <
0.05)
Change within noise threshold.
Found 5 outliers among 100 measurements (5.00%)
3 (3.00%) high mild
2 (2.00%) high severe
4096 f32_small(0) - 1024
time: [163.03 µs 163.16 µs 163.37 µs]
change: [-0.1988% -0.1012% +0.0076%] (p = 0.05 <
0.05)
Change within noise threshold.
Found 7 outliers among 100 measurements (7.00%)
4 (4.00%) high mild
3 (3.00%) high severe
4096 f32_small(0) - 4096
time: [162.22 µs 162.64 µs 163.13 µs]
change: [+0.8514% +1.0582% +1.2712%] (p = 0.00 <
0.05)
Change within noise threshold.
Found 1 outliers among 100 measurements (1.00%)
1 (1.00%) high mild
4096 f32(0) - 128 time: [239.99 µs 240.17 µs 240.35 µs]
change: [-0.1242% -0.0110% +0.0980%] (p = 0.85 >
0.05)
No change in performance detected.
Found 5 outliers among 100 measurements (5.00%)
4 (4.00%) high mild
1 (1.00%) high severe
4096 f32(0) - 1024 time: [214.23 µs 214.37 µs 214.51 µs]
change: [+0.1882% +0.3555% +0.5017%] (p = 0.00 <
0.05)
Change within noise threshold.
Found 3 outliers among 100 measurements (3.00%)
2 (2.00%) high mild
1 (1.00%) high severe
4096 f32(0) - 4096 time: [220.51 µs 220.78 µs 221.05 µs]
change: [+0.0601% +0.3376% +0.6216%] (p = 0.02 <
0.05)
Change within noise threshold.
Found 4 outliers among 100 measurements (4.00%)
3 (3.00%) high mild
1 (1.00%) high severe
4096 f64_small(0) - 128 time: [189.29 µs 189.42 µs 189.54 µs]
change: [-0.3008% -0.1870% -0.0795%] (p = 0.00 <
0.05)
Change within noise threshold.
Found 2 outliers among 100 measurements (2.00%)
2 (2.00%) high mild
4096 f64_small(0) - 1024
time: [164.10 µs 164.15 µs 164.21 µs]
change: [+0.2648% +0.3359% +0.4127%] (p = 0.00 <
0.05)
Change within noise threshold.
Found 3 outliers among 100 measurements (3.00%)
2 (2.00%) high mild
1 (1.00%) high severe
4096 f64_small(0) - 4096
time: [161.84 µs 162.18 µs 162.65 µs]
change: [+0.3241% +0.4859% +0.6660%] (p = 0.00 <
0.05)
Change within noise threshold.
Found 9 outliers among 100 measurements (9.00%)
4 (4.00%) high mild
5 (5.00%) high severe
4096 f64(0) - 128 time: [264.38 µs 264.51 µs 264.63 µs]
change: [-0.2875% -0.1436% +0.0010%] (p = 0.05 >
0.05)
No change in performance detected.
Found 4 outliers among 100 measurements (4.00%)
2 (2.00%) high mild
2 (2.00%) high severe
4096 f64(0) - 1024 time: [236.75 µs 236.82 µs 236.90 µs]
change: [-0.1708% -0.0758% +0.0053%] (p = 0.09 >
0.05)
No change in performance detected.
Found 5 outliers among 100 measurements (5.00%)
2 (2.00%) high mild
3 (3.00%) high severe
4096 f64(0) - 4096 time: [250.96 µs 251.21 µs 251.45 µs]
change: [-1.6759% -1.4032% -1.1108%] (p = 0.00 <
0.05)
Performance has improved.
Found 1 outliers among 100 measurements (1.00%)
1 (1.00%) high severe
4096 string(10, 0) - 128
time: [120.43 µs 120.67 µs 120.96 µs]
change: [-1.5145% -1.3509% -1.1836%] (p = 0.00 <
0.05)
Performance has improved.
Found 1 outliers among 100 measurements (1.00%)
1 (1.00%) high severe
4096 string(10, 0) - 1024
time: [100.94 µs 100.97 µs 101.00 µs]
change: [-0.1498% -0.0674% +0.0154%] (p = 0.10 >
0.05)
No change in performance detected.
Found 9 outliers among 100 measurements (9.00%)
4 (4.00%) high mild
5 (5.00%) high severe
4096 string(10, 0) - 4096
time: [114.31 µs 114.59 µs 114.87 µs]
change: [-2.9515% -2.3473% -1.6384%] (p = 0.00 <
0.05)
Performance has improved.
Found 11 outliers among 100 measurements (11.00%)
3 (3.00%) low severe
4 (4.00%) low mild
4 (4.00%) high severe
4096 string(30, 0) - 128
time: [200.39 µs 200.64 µs 200.89 µs]
change: [-0.0823% +0.0362% +0.1587%] (p = 0.56 >
0.05)
No change in performance detected.
4096 string(30, 0) - 1024
time: [179.58 µs 179.79 µs 180.01 µs]
change: [+0.6443% +0.7986% +0.9492%] (p = 0.00 <
0.05)
Change within noise threshold.
Found 6 outliers among 100 measurements (6.00%)
4 (4.00%) high mild
2 (2.00%) high severe
4096 string(30, 0) - 4096
time: [193.35 µs 193.84 µs 194.27 µs]
change: [-1.1721% -0.8889% -0.6075%] (p = 0.00 <
0.05)
Change within noise threshold.
Found 5 outliers among 100 measurements (5.00%)
1 (1.00%) low mild
2 (2.00%) high mild
2 (2.00%) high severe
4096 string(100, 0) - 128
time: [432.10 µs 432.97 µs 433.84 µs]
change: [+0.2068% +0.3655% +0.5314%] (p = 0.00 <
0.05)
Change within noise threshold.
4096 string(100, 0) - 1024
time: [438.27 µs 446.28 µs 454.66 µs]
change: [-3.6040% -2.1777% -0.9708%] (p = 0.00 <
0.05)
Change within noise threshold.
4096 string(100, 0) - 4096
time: [441.42 µs 442.67 µs 443.87 µs]
change: [+1.1279% +1.4411% +1.7686%] (p = 0.00 <
0.05)
Performance has regressed.
Found 4 outliers among 100 measurements (4.00%)
2 (2.00%) high mild
2 (2.00%) high severe
4096 string(100, 0.5) - 128
time: [311.91 µs 312.07 µs 312.23 µs]
change: [-0.3517% -0.1610% +0.0619%] (p = 0.16 >
0.05)
No change in performance detected.
Found 10 outliers among 100 measurements (10.00%)
7 (7.00%) high mild
3 (3.00%) high severe
4096 string(100, 0.5) - 1024
time: [298.65 µs 300.69 µs 302.52 µs]
change: [+0.9439% +1.5945% +2.2157%] (p = 0.00 <
0.05)
Change within noise threshold.
4096 string(100, 0.5) - 4096
time: [313.77 µs 316.21 µs 318.96 µs]
change: [+0.8984% +1.5120% +2.1361%] (p = 0.00 <
0.05)
Change within noise threshold.
Found 5 outliers among 100 measurements (5.00%)
2 (2.00%) high mild
3 (3.00%) high severe
4096 string(20, 0.5), string(30, 0), string(100, 0), i64(0) - 128
time: [950.86 µs 951.80 µs 952.96 µs]
change: [+0.2635% +0.4109% +0.5742%] (p = 0.00 <
0.05)
Change within noise threshold.
4096 string(20, 0.5), string(30, 0), string(100, 0), i64(0) - 1024
time: [894.21 µs 897.00 µs 899.93 µs]
change: [-1.8062% -1.2546% -0.7216%] (p = 0.00 <
0.05)
Change within noise threshold.
Found 2 outliers among 100 measurements (2.00%)
2 (2.00%) high mild
4096 string(20, 0.5), string(30, 0), string(100, 0), i64(0) - 4096
time: [882.42 µs 888.96 µs 899.89 µs]
change: [-1.7490% -0.7579% +0.3868%] (p = 0.17 >
0.05)
No change in performance detected.
Found 9 outliers among 100 measurements (9.00%)
4 (4.00%) high mild
5 (5.00%) high severe
4096 string(20, 0.5), string(30, 0), f64(0), i64(0) - 128
time: [777.10 µs 777.90 µs 778.76 µs]
change: [+0.3738% +0.5824% +0.7619%] (p = 0.00 <
0.05)
Change within noise threshold.
Found 4 outliers among 100 measurements (4.00%)
2 (2.00%) low mild
2 (2.00%) high mild
4096 string(20, 0.5), string(30, 0), f64(0), i64(0) - 1024
time: [706.16 µs 707.73 µs 709.08 µs]
change: [-0.3864% -0.1618% +0.0398%] (p = 0.13 >
0.05)
No change in performance detected.
Found 1 outliers among 100 measurements (1.00%)
1 (1.00%) high mild
4096 string(20, 0.5), string(30, 0), f64(0), i64(0) - 4096
time: [702.01 µs 704.64 µs 707.09 µs]
change: [+0.4500% +0.7426% +1.0359%] (p = 0.00 <
0.05)
Change within noise threshold.
Found 6 outliers among 100 measurements (6.00%)
4 (4.00%) high mild
2 (2.00%) high severe
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
