vrongmeal commented on PR #4795:
URL: https://github.com/apache/arrow-rs/pull/4795#issuecomment-1714062518
The latest changes seem to do quite well:
```
4096 i32_small(0) - 128 time: [130.33 µs 132.50 µs 137.03 µs]
change: [-0.0644% +3.0447% +8.8724%] (p = 0.26 >
0.05)
No change in performance detected.
Found 3 outliers among 100 measurements (3.00%)
3 (3.00%) high severe
Benchmarking 4096 i32_small(0) - 1024: Collecting 100 samples in estimated
5.3598 s (50k iterations4096 i32_small(0) - 1024
time: [102.34 µs 102.38 µs 102.42 µs]
change: [+0.7437% +1.1352% +1.3835%] (p = 0.00 <
0.05)
Change within noise threshold.
Found 5 outliers among 100 measurements (5.00%)
4 (4.00%) high mild
1 (1.00%) high severe
4096 i32_small(0) - 4096
time: [99.582 µs 99.633 µs 99.689 µs]
change: [+0.6403% +0.8823% +1.0898%] (p = 0.00 <
0.05)
Change within noise threshold.
Found 8 outliers among 100 measurements (8.00%)
5 (5.00%) high mild
3 (3.00%) high severe
4096 i32(0) - 128 time: [205.46 µs 205.84 µs 206.24 µs]
change: [-3.7421% -2.6800% -1.8625%] (p = 0.00 <
0.05)
Performance has improved.
4096 i32(0) - 1024 time: [179.71 µs 179.89 µs 180.08 µs]
change: [-1.7409% -1.1017% -0.5766%] (p = 0.00 <
0.05)
Change within noise threshold.
4096 i32(0) - 4096 time: [183.32 µs 184.14 µs 184.85 µs]
change: [-1.9470% -1.4436% -0.9425%] (p = 0.00 <
0.05)
Change within noise threshold.
4096 u64_small(0) - 128 time: [130.11 µs 132.99 µs 137.36 µs]
change: [-3.7062% -1.0089% +1.6673%] (p = 0.49 >
0.05)
No change in performance detected.
Found 2 outliers among 100 measurements (2.00%)
2 (2.00%) high severe
4096 u64_small(0) - 1024
time: [103.47 µs 104.38 µs 106.41 µs]
change: [-0.9524% -0.0361% +1.5091%] (p = 0.96 >
0.05)
No change in performance detected.
Found 3 outliers among 100 measurements (3.00%)
1 (1.00%) high mild
2 (2.00%) high severe
4096 u64_small(0) - 4096
time: [100.65 µs 100.68 µs 100.71 µs]
change: [-3.0726% -1.7815% -0.8379%] (p = 0.00 <
0.05)
Change within noise threshold.
Found 6 outliers among 100 measurements (6.00%)
5 (5.00%) high mild
1 (1.00%) high severe
4096 u64(0) - 128 time: [219.91 µs 220.18 µs 220.45 µs]
change: [-0.5549% -0.4068% -0.2597%] (p = 0.00 <
0.05)
Change within noise threshold.
4096 u64(0) - 1024 time: [193.99 µs 194.13 µs 194.27 µs]
change: [+0.6830% +0.8368% +0.9901%] (p = 0.00 <
0.05)
Change within noise threshold.
Found 3 outliers among 100 measurements (3.00%)
1 (1.00%) high mild
2 (2.00%) high severe
4096 u64(0) - 4096 time: [207.50 µs 208.06 µs 208.60 µs]
change: [-0.7799% +2.1297% +3.8490%] (p = 0.07 >
0.05)
No change in performance detected.
Found 1 outliers among 100 measurements (1.00%)
1 (1.00%) high mild
4096 i64_small(0) - 128 time: [148.42 µs 148.73 µs 149.07 µs]
change: [-1.1969% -0.8221% -0.5079%] (p = 0.00 <
0.05)
Change within noise threshold.
Found 1 outliers among 100 measurements (1.00%)
1 (1.00%) high mild
4096 i64_small(0) - 1024
time: [122.20 µs 122.26 µs 122.33 µs]
change: [-0.4041% -0.2750% -0.1462%] (p = 0.00 <
0.05)
Change within noise threshold.
Found 4 outliers among 100 measurements (4.00%)
2 (2.00%) high mild
2 (2.00%) high severe
4096 i64_small(0) - 4096
time: [119.54 µs 119.59 µs 119.64 µs]
change: [-0.6484% -0.5039% -0.3624%] (p = 0.00 <
0.05)
Change within noise threshold.
Found 8 outliers among 100 measurements (8.00%)
5 (5.00%) high mild
3 (3.00%) high severe
4096 i64(0) - 128 time: [253.10 µs 253.41 µs 253.69 µs]
change: [-4.9520% -3.1144% -1.7332%] (p = 0.00 <
0.05)
Performance has improved.
4096 i64(0) - 1024 time: [225.76 µs 225.85 µs 225.95 µs]
change: [-1.8082% -1.0685% -0.5455%] (p = 0.00 <
0.05)
Change within noise threshold.
Found 8 outliers among 100 measurements (8.00%)
5 (5.00%) high mild
3 (3.00%) high severe
4096 i64(0) - 4096 time: [238.72 µs 238.94 µs 239.18 µs]
change: [-2.7638% -1.5168% -0.4835%] (p = 0.00 <
0.05)
Change within noise threshold.
Found 3 outliers among 100 measurements (3.00%)
1 (1.00%) high mild
2 (2.00%) high severe
4096 f32_small(0) - 128 time: [189.77 µs 190.08 µs 190.42 µs]
change: [-3.0283% -2.3206% -1.8603%] (p = 0.00 <
0.05)
Performance has improved.
4096 f32_small(0) - 1024
time: [163.36 µs 163.72 µs 164.23 µs]
change: [-0.8370% -0.5834% -0.3310%] (p = 0.00 <
0.05)
Change within noise threshold.
Found 10 outliers among 100 measurements (10.00%)
2 (2.00%) high mild
8 (8.00%) high severe
4096 f32_small(0) - 4096
time: [160.66 µs 160.71 µs 160.76 µs]
change: [-8.3509% -4.9169% -2.0822%] (p = 0.00 <
0.05)
Performance has improved.
Found 3 outliers among 100 measurements (3.00%)
1 (1.00%) high mild
2 (2.00%) high severe
4096 f32(0) - 128 time: [239.17 µs 239.41 µs 239.68 µs]
change: [-5.3499% -3.4406% -2.0248%] (p = 0.00 <
0.05)
Performance has improved.
4096 f32(0) - 1024 time: [212.94 µs 213.14 µs 213.34 µs]
change: [-8.6109% -3.8452% -0.4918%] (p = 0.05 >
0.05)
No change in performance detected.
Found 20 outliers among 100 measurements (20.00%)
9 (9.00%) low mild
8 (8.00%) high mild
3 (3.00%) high severe
4096 f32(0) - 4096 time: [220.86 µs 221.13 µs 221.41 µs]
change: [-0.4012% -0.0585% +0.2439%] (p = 0.73 >
0.05)
No change in performance detected.
4096 f64_small(0) - 128 time: [190.02 µs 190.37 µs 190.71 µs]
change: [-1.2881% -1.1310% -0.9806%] (p = 0.00 <
0.05)
Change within noise threshold.
4096 f64_small(0) - 1024
time: [163.64 µs 163.70 µs 163.76 µs]
change: [-0.0629% +0.0076% +0.0864%] (p = 0.85 >
0.05)
No change in performance detected.
Found 4 outliers among 100 measurements (4.00%)
1 (1.00%) high mild
3 (3.00%) high severe
4096 f64_small(0) - 4096
time: [161.39 µs 161.45 µs 161.52 µs]
change: [-4.5426% -1.5487% +0.0554%] (p = 0.40 >
0.05)
No change in performance detected.
Found 6 outliers among 100 measurements (6.00%)
1 (1.00%) low mild
2 (2.00%) high mild
3 (3.00%) high severe
4096 f64(0) - 128 time: [265.23 µs 265.51 µs 265.82 µs]
change: [-8.5473% -3.9192% -0.5514%] (p = 0.06 >
0.05)
No change in performance detected.
4096 f64(0) - 1024 time: [236.95 µs 237.05 µs 237.14 µs]
change: [-0.1151% -0.0326% +0.0409%] (p = 0.44 >
0.05)
No change in performance detected.
Found 5 outliers among 100 measurements (5.00%)
4 (4.00%) high mild
1 (1.00%) high severe
4096 f64(0) - 4096 time: [246.06 µs 247.75 µs 249.53 µs]
change: [-0.9090% -0.2578% +0.4224%] (p = 0.44 >
0.05)
No change in performance detected.
4096 string(10, 0) - 128
time: [120.45 µs 120.63 µs 120.79 µs]
change: [-1.1682% -1.0363% -0.9072%] (p = 0.00 <
0.05)
Change within noise threshold.
Found 1 outliers among 100 measurements (1.00%)
1 (1.00%) high mild
4096 string(10, 0) - 1024
time: [101.01 µs 101.04 µs 101.08 µs]
change: [-0.0422% +0.0156% +0.0662%] (p = 0.58 >
0.05)
No change in performance detected.
Found 2 outliers among 100 measurements (2.00%)
2 (2.00%) high mild
4096 string(10, 0) - 4096
time: [118.38 µs 118.87 µs 119.37 µs]
change: [+7.7898% +8.1893% +8.5642%] (p = 0.00 <
0.05)
Performance has regressed.
4096 string(30, 0) - 128
time: [199.88 µs 199.99 µs 200.08 µs]
change: [-0.5842% -0.5261% -0.4711%] (p = 0.00 <
0.05)
Change within noise threshold.
Found 3 outliers among 100 measurements (3.00%)
3 (3.00%) high mild
4096 string(30, 0) - 1024
time: [178.55 µs 178.81 µs 179.19 µs]
change: [-0.1499% -0.0245% +0.0912%] (p = 0.71 >
0.05)
No change in performance detected.
Found 6 outliers among 100 measurements (6.00%)
3 (3.00%) high mild
3 (3.00%) high severe
4096 string(30, 0) - 4096
time: [194.25 µs 194.96 µs 195.63 µs]
change: [+1.5664% +2.1975% +2.7777%] (p = 0.00 <
0.05)
Performance has regressed.
Found 8 outliers among 100 measurements (8.00%)
2 (2.00%) low severe
5 (5.00%) low mild
1 (1.00%) high mild
4096 string(100, 0) - 128
time: [430.01 µs 430.44 µs 430.87 µs]
change: [-0.7295% -0.5852% -0.3922%] (p = 0.00 <
0.05)
Change within noise threshold.
Found 2 outliers among 100 measurements (2.00%)
1 (1.00%) high mild
1 (1.00%) high severe
4096 string(100, 0) - 1024
time: [443.96 µs 452.71 µs 460.82 µs]
change: [-5.6742% -4.0818% -2.5121%] (p = 0.00 <
0.05)
Performance has improved.
4096 string(100, 0) - 4096
time: [438.85 µs 439.64 µs 440.49 µs]
change: [-0.4321% -0.1080% +0.2058%] (p = 0.51 >
0.05)
No change in performance detected.
Found 5 outliers among 100 measurements (5.00%)
2 (2.00%) high mild
3 (3.00%) high severe
4096 string(100, 0.5) - 128
time: [312.43 µs 312.86 µs 313.23 µs]
change: [-0.4155% -0.2683% -0.1336%] (p = 0.00 <
0.05)
Change within noise threshold.
Found 24 outliers among 100 measurements (24.00%)
11 (11.00%) low severe
12 (12.00%) low mild
1 (1.00%) high mild
4096 string(100, 0.5) - 1024
time: [293.62 µs 294.98 µs 296.50 µs]
change: [-2.3134% -1.8461% -1.3810%] (p = 0.00 <
0.05)
Performance has improved.
4096 string(100, 0.5) - 4096
time: [300.82 µs 302.16 µs 303.50 µs]
change: [-1.0920% -0.7349% -0.3356%] (p = 0.00 <
0.05)
Change within noise threshold.
4096 string(20, 0.5), string(30, 0), string(100, 0), i64(0) - 128
time: [950.54 µs 951.49 µs 952.46 µs]
change: [-0.0619% +0.0370% +0.1391%] (p = 0.47 >
0.05)
No change in performance detected.
4096 string(20, 0.5), string(30, 0), string(100, 0), i64(0) - 1024
time: [882.83 µs 886.20 µs 889.62 µs]
change: [-2.5615% -2.1071% -1.6910%] (p = 0.00 <
0.05)
Performance has improved.
4096 string(20, 0.5), string(30, 0), string(100, 0), i64(0) - 4096
time: [879.77 µs 881.35 µs 882.90 µs]
change: [-3.4161% -2.7473% -2.0204%] (p = 0.00 <
0.05)
Performance has improved.
Found 7 outliers among 100 measurements (7.00%)
2 (2.00%) high mild
5 (5.00%) high severe
4096 string(20, 0.5), string(30, 0), f64(0), i64(0) - 128
time: [768.85 µs 769.68 µs 770.49 µs]
change: [-1.3762% -0.9834% -0.6629%] (p = 0.00 <
0.05)
Change within noise threshold.
Found 2 outliers among 100 measurements (2.00%)
2 (2.00%) high mild
4096 string(20, 0.5), string(30, 0), f64(0), i64(0) - 1024
time: [706.64 µs 707.19 µs 707.71 µs]
change: [-3.9781% -3.2693% -2.6500%] (p = 0.00 <
0.05)
Performance has improved.
Found 3 outliers among 100 measurements (3.00%)
1 (1.00%) high mild
2 (2.00%) high severe
4096 string(20, 0.5), string(30, 0), f64(0), i64(0) - 4096
time: [700.29 µs 701.35 µs 702.44 µs]
change: [-0.9342% -0.4612% -0.0244%] (p = 0.05 <
0.05)
Change within noise threshold.
Found 7 outliers among 100 measurements (7.00%)
6 (6.00%) high mild
1 (1.00%) high severe
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
