viirya commented on PR #3361:
URL: https://github.com/apache/arrow-rs/pull/3361#issuecomment-1356936378
For no-bloom filter ones, the regression is slightly less than 6.9%, but for
bool types, it is 50~143% improvement.
For bloom filter ones, 54% (string non-null with bloom filter) regression is
worst one.
```
write_batch primitive/4096 values primitive
time: [645.50 µs 646.45 µs 647.66 µs]
thrpt: [272.55 MiB/s 273.06 MiB/s 273.46 MiB/s]
change:
time: [+3.2843% +3.6252% +3.9598%] (p = 0.00 <
0.05)
thrpt: [-3.8090% -3.4984% -3.1799%]
Performance has regressed.
Found 1 outliers among 100 measurements (1.00%)
1 (1.00%) high mild
write_batch primitive/4096 values primitive with bloom filter
time: [4.7123 ms 4.7255 ms 4.7396 ms]
thrpt: [37.244 MiB/s 37.355 MiB/s 37.459 MiB/s]
change:
time: [-3.1886% +0.7072% +4.7452%] (p = 0.73 >
0.05)
thrpt: [-4.5302% -0.7023% +3.2936%]
No change in performance detected.
Found 4 outliers among 100 measurements (4.00%)
3 (3.00%) high mild
1 (1.00%) high severe
write_batch primitive/4096 values primitive non-null
time: [518.44 µs 520.65 µs 522.66 µs]
thrpt: [331.20 MiB/s 332.48 MiB/s 333.89 MiB/s]
change:
time: [-3.3975% -2.9062% -2.4188%] (p = 0.00 <
0.05)
thrpt: [+2.4787% +2.9932% +3.5170%]
Performance has improved.
Found 38 outliers among 100 measurements (38.00%)
15 (15.00%) low severe
4 (4.00%) low mild
1 (1.00%) high mild
18 (18.00%) high severe
write_batch primitive/4096 values primitive non-null with bloom filter
time: [4.5171 ms 4.5242 ms 4.5318 ms]
thrpt: [38.198 MiB/s 38.262 MiB/s 38.321 MiB/s]
change:
time: [+14.987% +15.610% +16.218%] (p = 0.00 <
0.05)
thrpt: [-13.955% -13.503% -13.033%]
Performance has regressed.
Found 4 outliers among 100 measurements (4.00%)
1 (1.00%) low mild
1 (1.00%) high mild
2 (2.00%) high severe
write_batch primitive/4096 values bool
time: [55.116 µs 55.555 µs 56.063 µs]
thrpt: [20.549 MiB/s 20.737 MiB/s 20.902 MiB/s]
change:
time: [-33.749% -33.206% -32.669%] (p = 0.00 <
0.05)
thrpt: [+48.519% +49.713% +50.941%]
Performance has improved.
write_batch primitive/4096 values bool non-null
time: [30.195 µs 30.200 µs 30.205 µs]
thrpt: [21.975 MiB/s 21.979 MiB/s 21.983 MiB/s]
change:
time: [-58.990% -58.969% -58.951%] (p = 0.00 <
0.05)
thrpt: [+143.61% +143.72% +143.84%]
Performance has improved.
Found 5 outliers among 100 measurements (5.00%)
1 (1.00%) low mild
2 (2.00%) high mild
2 (2.00%) high severe
write_batch primitive/4096 values string
time: [288.85 µs 291.28 µs 294.04 µs]
thrpt: [270.83 MiB/s 273.40 MiB/s 275.70 MiB/s]
change:
time: [+6.2048% +6.8701% +7.5217%] (p = 0.00 <
0.05)
thrpt: [-6.9955% -6.4285% -5.8423%]
Performance has regressed.
write_batch primitive/4096 values string with bloom filter
time: [1.1640 ms 1.1804 ms 1.1949 ms]
thrpt: [66.645 MiB/s 67.463 MiB/s 68.413 MiB/s]
change:
time: [+30.161% +32.152% +34.602%] (p = 0.00 <
0.05)
thrpt: [-25.707% -24.330% -23.172%]
Performance has regressed.
Found 8 outliers among 100 measurements (8.00%)
8 (8.00%) high mild
write_batch primitive/4096 values string dictionary
time: [153.55 µs 154.83 µs 156.01 µs]
thrpt: [308.57 MiB/s 310.93 MiB/s 313.52 MiB/s]
change:
time: [+0.9110% +1.8998% +2.9084%] (p = 0.00 <
0.05)
thrpt: [-2.8262% -1.8644% -0.9028%]
Change within noise threshold.
write_batch primitive/4096 values string dictionary with bloom filter
time: [561.37 µs 561.54 µs 561.71 µs]
thrpt: [85.705 MiB/s 85.731 MiB/s 85.757 MiB/s]
change:
time: [+13.056% +13.385% +13.678%] (p = 0.00 <
0.05)
thrpt: [-12.032% -11.805% -11.548%]
Performance has regressed.
Found 4 outliers among 100 measurements (4.00%)
1 (1.00%) high mild
3 (3.00%) high severe
write_batch primitive/4096 values string non-null
time: [329.27 µs 329.37 µs 329.49 µs]
thrpt: [238.73 MiB/s 238.82 MiB/s 238.89 MiB/s]
change:
time: [-0.6052% -0.3595% -0.0651%] (p = 0.00 <
0.05)
thrpt: [+0.0651% +0.3608% +0.6088%]
Change within noise threshold.
Found 14 outliers among 100 measurements (14.00%)
2 (2.00%) low mild
2 (2.00%) high mild
10 (10.00%) high severe
Benchmarking write_batch primitive/4096 values string non-null with bloom
filter: Warming up for 3.0000 s
Warning: Unable to complete 100 samples in 5.0s. You may wish to increase
target time to 5.6s, enable flat sampling, or reduce sample count to 60.
write_batch primitive/4096 values string non-null with bloom filter
time: [1.3000 ms 1.3275 ms 1.3563 ms]
thrpt: [57.995 MiB/s 59.253 MiB/s 60.506 MiB/s]
change:
time: [+52.898% +54.765% +57.194%] (p = 0.00 <
0.05)
thrpt: [-36.384% -35.386% -34.597%]
Performance has regressed.
Found 22 outliers among 100 measurements (22.00%)
1 (1.00%) high mild
21 (21.00%) high severe
write_batch nested/4096 values primitive list
time: [958.15 µs 964.37 µs 970.71 µs]
thrpt: [168.82 MiB/s 169.93 MiB/s 171.04 MiB/s]
change:
time: [+3.4025% +4.0153% +4.6257%] (p = 0.00 <
0.05)
thrpt: [-4.4212% -3.8603% -3.2905%]
Performance has regressed.
Benchmarking write_batch nested/4096 values primitive list non-null: Warming
up for 3.0000 s
Warning: Unable to complete 100 samples in 5.0s. You may wish to increase
target time to 5.6s, enable flat sampling, or reduce sample count to 60.
write_batch nested/4096 values primitive list non-null
time: [1.0984 ms 1.0990 ms 1.0999 ms]
thrpt: [173.25 MiB/s 173.38 MiB/s 173.48 MiB/s]
change:
time: [+1.2422% +1.3060% +1.3765%] (p = 0.00 <
0.05)
thrpt: [-1.3578% -1.2892% -1.2269%]
Performance has regressed.
Found 4 outliers among 100 measurements (4.00%)
4 (4.00%) high severe
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
