viirya opened a new pull request, #3361:
URL: https://github.com/apache/arrow-rs/pull/3361
# Which issue does this PR close?
<!--
We generally require a GitHub issue to be filed for all bug fixes and
enhancements and this helps us generate change logs for our releases. You can
link an issue to this PR using the GitHub syntax. For example `Closes #123`
indicates that this PR will close issue #123.
-->
Closes #.
# Rationale for this change
<!--
Why are you proposing this change? If this is already explained clearly in
the issue then this section is not needed.
Explaining clearly why changes are proposed helps reviewers understand your
changes and offer better suggestions for fixes.
-->
To improve writing performance.
```
write_batch primitive/4096 values primitive
time: [817.22 µs 822.51 µs 828.48 µs]
thrpt: [213.07 MiB/s 214.61 MiB/s 216.00 MiB/s]
change:
time: [-65.811% -65.543% -65.248%] (p = 0.00 <
0.05)
thrpt: [+187.76% +190.22% +192.49%]
Performance has improved.
Found 2 outliers among 100 measurements (2.00%)
2 (2.00%) high severe
write_batch primitive/4096 values primitive with bloom filter
time: [6.8151 ms 6.8616 ms 6.9182 ms]
thrpt: [25.516 MiB/s 25.726 MiB/s 25.901 MiB/s]
change:
time: [-13.979% -12.918% -11.881%] (p = 0.00 <
0.05)
thrpt: [+13.483% +14.834% +16.251%]
Performance has improved.
Found 13 outliers among 100 measurements (13.00%)
5 (5.00%) high mild
8 (8.00%) high severe
write_batch primitive/4096 values primitive non-null
time: [699.61 µs 704.04 µs 708.41 µs]
thrpt: [244.35 MiB/s 245.87 MiB/s 247.43 MiB/s]
change:
time: [-69.195% -68.820% -68.475%] (p = 0.00 <
0.05)
thrpt: [+217.21% +220.71% +224.63%]
Performance has improved.
Found 1 outliers among 100 measurements (1.00%)
1 (1.00%) high mild
write_batch primitive/4096 values primitive non-null with bloom filter
time: [6.7241 ms 6.7664 ms 6.8179 ms]
thrpt: [25.390 MiB/s 25.583 MiB/s 25.744 MiB/s]
change:
time: [-12.194% -11.531% -10.773%] (p = 0.00 <
0.05)
thrpt: [+12.074% +13.034% +13.887%]
Performance has improved.
Found 9 outliers among 100 measurements (9.00%)
1 (1.00%) high mild
8 (8.00%) high severe
write_batch primitive/4096 values bool
time: [109.06 µs 111.13 µs 113.51 µs]
thrpt: [10.149 MiB/s 10.367 MiB/s 10.564 MiB/s]
change:
time: [-70.096% -69.467% -68.691%] (p = 0.00 <
0.05)
thrpt: [+219.39% +227.51% +234.40%]
Performance has improved.
Found 4 outliers among 100 measurements (4.00%)
1 (1.00%) high mild
3 (3.00%) high severe
write_batch primitive/4096 values bool non-null
time: [84.165 µs 84.416 µs 84.695 µs]
thrpt: [7.8371 MiB/s 7.8629 MiB/s 7.8864 MiB/s]
change:
time: [-76.492% -75.147% -74.298%] (p = 0.00 <
0.05)
thrpt: [+289.07% +302.36% +325.38%]
Performance has improved.
Found 12 outliers among 100 measurements (12.00%)
1 (1.00%) low mild
2 (2.00%) high mild
9 (9.00%) high severe
write_batch primitive/4096 values string
time: [396.50 µs 397.06 µs 397.79 µs]
thrpt: [200.19 MiB/s 200.56 MiB/s 200.85 MiB/s]
change:
time: [-56.044% -55.808% -55.529%] (p = 0.00 <
0.05)
thrpt: [+124.87% +126.28% +127.50%]
Performance has improved.
Found 8 outliers among 100 measurements (8.00%)
3 (3.00%) high mild
5 (5.00%) high severe
write_batch primitive/4096 values string with bloom filter
time: [2.3735 ms 2.3988 ms 2.4264 ms]
thrpt: [32.820 MiB/s 33.199 MiB/s 33.551 MiB/s]
change:
time: [-3.0282% -1.8635% -0.6440%] (p = 0.00 <
0.05)
thrpt: [+0.6482% +1.8989% +3.1228%]
Change within noise threshold.
Found 14 outliers among 100 measurements (14.00%)
6 (6.00%) high mild
8 (8.00%) high severe
write_batch primitive/4096 values string dictionary
time: [231.39 µs 231.85 µs 232.38 µs]
thrpt: [207.16 MiB/s 207.64 MiB/s 208.06 MiB/s]
change:
time: [-57.620% -55.943% -54.979%] (p = 0.00 <
0.05)
thrpt: [+122.12% +126.98% +135.96%]
Performance has improved.
Found 6 outliers among 100 measurements (6.00%)
3 (3.00%) high mild
3 (3.00%) high severe
Benchmarking write_batch primitive/4096 values string dictionary with bloom
filter: Warming up for 3.0000 s
Warning: Unable to complete 100 samples in 5.0s. You may wish to increase
target time to 6.0s, enable flat sampling, or reduce sample count to 60.
write_batch primitive/4096 values string dictionary with bloom filter
time: [1.1996 ms 1.2117 ms 1.2253 ms]
thrpt: [39.289 MiB/s 39.731 MiB/s 40.131 MiB/s]
change:
time: [-7.5271% -6.1918% -4.7965%] (p = 0.00 <
0.05)
thrpt: [+5.0381% +6.6005% +8.1398%]
Performance has improved.
Found 14 outliers among 100 measurements (14.00%)
1 (1.00%) low mild
4 (4.00%) high mild
9 (9.00%) high severe
write_batch primitive/4096 values string non-null
time: [474.16 µs 479.55 µs 484.84 µs]
thrpt: [162.24 MiB/s 164.03 MiB/s 165.89 MiB/s]
change:
time: [-50.271% -49.696% -49.133%] (p = 0.00 <
0.05)
thrpt: [+96.591% +98.792% +101.09%]
Performance has improved.
write_batch primitive/4096 values string non-null with bloom filter
time: [2.6152 ms 2.6373 ms 2.6627 ms]
thrpt: [29.541 MiB/s 29.826 MiB/s 30.078 MiB/s]
change:
time: [-0.2535% +1.2207% +2.7549%] (p = 0.11 >
0.05)
thrpt: [-2.6811% -1.2059% +0.2541%]
No change in performance detected.
Found 11 outliers among 100 measurements (11.00%)
5 (5.00%) high mild
6 (6.00%) high severe
Benchmarking write_batch nested/4096 values primitive list: Warming up for
3.0000 s
Warning: Unable to complete 100 samples in 5.0s. You may wish to increase
target time to 5.7s, enable flat sampling, or reduce sample count to 60.
write_batch nested/4096 values primitive list
time: [1.1079 ms 1.1170 ms 1.1322 ms]
thrpt: [144.75 MiB/s 146.71 MiB/s 147.93 MiB/s]
change:
time: [-41.536% -41.148% -40.713%] (p = 0.00 <
0.05)
thrpt: [+68.672% +69.916% +71.045%]
Performance has improved.
Found 2 outliers among 100 measurements (2.00%)
1 (1.00%) high mild
1 (1.00%) high severe
Benchmarking write_batch nested/4096 values primitive list non-null: Warming
up for 3.0000 s
Warning: Unable to complete 100 samples in 5.0s. You may wish to increase
target time to 6.6s, enable flat sampling, or reduce sample count to 60.
write_batch nested/4096 values primitive list non-null
time: [1.3208 ms 1.3322 ms 1.3435 ms]
thrpt: [141.83 MiB/s 143.04 MiB/s 144.27 MiB/s]
change:
time: [-36.827% -36.343% -35.860%] (p = 0.00 <
0.05)
thrpt: [+55.909% +57.092% +58.296%]
Performance has improved.
```
# What changes are included in this PR?
<!--
There is no need to duplicate the description in the issue here but it is
sometimes worth providing a summary of the individual changes in this PR.
-->
# Are there any user-facing changes?
<!--
If there are user-facing changes then we may require documentation to be
updated before approving the PR.
-->
<!---
If there are any breaking changes to public APIs, please add the `breaking
change` label.
-->
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
