HaoYang670 commented on PR #2127:
URL: https://github.com/apache/arrow-rs/pull/2127#issuecomment-1192997932
Thank you @tustvold !. The benchmark result is really interesting. I can
reproduce some of the result:
Tested on M1 Pro Mac:
```
remziy@remziy-mlt:Arrow-rs$ cargo bench --bench builder -- --baseline master
Compiling arrow v18.0.0
(/Users/remziy/FangcloudV2/personal_space.localized/working/Arrow-rs/arrow)
Finished bench [optimized] target(s) in 39.38s
Running benches/builder.rs
(target/release/deps/builder-dd4589600d542fc3)
WARNING: HTML report generation will become a non-default optional feature
in Criterion.rs 0.4.0.
This feature is being moved to cargo-criterion
(https://github.com/bheisler/cargo-criterion) and will be optional in a future
version of Criterion.rs. To silence this warning, either switch to
cargo-criterion or enable the 'html_reports' feature in your Cargo.toml.
WARNING: In Criterion.rs 0.4.0, running criterion benchmarks outside of
cargo-criterion will become a default optional feature.
The statistical analysis and reporting is being moved to cargo-criterion
(https://github.com/bheisler/cargo-criterion) and will be optional in a future
version of Criterion.rs. To silence this warning, either switch to
cargo-criterion or enable the 'cargo_bench_support' feature in your Cargo.toml.
Gnuplot not found, using plotters backend
bench_primitive/bench_primitive
time: [642.48 us 646.76 us 651.53 us]
thrpt: [5.9955 GiB/s 6.0397 GiB/s 6.0799 GiB/s]
change:
time: [+1.3793% +3.3046% +5.3602%] (p = 0.00 <
0.05)
thrpt: [-5.0875% -3.1989% -1.3605%]
Performance has regressed.
Found 15 outliers among 100 measurements (15.00%)
10 (10.00%) high mild
5 (5.00%) high severe
Benchmarking bench_primitive_nulls/bench_primitive_nulls: Warming up for
3.0000 s
Warning: Unable to complete 100 samples in 5.0s. You may wish to increase
target time to 7.3s, enable flat sampling, or reduce sample count to 50.
bench_primitive_nulls/bench_primitive_nulls
time: [1.4423 ms 1.4447 ms 1.4480 ms]
change: [+4.0353% +4.8072% +5.8474%] (p = 0.00 <
0.05)
Performance has regressed.
Found 9 outliers among 100 measurements (9.00%)
1 (1.00%) high mild
8 (8.00%) high severe
bench_bool/bench_bool time: [658.62 us 687.66 us 722.13 us]
thrpt: [692.40 MiB/s 727.10 MiB/s 759.16 MiB/s]
change:
time: [-0.8443% +1.7920% +4.6238%] (p = 0.23 >
0.05)
thrpt: [-4.4195% -1.7604% +0.8515%]
No change in performance detected.
Found 30 outliers among 100 measurements (30.00%)
20 (20.00%) low severe
1 (1.00%) low mild
9 (9.00%) high severe
bench_primitive/bench_string
time: [3.6955 ms 3.7110 ms 3.7281 ms]
thrpt: [1.7027 GiB/s 1.7105 GiB/s 1.7177 GiB/s]
change:
time: [+29.974% +30.513% +31.173%] (p = 0.00 <
0.05)
thrpt: [-23.765% -23.379% -23.062%]
Performance has regressed.
Found 2 outliers among 100 measurements (2.00%)
1 (1.00%) high mild
1 (1.00%) high severe
```
The most obvious regression happened on `bench_string` which didn't use the
`lazy materialization` previously. I will try to reproduce this result on my
Intel desktop.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscr...@arrow.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
