crepererum opened a new pull request, #4651:
URL: https://github.com/apache/arrow-datafusion/pull/4651
# Which issue does this PR close?
\-
# Rationale for this change
Just found a bunch of CPU-cycles spent in `clone` for large aggregations
that involve strings. Seems that we don't need to clone that much data.
# What changes are included in this PR?
A bit more data moving instead of cloning.
# Are these changes tested?
```console
❯ cargo bench -p datafusion --bench aggregate_query_sql -- --baseline
avoid_hash_group_scalarvalue_copy
aggregate_query_no_group_by 15 12
time: [681.31 µs 682.55 µs 683.93 µs]
change: [-1.3475% -0.9347% -0.4997%] (p = 0.00 <
0.05)
Change within noise threshold.
Found 7 outliers among 100 measurements (7.00%)
3 (3.00%) high mild
4 (4.00%) high severe
aggregate_query_no_group_by_min_max_f64
time: [623.32 µs 624.53 µs 625.77 µs]
change: [-1.5852% -1.1752% -0.7877%] (p = 0.00 <
0.05)
Change within noise threshold.
Found 5 outliers among 100 measurements (5.00%)
1 (1.00%) low severe
1 (1.00%) high mild
3 (3.00%) high severe
aggregate_query_no_group_by_count_distinct_wide
time: [2.4782 ms 2.4970 ms 2.5157 ms]
change: [-0.4410% +0.4967% +1.5074%] (p = 0.35 >
0.05)
No change in performance detected.
Found 1 outliers among 100 measurements (1.00%)
1 (1.00%) low mild
Benchmarking aggregate_query_no_group_by_count_distinct_narrow: Warming up
for 3.0000 s
Warning: Unable to complete 100 samples in 5.0s. You may wish to increase
target time to 8.6s, enable flat sampling, or reduce sample count to 50.
aggregate_query_no_group_by_count_distinct_narrow
time: [1.6871 ms 1.6945 ms 1.7022 ms]
change: [-1.6426% -0.7138% +0.2828%] (p = 0.16 >
0.05)
No change in performance detected.
Found 4 outliers among 100 measurements (4.00%)
1 (1.00%) low mild
2 (2.00%) high mild
1 (1.00%) high severe
aggregate_query_group_by
time: [2.2208 ms 2.2368 ms 2.2537 ms]
change: [-1.7788% -0.7814% +0.2329%] (p = 0.13 >
0.05)
No change in performance detected.
Found 2 outliers among 100 measurements (2.00%)
1 (1.00%) high mild
1 (1.00%) high severe
Benchmarking aggregate_query_group_by_with_filter: Warming up for 3.0000 s
Warning: Unable to complete 100 samples in 5.0s. You may wish to increase
target time to 5.7s, enable flat sampling, or reduce sample count to 60.
aggregate_query_group_by_with_filter
time: [1.1228 ms 1.1256 ms 1.1288 ms]
change: [-3.8888% -3.0339% -2.2444%] (p = 0.00 <
0.05)
Performance has improved.
Found 5 outliers among 100 measurements (5.00%)
1 (1.00%) high mild
4 (4.00%) high severe
aggregate_query_group_by_u64 15 12
time: [2.2511 ms 2.2662 ms 2.2822 ms]
change: [-1.3890% -0.4107% +0.6080%] (p = 0.43 >
0.05)
No change in performance detected.
Found 2 outliers among 100 measurements (2.00%)
2 (2.00%) high mild
Benchmarking aggregate_query_group_by_with_filter_u64 15 12: Warming up for
3.0000 s
Warning: Unable to complete 100 samples in 5.0s. You may wish to increase
target time to 5.7s, enable flat sampling, or reduce sample count to 60.
aggregate_query_group_by_with_filter_u64 15 12
time: [1.1191 ms 1.1208 ms 1.1227 ms]
change: [-2.0168% -1.7182% -1.3923%] (p = 0.00 <
0.05)
Performance has improved.
Found 7 outliers among 100 measurements (7.00%)
3 (3.00%) low mild
3 (3.00%) high mild
1 (1.00%) high severe
aggregate_query_group_by_u64_multiple_keys
time: [14.791 ms 15.115 ms 15.444 ms]
change: [-7.4168% -4.2996% -1.0317%] (p = 0.01 <
0.05)
Performance has improved.
aggregate_query_approx_percentile_cont_on_u64
time: [3.7578 ms 3.7899 ms 3.8222 ms]
change: [-1.6803% -0.4507% +0.7961%] (p = 0.49 >
0.05)
No change in performance detected.
Found 1 outliers among 100 measurements (1.00%)
1 (1.00%) high mild
aggregate_query_approx_percentile_cont_on_f32
time: [3.2097 ms 3.2302 ms 3.2508 ms]
change: [-1.2514% -0.2948% +0.6973%] (p = 0.55 >
0.05)
No change in performance detected.
Found 3 outliers among 100 measurements (3.00%)
1 (1.00%) low mild
2 (2.00%) high mild
```
# Are there any user-facing changes?
Faster group-bys.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
