msirek commented on PR #8038:
URL:
https://github.com/apache/arrow-datafusion/pull/8038#issuecomment-1791460977
### Results from criterion_benchmark_limited_distinct
#### baseline
custom-measurement-time/distinct_group_by_u64_narrow_limit_10
time: [120.79 ms 121.22 ms 121.67 ms]
Found 3 outliers among 100 measurements (3.00%)
3 (3.00%) high mild
custom-measurement-time/distinct_group_by_u64_narrow_limit_100
time: [124.01 ms 124.55 ms 125.12 ms]
Found 3 outliers among 100 measurements (3.00%)
1 (1.00%) low mild
1 (1.00%) high mild
1 (1.00%) high severe
custom-measurement-time/distinct_group_by_u64_narrow_limit_1000
time: [124.04 ms 124.48 ms 124.94 ms]
Found 1 outliers among 100 measurements (1.00%)
1 (1.00%) high severe
custom-measurement-time/distinct_group_by_u64_narrow_limit_10000
time: [127.71 ms 129.49 ms 131.52 ms]
Found 15 outliers among 100 measurements (15.00%)
9 (9.00%) high mild
6 (6.00%) high severe
Benchmarking custom-measurement-time/group_by_multiple_columns_limit_10:
Warming up for 3.0000 s
Warning: Unable to complete 100 samples in 40.0s. You may wish to increase
target time to 136.5s, or reduce sample count to 20.
custom-measurement-time/group_by_multiple_columns_limit_10
time: [1.3416 s 1.3465 s 1.3517 s]
Found 5 outliers among 100 measurements (5.00%)
5 (5.00%) high mild
#### new
custom-measurement-time/distinct_group_by_u64_narrow_limit_10
time: [83.726 ms 84.168 ms 84.596 ms]
**change: [-31.003% -30.568% -30.128%] (p = 0.00 <
0.05)**
Performance has improved.
Found 4 outliers among 100 measurements (4.00%)
3 (3.00%) low mild
1 (1.00%) high mild
custom-measurement-time/distinct_group_by_u64_narrow_limit_100
time: [112.00 ms 112.33 ms 112.66 ms]
**change: [-10.299% -9.8102% -9.3399%] (p = 0.00 <
0.05)**
Performance has improved.
Found 1 outliers among 100 measurements (1.00%)
1 (1.00%) low mild
custom-measurement-time/distinct_group_by_u64_narrow_limit_1000
time: [112.01 ms 112.37 ms 112.73 ms]
**change: [-10.152% -9.7257% -9.2833%] (p = 0.00 <
0.05)**
Performance has improved.
Found 1 outliers among 100 measurements (1.00%)
1 (1.00%) low mild
custom-measurement-time/distinct_group_by_u64_narrow_limit_10000
time: [111.86 ms 112.21 ms 112.55 ms]
**change: [-14.699% -13.350% -12.114%] (p = 0.00 <
0.05)**
Performance has improved.
Found 2 outliers among 100 measurements (2.00%)
1 (1.00%) low mild
1 (1.00%) high mild
Benchmarking
custom-measurement-time/aggregate_group_by_multiple_columns_limit_10: Warming
up for 3.0000 s
Warning: Unable to complete 100 samples in 40.0s. You may wish to increase
target time to 114.8s, or reduce sample count to 30.
custom-measurement-time/group_by_multiple_columns_limit_10
time: [1.1212 s 1.1247 s 1.1285 s]
**change: [-16.881% -16.473% -16.075%] (p = 0.00 <
0.05)**
Performance has improved.
Found 3 outliers among 100 measurements (3.00%)
2 (2.00%) high mild
1 (1.00%) high severe
### Results from criterion_benchmark_limited_distinct_sampled
#### baseline
Benchmarking distinct query with 100 partitions and 100000 samples per
partition with limit 10: Warming up for 3.0000 s
Warning: Unable to complete 100 samples in 5.0s. You may wish to increase
target time to 15.3s, or reduce sample count to 30.
distinct query with 100 partitions and 100000 samples per partition with
limit 10
time: [151.67 ms 151.93 ms 152.19 ms]
Found 1 outliers among 100 measurements (1.00%)
1 (1.00%) low mild
Benchmarking distinct query with 10 partitions and 1000000 samples per
partition with limit 10: Warming up for 3.0000 s
Warning: Unable to complete 100 samples in 5.0s. You may wish to increase
target time to 97.2s, or reduce sample count to 10.
distinct query with 10 partitions and 1000000 samples per partition with
limit 10
time: [922.72 ms 933.79 ms 944.47 ms]
Found 23 outliers among 100 measurements (23.00%)
20 (20.00%) low severe
3 (3.00%) low mild
Benchmarking distinct query with 1 partitions and 10000000 samples per
partition with limit 10: Warming up for 3.0000 s
Warning: Unable to complete 100 samples in 5.0s. You may wish to increase
target time to 105.1s, or reduce sample count to 10.
distinct query with 1 partitions and 10000000 samples per partition with
limit 10
time: [1.0396 s 1.0424 s 1.0454 s]
Found 3 outliers among 100 measurements (3.00%)
3 (3.00%) high mild
#### new
Benchmarking distinct query with 100 partitions and 100000 samples per
partition with limit 10: Warming up for 3.0000 s
Warning: Unable to complete 100 samples in 5.0s. You may wish to increase
target time to 8.9s, or reduce sample count to 50.
distinct query with 100 partitions and 100000 samples per partition with
limit 10
time: [85.114 ms 87.070 ms 88.822 ms]
**change: [-44.085% -42.690% -41.525%] (p = 0.00 <
0.05)**
Performance has improved.
Found 20 outliers among 100 measurements (20.00%)
16 (16.00%) low severe
4 (4.00%) low mild
Benchmarking distinct query with 10 partitions and 1000000 samples per
partition with limit 10: Warming up for 3.0000 s
Warning: Unable to complete 100 samples in 5.0s. You may wish to increase
target time to 10.2s, or reduce sample count to 40.
distinct query with 10 partitions and 1000000 samples per partition with
limit 10
time: [104.73 ms 106.50 ms 108.28 ms]
**change: [-88.813% -88.594% -88.348%] (p = 0.00 <
0.05)**
Performance has improved.
Found 4 outliers among 100 measurements (4.00%)
1 (1.00%) low mild
3 (3.00%) high mild
Benchmarking distinct query with 1 partitions and 10000000 samples per
partition with limit 10: Warming up for 3.0000 s
Warning: Unable to complete 100 samples in 5.0s. You may wish to increase
target time to 97.1s, or reduce sample count to 10.
distinct query with 1 partitions and 10000000 samples per partition with
limit 10
time: [952.15 ms 954.94 ms 957.80 ms]
**change: [-8.7663% -8.3904% -8.0083%] (p = 0.00 <
0.05)**
Performance has improved.
Found 3 outliers among 100 measurements (3.00%)
3 (3.00%) high mild
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
