RinChanNOWWW commented on PR #5624:
URL: https://github.com/apache/arrow-rs/pull/5624#issuecomment-2053861768
# Benchmark
## Test Machine
MacBook M1 Pro (10 Cores, 32G RAM)
## Bench Results
### Before this PR
```
filter optimize (kept 1/2)
time: [108.85 µs 109.10 µs 109.38 µs]
Found 14 outliers among 100 measurements (14.00%)
12 (12.00%) high mild
2 (2.00%) high severe
filter optimize high selectivity (kept 1023/1024)
time: [1.2401 µs 1.2460 µs 1.2546 µs]
Found 13 outliers among 100 measurements (13.00%)
3 (3.00%) low mild
5 (5.00%) high mild
5 (5.00%) high severe
filter optimize low selectivity (kept 1/1024)
time: [1.1569 µs 1.1718 µs 1.1925 µs]
Found 12 outliers among 100 measurements (12.00%)
2 (2.00%) high mild
10 (10.00%) high severe
filter u8 (kept 1/2) time: [107.89 µs 108.18 µs 108.63 µs]
Found 12 outliers among 100 measurements (12.00%)
5 (5.00%) low mild
5 (5.00%) high mild
2 (2.00%) high severe
filter u8 high selectivity (kept 1023/1024)
time: [2.3864 µs 2.3994 µs 2.4165 µs]
Found 18 outliers among 100 measurements (18.00%)
1 (1.00%) low severe
2 (2.00%) low mild
6 (6.00%) high mild
9 (9.00%) high severe
filter u8 low selectivity (kept 1/1024)
time: [1.2988 µs 1.3011 µs 1.3036 µs]
Found 5 outliers among 100 measurements (5.00%)
2 (2.00%) low mild
3 (3.00%) high mild
filter context u8 (kept 1/2)
time: [10.775 µs 10.803 µs 10.835 µs]
Found 2 outliers among 100 measurements (2.00%)
1 (1.00%) high mild
1 (1.00%) high severe
filter context u8 high selectivity (kept 1023/1024)
time: [1.3241 µs 1.3297 µs 1.3371 µs]
Found 5 outliers among 100 measurements (5.00%)
4 (4.00%) high mild
1 (1.00%) high severe
filter context u8 low selectivity (kept 1/1024)
time: [218.56 ns 219.12 ns 219.68 ns]
Found 5 outliers among 100 measurements (5.00%)
2 (2.00%) low mild
3 (3.00%) high mild
filter i32 (kept 1/2) time: [109.58 µs 109.70 µs 109.83 µs]
Found 3 outliers among 100 measurements (3.00%)
2 (2.00%) high mild
1 (1.00%) high severe
filter i32 high selectivity (kept 1023/1024)
time: [6.9736 µs 6.9930 µs 7.0150 µs]
Found 9 outliers among 100 measurements (9.00%)
1 (1.00%) low mild
5 (5.00%) high mild
3 (3.00%) high severe
filter i32 low selectivity (kept 1/1024)
time: [1.2492 µs 1.2539 µs 1.2607 µs]
Found 2 outliers among 100 measurements (2.00%)
1 (1.00%) high mild
1 (1.00%) high severe
filter context i32 (kept 1/2)
time: [11.006 µs 11.039 µs 11.088 µs]
Found 1 outliers among 100 measurements (1.00%)
1 (1.00%) high severe
filter context i32 high selectivity (kept 1023/1024)
time: [6.4559 µs 6.5295 µs 6.6090 µs]
Found 7 outliers among 100 measurements (7.00%)
2 (2.00%) high mild
5 (5.00%) high severe
filter context i32 low selectivity (kept 1/1024)
time: [187.45 ns 189.07 ns 190.80 ns]
Found 3 outliers among 100 measurements (3.00%)
3 (3.00%) high mild
filter context i32 w NULLs (kept 1/2)
time: [39.095 µs 39.134 µs 39.174 µs]
Found 10 outliers among 100 measurements (10.00%)
4 (4.00%) low mild
3 (3.00%) high mild
3 (3.00%) high severe
filter context i32 w NULLs high selectivity (kept 1023/1024)
time: [9.8810 µs 9.9135 µs 9.9479 µs]
Found 7 outliers among 100 measurements (7.00%)
5 (5.00%) high mild
2 (2.00%) high severe
filter context i32 w NULLs low selectivity (kept 1/1024)
time: [378.16 ns 379.77 ns 381.76 ns]
Found 7 outliers among 100 measurements (7.00%)
4 (4.00%) high mild
3 (3.00%) high severe
filter context u8 w NULLs (kept 1/2)
time: [38.769 µs 39.277 µs 40.111 µs]
Found 7 outliers among 100 measurements (7.00%)
1 (1.00%) low mild
1 (1.00%) high mild
5 (5.00%) high severe
filter context u8 w NULLs high selectivity (kept 1023/1024)
time: [5.2810 µs 5.2918 µs 5.3034 µs]
Found 3 outliers among 100 measurements (3.00%)
1 (1.00%) low mild
2 (2.00%) high severe
filter context u8 w NULLs low selectivity (kept 1/1024)
time: [396.76 ns 398.03 ns 399.52 ns]
Found 4 outliers among 100 measurements (4.00%)
3 (3.00%) high mild
1 (1.00%) high severe
filter f32 (kept 1/2) time: [225.86 µs 226.22 µs 226.62 µs]
Found 10 outliers among 100 measurements (10.00%)
1 (1.00%) low mild
7 (7.00%) high mild
2 (2.00%) high severe
filter context f32 (kept 1/2)
time: [39.001 µs 39.063 µs 39.133 µs]
Found 6 outliers among 100 measurements (6.00%)
1 (1.00%) low mild
4 (4.00%) high mild
1 (1.00%) high severe
filter context f32 high selectivity (kept 1023/1024)
time: [9.8491 µs 9.8671 µs 9.8865 µs]
Found 6 outliers among 100 measurements (6.00%)
6 (6.00%) high mild
filter context f32 low selectivity (kept 1/1024)
time: [382.18 ns 384.13 ns 387.05 ns]
Found 10 outliers among 100 measurements (10.00%)
7 (7.00%) high mild
3 (3.00%) high severe
filter decimal128 (kept 1/2)
time: [114.21 µs 114.59 µs 115.03 µs]
Found 14 outliers among 100 measurements (14.00%)
3 (3.00%) low mild
6 (6.00%) high mild
5 (5.00%) high severe
filter decimal128 high selectivity (kept 1023/1024)
time: [19.883 µs 19.931 µs 19.979 µs]
Found 2 outliers among 100 measurements (2.00%)
2 (2.00%) high mild
filter decimal128 low selectivity (kept 1/1024)
time: [1.2756 µs 1.2800 µs 1.2872 µs]
Found 11 outliers among 100 measurements (11.00%)
3 (3.00%) low mild
6 (6.00%) high mild
2 (2.00%) high severe
filter context decimal128 (kept 1/2)
time: [15.669 µs 15.728 µs 15.792 µs]
Found 4 outliers among 100 measurements (4.00%)
4 (4.00%) high mild
filter context decimal128 high selectivity (kept 1023/1024)
time: [19.516 µs 19.576 µs 19.657 µs]
Found 5 outliers among 100 measurements (5.00%)
3 (3.00%) high mild
2 (2.00%) high severe
filter context decimal128 low selectivity (kept 1/1024)
time: [198.30 ns 199.32 ns 200.72 ns]
Found 4 outliers among 100 measurements (4.00%)
3 (3.00%) high mild
1 (1.00%) high severe
filter context string (kept 1/2)
time: [248.81 µs 249.42 µs 250.03 µs]
Found 2 outliers among 100 measurements (2.00%)
2 (2.00%) high severe
filter context string high selectivity (kept 1023/1024)
time: [101.29 µs 102.20 µs 103.12 µs]
Found 3 outliers among 100 measurements (3.00%)
3 (3.00%) high mild
filter context string low selectivity (kept 1/1024)
time: [789.42 ns 791.92 ns 794.52 ns]
Found 7 outliers among 100 measurements (7.00%)
3 (3.00%) low mild
4 (4.00%) high mild
filter context string dictionary (kept 1/2)
time: [11.479 µs 11.491 µs 11.504 µs]
Found 17 outliers among 100 measurements (17.00%)
3 (3.00%) low severe
5 (5.00%) low mild
6 (6.00%) high mild
3 (3.00%) high severe
filter context string dictionary high selectivity (kept 1023/1024)
time: [6.7771 µs 6.8049 µs 6.8496 µs]
Found 11 outliers among 100 measurements (11.00%)
3 (3.00%) low mild
3 (3.00%) high mild
5 (5.00%) high severe
filter context string dictionary low selectivity (kept 1/1024)
time: [486.21 ns 1.0693 µs 2.3499 µs]
Found 2 outliers among 100 measurements (2.00%)
1 (1.00%) high mild
1 (1.00%) high severe
filter context string dictionary w NULLs (kept 1/2)
time: [39.316 µs 39.395 µs 39.481 µs]
Found 10 outliers among 100 measurements (10.00%)
5 (5.00%) low mild
3 (3.00%) high mild
2 (2.00%) high severe
filter context string dictionary w NULLs high selectivity (kept 1023/1024)
time: [10.224 µs 10.256 µs 10.289 µs]
Found 15 outliers among 100 measurements (15.00%)
1 (1.00%) low severe
2 (2.00%) low mild
9 (9.00%) high mild
3 (3.00%) high severe
filter context string dictionary w NULLs low selectivity (kept 1/1024)
time: [683.57 ns 685.92 ns 689.36 ns]
Found 15 outliers among 100 measurements (15.00%)
1 (1.00%) low mild
4 (4.00%) high mild
10 (10.00%) high severe
filter single record batch
time: [108.48 µs 108.71 µs 108.96 µs]
Found 14 outliers among 100 measurements (14.00%)
2 (2.00%) low mild
10 (10.00%) high mild
2 (2.00%) high severe
```
### This PR
```
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
