Jefffrey commented on PR #10262:
URL: https://github.com/apache/arrow-rs/pull/10262#issuecomment-4899367844
i get some pretty good improvements locally which seem to align with your
table of when the optimization should fire:
```diff
encode/fixed/8192x4 time: [6.3223 µs 6.3640 µs 6.4222 µs]
thrpt: [38.071 GiB/s 38.419 GiB/s 38.672 GiB/s]
change:
time: [+0.2755% +0.7585% +1.4886%] (p = 0.01 <
0.05)
thrpt: [−1.4668% −0.7528% −0.2747%]
Change within noise threshold.
Found 8 outliers among 100 measurements (8.00%)
1 (1.00%) high mild
7 (7.00%) high severe
encode/fixed/8192x8 time: [11.304 µs 11.314 µs 11.325 µs]
thrpt: [43.178 GiB/s 43.221 GiB/s 43.258 GiB/s]
change:
time: [+0.5348% +0.6824% +0.8462%] (p = 0.00 <
0.05)
thrpt: [−0.8391% −0.6778% −0.5319%]
Change within noise threshold.
Found 5 outliers among 100 measurements (5.00%)
3 (3.00%) high mild
2 (2.00%) high severe
encode/fixed/65536x4 time: [41.689 µs 41.794 µs 41.910 µs]
thrpt: [46.611 GiB/s 46.741 GiB/s 46.859 GiB/s]
change:
time: [+0.3794% +0.6562% +0.9363%] (p = 0.00 <
0.05)
thrpt: [−0.9277% −0.6520% −0.3779%]
Change within noise threshold.
Found 5 outliers among 100 measurements (5.00%)
5 (5.00%) high mild
encode/fixed/65536x8 time: [71.963 µs 72.078 µs 72.204 µs]
thrpt: [54.110 GiB/s 54.205 GiB/s 54.291 GiB/s]
change:
+ time: [−15.766% −15.492% −15.202%] (p = 0.00 <
0.05)
+ thrpt: [+17.927% +18.332% +18.717%]
+ Performance has improved.
Found 13 outliers among 100 measurements (13.00%)
10 (10.00%) high mild
3 (3.00%) high severe
encode/nested/8192x4 time: [13.922 µs 13.934 µs 13.947 µs]
thrpt: [43.819 GiB/s 43.859 GiB/s 43.897 GiB/s]
change:
time: [+0.0302% +0.2961% +0.5299%] (p = 0.02 <
0.05)
thrpt: [−0.5271% −0.2952% −0.0302%]
Change within noise threshold.
Found 3 outliers among 100 measurements (3.00%)
2 (2.00%) high mild
1 (1.00%) high severe
encode/nested/8192x8 time: [26.193 µs 26.229 µs 26.281 µs]
thrpt: [46.509 GiB/s 46.600 GiB/s 46.664 GiB/s]
change:
time: [+0.6524% +0.8748% +1.0919%] (p = 0.00 <
0.05)
thrpt: [−1.0801% −0.8672% −0.6481%]
Change within noise threshold.
Found 12 outliers among 100 measurements (12.00%)
6 (6.00%) high mild
6 (6.00%) high severe
encode/nested/65536x4 time: [115.74 µs 116.08 µs 116.39 µs]
thrpt: [41.958 GiB/s 42.070 GiB/s 42.195 GiB/s]
change:
+ time: [−28.716% −28.414% −28.110%] (p = 0.00 <
0.05)
+ thrpt: [+39.102% +39.692% +40.284%]
+ Performance has improved.
Found 1 outliers among 100 measurements (1.00%)
1 (1.00%) high mild
encode/nested/65536x8 time: [234.71 µs 234.93 µs 235.16 µs]
thrpt: [41.535 GiB/s 41.576 GiB/s 41.615 GiB/s]
change:
+ time: [−37.571% −36.777% −36.012%] (p = 0.00 <
0.05)
+ thrpt: [+56.279% +58.169% +60.183%]
+ Performance has improved.
Found 21 outliers among 100 measurements (21.00%)
15 (15.00%) high mild
6 (6.00%) high severe
encode/variable/8192x4 time: [16.089 µs 16.187 µs 16.314 µs]
thrpt: [67.386 GiB/s 67.912 GiB/s 68.327 GiB/s]
change:
- time: [+4.3447% +5.2324% +5.9907%] (p = 0.00 <
0.05)
- thrpt: [−5.6521% −4.9723% −4.1638%]
- Performance has regressed.
encode/variable/8192x8 time: [37.548 µs 37.609 µs 37.707 µs]
thrpt: [58.308 GiB/s 58.460 GiB/s 58.555 GiB/s]
change:
+ time: [−17.988% −15.154% −12.140%] (p = 0.00 <
0.05)
+ thrpt: [+13.818% +17.860% +21.933%]
+ Performance has improved.
Found 7 outliers among 100 measurements (7.00%)
3 (3.00%) high mild
4 (4.00%) high severe
encode/variable/65536x4 time: [164.37 µs 165.07 µs 165.61 µs]
thrpt: [53.075 GiB/s 53.250 GiB/s 53.475 GiB/s]
change:
+ time: [−41.190% −40.372% −39.560%] (p = 0.00 <
0.05)
+ thrpt: [+65.454% +67.706% +70.038%]
+ Performance has improved.
encode/variable/65536x8 time: [323.54 µs 325.42 µs 327.03 µs]
thrpt: [53.755 GiB/s 54.021 GiB/s 54.334 GiB/s]
change:
+ time: [−38.285% −37.638% −37.081%] (p = 0.00 <
0.05)
+ thrpt: [+58.934% +60.354% +62.034%]
+ Performance has improved.
```
- for reference i'm running on an M4 macbook
only bit of noise is `encode/variable/8192x4` showing a minor regression
even though the optimization apparently shouldn't have effect. but the other
results are quite clearly a significant improvement. maybe theres something
about how the benchmark runner is setup where memory optimizations like this
might not be as apparent?
can you produce these benchmark results locally as well?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
