pitrou commented on PR #43772:
URL: https://github.com/apache/arrow/pull/43772#issuecomment-2299365559
I'm getting the following performance numbers here (on a AMD Ryzen 9 3900X).
I'm quite surprised by the large performance improvement when the selection
factor gets smaller, I may be hitting a happier spot on this CPU's cache
performance profile.
* ChunkedChunkedInt64, RandomIndicesWithNulls:
```
TakeChunkedChunkedInt64RandomIndicesWithNulls/524288/1000 2856147 ns
2854511 ns 247 items_per_second=183.67M/s null_percent=0.1
selection_factor=1 size=524.288k
TakeChunkedChunkedInt64RandomIndicesWithNulls/524288/10 4053631 ns
4051724 ns 172 items_per_second=129.399M/s null_percent=10
selection_factor=1 size=524.288k
TakeChunkedChunkedInt64RandomIndicesWithNulls/524288/2 5930464 ns
5927578 ns 116 items_per_second=88.4489M/s null_percent=50
selection_factor=1 size=524.288k
TakeChunkedChunkedInt64RandomIndicesWithNulls/524288/1 495851 ns
495563 ns 1526 items_per_second=1.05796G/s null_percent=100
selection_factor=1 size=524.288k
TakeChunkedChunkedInt64RandomIndicesWithNulls/524288/0 1112236 ns
1111711 ns 627 items_per_second=471.605M/s null_percent=0
selection_factor=1 size=524.288k
TakeChunkedChunkedInt64FewRandomIndicesWithNulls/524288/1000 249334 ns
249258 ns 2837 items_per_second=2.10339G/s null_percent=0.1
selection_factor=0.05 size=524.288k
TakeChunkedChunkedInt64FewRandomIndicesWithNulls/524288/10 305795 ns
305688 ns 2281 items_per_second=1.71511G/s null_percent=10
selection_factor=0.05 size=524.288k
TakeChunkedChunkedInt64FewRandomIndicesWithNulls/524288/2 396929 ns
396797 ns 1727 items_per_second=1.3213G/s null_percent=50
selection_factor=0.05 size=524.288k
TakeChunkedChunkedInt64FewRandomIndicesWithNulls/524288/1 126274 ns
126226 ns 5105 items_per_second=4.15358G/s null_percent=100
selection_factor=0.05 size=524.288k
TakeChunkedChunkedInt64FewRandomIndicesWithNulls/524288/0 142551 ns
142508 ns 4943 items_per_second=3.679G/s null_percent=0
selection_factor=0.05 size=524.288k
```
* ChunkedChunkedInt64, MonotonicIndices:
```
TakeChunkedChunkedInt64MonotonicIndices/524288/1000 1851022 ns
1850096 ns 378 items_per_second=283.384M/s null_percent=0.1
selection_factor=1 size=524.288k
TakeChunkedChunkedInt64MonotonicIndices/524288/10 2243297 ns
2240733 ns 314 items_per_second=233.981M/s null_percent=10
selection_factor=1 size=524.288k
TakeChunkedChunkedInt64MonotonicIndices/524288/2 2984875 ns
2983417 ns 234 items_per_second=175.734M/s null_percent=50
selection_factor=1 size=524.288k
TakeChunkedChunkedInt64MonotonicIndices/524288/1 1462448 ns
1461616 ns 475 items_per_second=358.704M/s null_percent=100
selection_factor=1 size=524.288k
TakeChunkedChunkedInt64MonotonicIndices/524288/0 964273 ns
963779 ns 753 items_per_second=543.992M/s null_percent=0
selection_factor=1 size=524.288k
TakeChunkedChunkedInt64FewMonotonicIndices/524288/1000 235511 ns
235394 ns 3153 items_per_second=2.22728G/s null_percent=0.1
selection_factor=0.05 size=524.288k
TakeChunkedChunkedInt64FewMonotonicIndices/524288/10 230951 ns
230862 ns 3070 items_per_second=2.271G/s null_percent=10
selection_factor=0.05 size=524.288k
TakeChunkedChunkedInt64FewMonotonicIndices/524288/2 277764 ns
277652 ns 2465 items_per_second=1.88829G/s null_percent=50
selection_factor=0.05 size=524.288k
TakeChunkedChunkedInt64FewMonotonicIndices/524288/1 176958 ns
176892 ns 3861 items_per_second=2.96388G/s null_percent=100
selection_factor=0.05 size=524.288k
TakeChunkedChunkedInt64FewMonotonicIndices/524288/0 147876 ns
147817 ns 4446 items_per_second=3.54688G/s null_percent=0
selection_factor=0.05 size=524.288k
```
* ChunkedChunkedString, RandomIndicesWithNulls:
```
TakeChunkedChunkedStringRandomIndicesWithNulls/524288/1000 15470666 ns
15461055 ns 47 items_per_second=33.9102M/s null_percent=0.1
selection_factor=1 size=524.288k
TakeChunkedChunkedStringRandomIndicesWithNulls/524288/10 12907336 ns
12900470 ns 51 items_per_second=40.641M/s null_percent=10
selection_factor=1 size=524.288k
TakeChunkedChunkedStringRandomIndicesWithNulls/524288/2 8990573 ns
8985964 ns 77 items_per_second=58.3452M/s null_percent=50
selection_factor=1 size=524.288k
TakeChunkedChunkedStringRandomIndicesWithNulls/524288/1 394163 ns
394022 ns 1806 items_per_second=1.3306G/s null_percent=100
selection_factor=1 size=524.288k
TakeChunkedChunkedStringRandomIndicesWithNulls/524288/0 12734928 ns
12727596 ns 54 items_per_second=41.193M/s null_percent=0
selection_factor=1 size=524.288k
TakeChunkedChunkedStringFewRandomIndicesWithNulls/524288/1000 1942356 ns
1941095 ns 342 items_per_second=270.099M/s null_percent=0.1
selection_factor=0.05 size=524.288k
TakeChunkedChunkedStringFewRandomIndicesWithNulls/524288/10 1580940 ns
1579965 ns 438 items_per_second=331.835M/s null_percent=10
selection_factor=0.05 size=524.288k
TakeChunkedChunkedStringFewRandomIndicesWithNulls/524288/2 695625 ns
695341 ns 993 items_per_second=754.001M/s null_percent=50
selection_factor=0.05 size=524.288k
TakeChunkedChunkedStringFewRandomIndicesWithNulls/524288/1 95347 ns
95316 ns 7409 items_per_second=5.50053G/s null_percent=100
selection_factor=0.05 size=524.288k
TakeChunkedChunkedStringFewRandomIndicesWithNulls/524288/0 1698761 ns
1697593 ns 414 items_per_second=308.842M/s null_percent=0
selection_factor=0.05 size=524.288k
```
* ChunkedChunkedString, MonotonicIndices:
```
TakeChunkedChunkedStringMonotonicIndices/524288/1000 7325552 ns
7320937 ns 96 items_per_second=71.6149M/s null_percent=0.1
selection_factor=1 size=524.288k
TakeChunkedChunkedStringMonotonicIndices/524288/10 7200098 ns
7195196 ns 97 items_per_second=72.8664M/s null_percent=10
selection_factor=1 size=524.288k
TakeChunkedChunkedStringMonotonicIndices/524288/2 5921411 ns
5918158 ns 119 items_per_second=88.5897M/s null_percent=50
selection_factor=1 size=524.288k
TakeChunkedChunkedStringMonotonicIndices/524288/1 2297901 ns
2297025 ns 305 items_per_second=228.247M/s null_percent=100
selection_factor=1 size=524.288k
TakeChunkedChunkedStringMonotonicIndices/524288/0 6317765 ns
6313923 ns 110 items_per_second=83.0368M/s null_percent=0
selection_factor=1 size=524.288k
TakeChunkedChunkedStringFewMonotonicIndices/524288/1000 1395142 ns
1394251 ns 485 items_per_second=376.036M/s null_percent=0.1
selection_factor=0.05 size=524.288k
TakeChunkedChunkedStringFewMonotonicIndices/524288/10 1156639 ns
1156008 ns 608 items_per_second=453.533M/s null_percent=10
selection_factor=0.05 size=524.288k
TakeChunkedChunkedStringFewMonotonicIndices/524288/2 648779 ns
648373 ns 1134 items_per_second=808.621M/s null_percent=50
selection_factor=0.05 size=524.288k
TakeChunkedChunkedStringFewMonotonicIndices/524288/1 189397 ns
189299 ns 3760 items_per_second=2.76963G/s null_percent=100
selection_factor=0.05 size=524.288k
TakeChunkedChunkedStringFewMonotonicIndices/524288/0 1254391 ns
1253469 ns 467 items_per_second=418.27M/s null_percent=0
selection_factor=0.05 size=524.288k
```
* ChunkedFlatInt64, RandomIndicesWithNulls:
```
TakeChunkedFlatInt64RandomIndicesWithNulls/524288/1000 2890688 ns
2888864 ns 244 items_per_second=181.486M/s null_percent=0.1
selection_factor=1 size=524.288k
TakeChunkedFlatInt64RandomIndicesWithNulls/524288/10 4091123 ns
4087122 ns 171 items_per_second=128.278M/s null_percent=10
selection_factor=1 size=524.288k
TakeChunkedFlatInt64RandomIndicesWithNulls/524288/2 5781369 ns
5778946 ns 119 items_per_second=90.7238M/s null_percent=50
selection_factor=1 size=524.288k
TakeChunkedFlatInt64RandomIndicesWithNulls/524288/1 278223 ns
278124 ns 2490 items_per_second=1.88509G/s null_percent=100
selection_factor=1 size=524.288k
TakeChunkedFlatInt64RandomIndicesWithNulls/524288/0 962126 ns
961576 ns 728 items_per_second=545.238M/s null_percent=0
selection_factor=1 size=524.288k
TakeChunkedFlatInt64FewRandomIndicesWithNulls/524288/1000 242604 ns
242527 ns 2899 items_per_second=2.16177G/s null_percent=0.1
selection_factor=0.05 size=524.288k
TakeChunkedFlatInt64FewRandomIndicesWithNulls/524288/10 301047 ns
300942 ns 2293 items_per_second=1.74216G/s null_percent=10
selection_factor=0.05 size=524.288k
TakeChunkedFlatInt64FewRandomIndicesWithNulls/524288/2 404648 ns
404492 ns 1699 items_per_second=1.29616G/s null_percent=50
selection_factor=0.05 size=524.288k
TakeChunkedFlatInt64FewRandomIndicesWithNulls/524288/1 125410 ns
125361 ns 5539 items_per_second=4.18222G/s null_percent=100
selection_factor=0.05 size=524.288k
TakeChunkedFlatInt64FewRandomIndicesWithNulls/524288/0 139041 ns
138994 ns 5080 items_per_second=3.77203G/s null_percent=0
selection_factor=0.05 size=524.288k
```
* ChunkedFlatInt64, MonotonicIndices:
```
TakeChunkedFlatInt64MonotonicIndices/524288/1000 1688520 ns
1687696 ns 415 items_per_second=310.653M/s null_percent=0.1
selection_factor=1 size=524.288k
TakeChunkedFlatInt64MonotonicIndices/524288/10 2032234 ns
2031098 ns 333 items_per_second=258.13M/s null_percent=10
selection_factor=1 size=524.288k
TakeChunkedFlatInt64MonotonicIndices/524288/2 2790558 ns
2789110 ns 251 items_per_second=187.977M/s null_percent=50
selection_factor=1 size=524.288k
TakeChunkedFlatInt64MonotonicIndices/524288/1 1292026 ns
1291438 ns 542 items_per_second=405.972M/s null_percent=100
selection_factor=1 size=524.288k
TakeChunkedFlatInt64MonotonicIndices/524288/0 762671 ns
762246 ns 917 items_per_second=687.82M/s null_percent=0
selection_factor=1 size=524.288k
TakeChunkedFlatInt64FewMonotonicIndices/524288/1000 217504 ns
217440 ns 3201 items_per_second=2.41118G/s null_percent=0.1
selection_factor=0.05 size=524.288k
TakeChunkedFlatInt64FewMonotonicIndices/524288/10 226309 ns
226222 ns 3100 items_per_second=2.31758G/s null_percent=10
selection_factor=0.05 size=524.288k
TakeChunkedFlatInt64FewMonotonicIndices/524288/2 274772 ns
274689 ns 2579 items_per_second=1.90866G/s null_percent=50
selection_factor=0.05 size=524.288k
TakeChunkedFlatInt64FewMonotonicIndices/524288/1 172513 ns
172440 ns 4023 items_per_second=3.0404G/s null_percent=100
selection_factor=0.05 size=524.288k
TakeChunkedFlatInt64FewMonotonicIndices/524288/0 143504 ns
143445 ns 4902 items_per_second=3.65499G/s null_percent=0
selection_factor=0.05 size=524.288k
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
