lidavidm commented on pull request #10520: URL: https://github.com/apache/arrow/pull/10520#issuecomment-861591295
Awkwardly, it's slower (with fewer columns, much faster with a lot of columns) once made columnwise. Before: ``` ----------------------------------------------------------------------------------------------- Benchmark Time CPU Iterations UserCounters... ----------------------------------------------------------------------------------------------- BinaryJoinArrayScalar 104808 ns 104807 ns 6670 bytes_per_second=1110.69M/s BinaryJoinArrayArray 115151 ns 115150 ns 6081 bytes_per_second=1010.93M/s BinaryJoinElementWiseArrayScalar/2 48782 ns 48781 ns 14300 bytes_per_second=481.829M/s BinaryJoinElementWiseArrayScalar/8 172532 ns 172526 ns 4088 bytes_per_second=540.323M/s BinaryJoinElementWiseArrayScalar/64 873553 ns 873533 ns 801 bytes_per_second=855.336M/s BinaryJoinElementWiseArrayScalar/128 1240809 ns 1240796 ns 568 bytes_per_second=1.17663G/s BinaryJoinElementWiseArrayArray/2 57097 ns 57096 ns 12238 bytes_per_second=411.659M/s BinaryJoinElementWiseArrayArray/8 184990 ns 184989 ns 3726 bytes_per_second=503.921M/s BinaryJoinElementWiseArrayArray/64 886392 ns 886393 ns 781 bytes_per_second=842.927M/s BinaryJoinElementWiseArrayArray/128 1229550 ns 1229457 ns 569 bytes_per_second=1.18749G/s ``` After: ``` ----------------------------------------------------------------------------------------------- Benchmark Time CPU Iterations UserCounters... ----------------------------------------------------------------------------------------------- BinaryJoinArrayScalar 107844 ns 107840 ns 6519 bytes_per_second=1079.45M/s BinaryJoinArrayArray 115818 ns 115816 ns 5997 bytes_per_second=1005.11M/s BinaryJoinElementWiseArrayScalar/2 46680 ns 46680 ns 14774 bytes_per_second=503.519M/s BinaryJoinElementWiseArrayScalar/8 164875 ns 164871 ns 4230 bytes_per_second=565.411M/s BinaryJoinElementWiseArrayScalar/64 796642 ns 796620 ns 884 bytes_per_second=937.919M/s BinaryJoinElementWiseArrayScalar/128 735257 ns 735242 ns 954 bytes_per_second=1.98569G/s BinaryJoinElementWiseArrayArray/2 58653 ns 58651 ns 11987 bytes_per_second=400.746M/s BinaryJoinElementWiseArrayArray/8 222740 ns 222735 ns 3147 bytes_per_second=418.523M/s BinaryJoinElementWiseArrayArray/64 966210 ns 966211 ns 727 bytes_per_second=773.294M/s BinaryJoinElementWiseArrayArray/128 833193 ns 833171 ns 850 bytes_per_second=1.75229G/s ``` This holds even for longer input arrays, though the crossover point moves down a bit. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
