WillAyd opened a new pull request, #326: URL: https://github.com/apache/arrow-nanoarrow/pull/326
In https://github.com/apache/arrow-nanoarrow/pull/280 the algorithms seemed to make a huge difference in performance on my intel x86 chip, but other platforms didn't see as much. I [asked on SO](https://stackoverflow.com/questions/77550709/x86-performance-difference-between-shift-and-add-when-packing-bits) why that is the case, and while I don't see anything yet definitive one of the commenters pointed to higher performance algorithms that should work more generally [This particular answer](https://stackoverflow.com/a/51750902/621736) gives an algorithm for using multiplication that might work more generically. When I test this in Cython from the steps in https://github.com/WillAyd/cython-nanoarrow I get the following numbers on my system: ```python In [1]: from comparisons import ComparisonManager ...: mgr = ComparisonManager() In [2]: %timeit mgr.unpack() ...: %timeit mgr.unpack_no_shift() ...: %timeit mgr.unpack_multiply() 300 µs ± 13 µs per loop (mean ± std. dev. of 7 runs, 1,000 loops each) 35 µs ± 1.07 µs per loop (mean ± std. dev. of 7 runs, 10,000 loops each) 61.4 µs ± 615 ns per loop (mean ± std. dev. of 7 runs, 10,000 loops each) In [3]: %timeit mgr.pack() ...: %timeit mgr.pack_no_shift() ...: %timeit mgr.pack_multiply() 80.2 µs ± 162 ns per loop (mean ± std. dev. of 7 runs, 10,000 loops each) 71.1 µs ± 123 ns per loop (mean ± std. dev. of 7 runs, 10,000 loops each) 65.5 µs ± 121 ns per loop (mean ± std. dev. of 7 runs, 10,000 loops each) ``` So this multiply technique for unpacking is actually a good deal slower on my x86, but might provide a slight performance boost when packing. On non x86 architecture I'd be curious to know if it makes a difference for anyone -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
