cyb70289 commented on pull request #9635: URL: https://github.com/apache/arrow/pull/9635#issuecomment-791328725
Benchmark this patch against Numpy (build with gcc-7.5, tested on skylake) - float64: arrow is a bit slower than numpy - float32: arrow is a bit faster than numpy - int32: arrow is much faster than numpy, surprise! - int64: arrow is a bit faster than numpy ``` In [1]: import numpy as np In [2]: import pyarrow.compute as pc In [3]: f32 = np.arange(12345678, dtype='float32') In [4]: f64 = np.arange(12345678, dtype='float64') In [5]: i32 = np.arange(12345678, dtype='int32') In [6]: i64 = np.arange(12345678, dtype='int64') # float64: arrow is a bit slower than numpy In [7]: %timeit np.sum(f64) 7.74 ms ± 21.3 µs per loop (mean ± std. dev. of 7 runs, 100 loops each) In [8]: %timeit pc.sum(f64) 8.24 ms ± 20.1 µs per loop (mean ± std. dev. of 7 runs, 100 loops each) # float32: arrow is a bit faster than numpy In [9]: %timeit np.sum(f32, dtype='float64') 7.52 ms ± 19.8 µs per loop (mean ± std. dev. of 7 runs, 100 loops each) In [10]: %timeit pc.sum(f32) 6.52 ms ± 39.8 µs per loop (mean ± std. dev. of 7 runs, 100 loops each) # int32: arrow is much faster than numpy, surprise! In [11]: %timeit np.sum(i32, dtype='int64') 10.4 ms ± 35 µs per loop (mean ± std. dev. of 7 runs, 100 loops each) In [12]: %timeit pc.sum(i32) 4.39 ms ± 26.6 µs per loop (mean ± std. dev. of 7 runs, 100 loops each) # int64: arrow is a bit faster than numpy In [13]: %timeit np.sum(i64) 8.86 ms ± 23.6 µs per loop (mean ± std. dev. of 7 runs, 100 loops each) In [14]: %timeit pc.sum(i64) 7.54 ms ± 12.6 µs per loop (mean ± std. dev. of 7 runs, 100 loops each) ``` ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
