pitrou commented on issue #40874: URL: https://github.com/apache/arrow/issues/40874#issuecomment-2025673422
By the way, it seems the mimalloc/jemalloc really occurs for a certain range of sizes, above and below which they seem to have similar perf: ```python >>> %timeit arr[:250_000].cast(pa.float64(), safe=False, memory_pool=pa.mimalloc_memory_pool()) 77 µs ± 955 ns per loop (mean ± std. dev. of 7 runs, 10,000 loops each) >>> %timeit arr[:250_000].cast(pa.float64(), safe=False, memory_pool=pa.jemalloc_memory_pool()) 77.1 µs ± 696 ns per loop (mean ± std. dev. of 7 runs, 10,000 loops each) >>> %timeit arr[:500_000].cast(pa.float64(), safe=False, memory_pool=pa.mimalloc_memory_pool()) 149 µs ± 744 ns per loop (mean ± std. dev. of 7 runs, 10,000 loops each) >>> %timeit arr[:500_000].cast(pa.float64(), safe=False, memory_pool=pa.jemalloc_memory_pool()) 146 µs ± 197 ns per loop (mean ± std. dev. of 7 runs, 10,000 loops each) >>> %timeit arr[:1_000_000].cast(pa.float64(), safe=False, memory_pool=pa.mimalloc_memory_pool()) 550 µs ± 543 ns per loop (mean ± std. dev. of 7 runs, 1,000 loops each) >>> %timeit arr[:1_000_000].cast(pa.float64(), safe=False, memory_pool=pa.jemalloc_memory_pool()) 2.21 ms ± 6.7 µs per loop (mean ± std. dev. of 7 runs, 100 loops each) >>> %timeit arr[:2_000_000].cast(pa.float64(), safe=False, memory_pool=pa.mimalloc_memory_pool()) 1.89 ms ± 6.83 µs per loop (mean ± std. dev. of 7 runs, 1,000 loops each) >>> %timeit arr[:2_000_000].cast(pa.float64(), safe=False, memory_pool=pa.jemalloc_memory_pool()) 4.92 ms ± 60.8 µs per loop (mean ± std. dev. of 7 runs, 100 loops each) >>> %timeit arr[:4_000_000].cast(pa.float64(), safe=False, memory_pool=pa.mimalloc_memory_pool()) 3.76 ms ± 1.07 µs per loop (mean ± std. dev. of 7 runs, 100 loops each) >>> %timeit arr[:4_000_000].cast(pa.float64(), safe=False, memory_pool=pa.jemalloc_memory_pool()) 10.3 ms ± 27.2 µs per loop (mean ± std. dev. of 7 runs, 100 loops each) >>> %timeit arr[:8_000_000].cast(pa.float64(), safe=False, memory_pool=pa.mimalloc_memory_pool()) 21.9 ms ± 254 µs per loop (mean ± std. dev. of 7 runs, 10 loops each) >>> %timeit arr[:8_000_000].cast(pa.float64(), safe=False, memory_pool=pa.jemalloc_memory_pool()) 22 ms ± 128 µs per loop (mean ± std. dev. of 7 runs, 10 loops each) >>> %timeit arr[:16_000_000].cast(pa.float64(), safe=False, memory_pool=pa.mimalloc_memory_pool()) 41.6 ms ± 21.6 µs per loop (mean ± std. dev. of 7 runs, 10 loops each) >>> %timeit arr[:16_000_000].cast(pa.float64(), safe=False, memory_pool=pa.jemalloc_memory_pool()) 40.4 ms ± 376 µs per loop (mean ± std. dev. of 7 runs, 10 loops each) ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
