sdf-jkl opened a new pull request, #11460: URL: https://github.com/apache/arrow-rs/pull/11460
## Which issue does this PR close? None. This is a sibling experiment to @HippoBaro’s [#11449](https://github.com/apache/arrow-rs/pull/11449). ## Rationale for this change #11449 reports substantial execution-order-dependent variance in Arrow writer benchmarks and addresses it using jemalloc with page purging disabled. This PR explores mimalloc as an alternative, consistent with the allocator used by DataFusion’s CLI and benchmark harness. We’ll compare default mimalloc against mimalloc with purging disabled to investigate performance and benchmark stability. ## What changes are included in this PR? Use mimalloc as the global allocator for the Linux `arrow_writer` benchmark executable. The dependency is development-only; library consumers are unaffected. Purging remains configurable through `MIMALLOC_PURGE_DELAY`. Two bot runs with four shards each are planned: - Default mimalloc settings. - `MIMALLOC_PURGE_DELAY=-1` on the PR side, retaining freed pages for reuse. Both runs will compare against the same baseline. These initial comparisons will help assess performance; repeated runs are needed to establish whether variance improves. ## Are these changes tested? - Formatting and compilation with the bot’s benchmark features passed. - The `primitive/default` smoke test passed with `MIMALLOC_PURGE_DELAY=-1`; allocator diagnostics confirmed purging was disabled. - Bot results pending. ## Are there any significant user-facing changes? None. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
