vyasr commented on issue #44342: URL: https://github.com/apache/arrow/issues/44342#issuecomment-2403406477
> Thanks but sorry. I couldn't find any hints in the thread apply all bt full result... No problem @kou, I know these kinds of issues can be a huge pain to track down, especially from this limited information. If it helps, you can see the error in [this GHA run](https://github.com/rapidsai/cudf/actions/runs/11259806360/job/31313370630?pr=17022) on [this PR](https://github.com/rapidsai/cudf/pull/17022). > When searching online, there seem to be issues with jemalloc on Linux aarch64, see https://github.com/jemalloc/jemalloc/issues/467 for example. @pitrou thanks for finding that! That makes sense since it certainly seems like the underlying issue comes from jemalloc and is not arrow-specific. > I would recommend you switch to mimalloc instead of jemalloc Good idea, at least for testing. I'm testing that now in [this GH workflow](https://github.com/rapidsai/cudf/actions/runs/11262786032?pr=17022). The arm wheel-tests-cudf job is the one to look out for, let's see if using mimalloc bypasses the issue. That being said: > Note that mimalloc becomes the default in 18.0.0 as well (see https://github.com/apache/arrow/issues/43254). On our side, perhaps we should simply disable jemalloc on Linux aarch64 wheels? This seems like the right long-term solution if your suggestion to try mimalloc works for me above. pyarrow is a common enough dependency that a user could end up having pyarrow loaded in their environment without even realizing it, and if the import alone is sufficient to trigger the seg fault it would be quite challenging for the average user to debug. Making mimalloc the default seems sufficient to me since IMHO it's reasonable to expect a user explicitly setting the allocator to recognize this as a potential cause, but I wouldn't be opposed to disabling jemalloc altogether on arm either. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
