niko86 commented on issue #10439: URL: https://github.com/apache/arrow-rs/issues/10439#issuecomment-5274981953
Correction to my previous comment, and a better citation — the conclusion is unchanged but two details in it were wrong. **1. The version that matters is pyarrow, not pandas.** I said the reproducer stopped reproducing under pandas 3.0.5. Isolating the variables properly, pandas is irrelevant: | | pyarrow 25.0.0 | pyarrow 25.0.1 | |---|---|---| | pandas 2.3.3 | SIGSEGV | survives | | pandas 3.0.5 | SIGSEGV | survives | **2. The right upstream issue is [microsoft/mimalloc#1327](https://github.com/microsoft/mimalloc/issues/1327)**, not #1287. I cited #1287 second-hand without reading it; it is "mimalloc >= 3.3.0 causes segmentation faults when used from multiple threads" — a threading bug, not this. #1327 is the co-residency report: *"Two independently-linked mimalloc v3 instances … SIGSEGV in `_mi_theap_collect_retired` at process exit"*. **Better still, Arrow already has this from its own side: [apache/arrow GH-50428](https://github.com/apache/arrow/issues/50428)** — *"pyarrow 24.0.0 regression: co-loading a native extension bundling mimalloc v3 SIGSEGVs in bundled mimalloc"* — milestone **25.0.1**, fixed by [#50549](https://github.com/apache/arrow/pull/50549) "Better mimalloc configuration on macOS". That issue independently names switching the extension's mimalloc copy to v2 as the workaround, which is what the A/B in my previous comment measured, and notes CPython 3.14 vendors its own mimalloc v3 as a third instance. So this was a pyarrow-side regression with an arrow-rs-shaped symptom, and it is fixed in pyarrow 25.0.1. Confirming across 50 array shapes on an unchanged mimalloc-v3 consumer: pyarrow 25.0.0 corrupts 44/50, pyarrow 25.0.1 corrupts 0/50. Staying closed — nothing here for arrow-rs. Apologies for the noisy trail; leaving the corrected record for anyone who lands on this from a search. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
