gkotian commented on issue #50471:
URL: https://github.com/apache/arrow/issues/50471#issuecomment-5003137160
We hit this same crash in production and can confirm it from a different
platform and a different trigger path than the original report.
**Environment:** Linux x86_64 (`python:3.14-slim` Debian container on
Kubernetes; also reproduced on a local Linux machine), Python 3.14,
pyarrow 25.0.0, polars 1.42.1, pandas 2.3.3, Streamlit 1.53.1.
**Trigger:** not the pandas 3 eager-import path described above — we are on
pandas 2.3.3. Our entry point is `polars.DataFrame.to_pandas()` inside a
Streamlit app: libarrow is first imported on a short-lived Streamlit
`ScriptRunner` thread, and the crash fires when the conversion
(`pyarrow.Table.to_pandas` → `table_to_dataframe`) performs its first
allocation on an Arrow C++ thread-pool worker. Same mechanism, different
entry point — so it is not specific to pandas 3 or `NdarrayToArrow`.
Faulthandler output (trimmed):
```
Fatal Python error: Segmentation fault
Current thread 0x00007fe72bfff6c0 [ScriptRunner.sc] (most recent call
first):
<no Python frame> # Arrow C++ pool worker; thread name inherited from
creator
Thread 0x00007fe6457ff6c0 [ScriptRunner.sc] (most recent call first):
File "site-packages/pyarrow/pandas_compat.py", line 827 in
table_to_dataframe
File "site-packages/polars/dataframe/frame.py", line 2691 in
_to_pandas_without_object_columns
File "site-packages/polars/dataframe/frame.py", line 2629 in to_pandas
...
Current thread's C stack trace (most recent call first):
libarrow.so.2500 +0x16f8b89 / +0x16feaa1 / +0x16ee5c7
libarrow.so.2500 +0xb953bb / +0xb9bd9c / +0xb9be29
libarrow.so.2500 arrow::AllocateBuffer(long long, long long,
arrow::MemoryPool*)+0x35
libarrow.so.2500 arrow::AllocateBuffer(long long,
arrow::MemoryPool*)+0x11
libarrow_python.so.2500 +0x5fc70 / +0x79cfb / +0x5d705 / +0x5f96d /
+0x5eb5e / +0x6f2c4
libarrow.so.2500 +0xbdb7f6 / +0xbdb1c4 # thread pool task loop
libstdc++.so.6 +0xeb919 # std::thread
```
Operational impact matches the report: the whole Streamlit server dies with
SIGSEGV and no Python traceback. On Kubernetes this presents as unexplained
pod crash-loops immediately after deployment (first page loads), which took
some effort to attribute.
**Workaround confirmed:** `ARROW_DEFAULT_MEMORY_POOL=system` — deployed to
production, no crashes since (previously two crashes within the first two
minutes of the pod's life).
**Possibly useful for resolving this:** mimalloc **v3.4.1** (tagged
2026-07-15) now contains the fix commit
microsoft/mimalloc@b92c116b67d0 — I verified the commit is an ancestor of
the `v3.4.1` tag. So a tagged mimalloc release containing the fix exists
now, which was not the case when this issue was opened; bumping
`ARROW_MIMALLOC_BUILD_VERSION` may be viable for a 25.0.1.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
