eaglenebulaplayer opened a new issue, #50503:
URL: https://github.com/apache/arrow/issues/50503
### Describe the bug, including details regarding any error messages,
version, and platform.
**Summary**: the pyarrow **25.0.0** manylinux **aarch64** wheel
deterministically fails to decode dictionary-encoded Parquet on **AWS Graviton3
(Neoverse V1, CPU part 0xd40)** hosts. The same files, same container image,
same code path work correctly on Graviton2 (Neoverse N1, 0xd0c) and Graviton4
(Neoverse V2, 0xd4f), and everything works on all three CPU generations with
pyarrow **24.0.0**.
**Symptoms on Graviton3 + 25.0.0**:
- `ParquetFile.iter_batches()` with default `use_threads=True` → hard
**SIGSEGV** (faulthandler shows the crash inside native decode).
- `use_threads=False` → no crash, but raises `ArrowInvalid: Index not in
dictionary bounds` — i.e. single-threaded decode is *also* wrong, so this is
not a threading race and `use_threads=False` is not a mitigation.
**Determinism evidence** (from an internal investigation; counts are per 8
identical scan tasks on a fixed corpus per arm, tasks landing on random ARM
host generations on AWS Fargate):
- pyarrow 25.0.0, defaults: 3/8 tasks failed — exactly the tasks placed on
0xd40 hosts.
- pyarrow 25.0.0, `use_threads=False`: 4/8 failed with `ArrowInvalid` —
again exactly the 0xd40 placements.
- Same base image with only `pyarrow==24.0.0` swapped in: 8/8 green across
all host generations.
- Within a failing task: re-decoding the exact same bytes twice fails
identically; re-downloading the file (sha256-verified identical) fails
identically; the same file decoded on N1/V2 hosts or on x86-64 is byte-for-byte
correct. 12 probe tasks reading `/proc/cpuinfo`: failure correlates 100% with
CPU part 0xd40.
- The failing wheel was installed from PyPI (`pyarrow==25.0.0`, cp312
manylinux aarch64) in a `python:3.12-slim` (Debian, glibc 2.41) container.
**Files that trigger it**: zstd-compressed Parquet written by earlier
pyarrow (`write_table`, defaults → dictionary-encoded string columns), ~500k
rows per file, ~27 columns (strings, int64, date32, bool, one timestamp).
Unfortunately the data is proprietary financial content and we cannot attach a
triggering file. We have not yet managed to isolate a synthetic reproducer (we
currently have no pinned-Graviton3 test host — Fargate placement is random),
but the failure is 100% deterministic per (file, 0xd40 host, 25.0.0).
**Hypothesis**: a Neoverse-V1-specific code path in the 25.0.0 aarch64 build
— V1 is the only one of the three affected/unaffected cores with 256-bit SVE
(N1 has no SVE, V2 is 128-bit SVE2), so an SVE(-width)-dispatched kernel in the
dictionary decode path is our leading suspect. Related plumbing: #47769
(dynamic dispatch on ARM64) / xsimd bump #49921.
**Repro shape** (on a Graviton3 host, e.g. EC2 c7g/m7g/r7g, with any
sufficiently large dictionary-encoded zstd Parquet):
```python
import pyarrow.parquet as pq
pf = pq.ParquetFile(path)
for batch in pf.iter_batches(batch_size=5000): # SIGSEGV on 0xd40 with
25.0.0
pass
# use_threads=False instead → ArrowInvalid: Index not in dictionary bounds
```
We are happy to run candidate wheels/builds or additional diagnostics
(faulthandler tracebacks, `ARROW_USER_SIMD_LEVEL` sweeps, etc.) on Graviton3
and report back — please tell us what would be most useful.
### Component(s)
Python, C++
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
