cyb70289 commented on issue #50503:
URL: https://github.com/apache/arrow/issues/50503#issuecomment-5041932116
This is strange. I can reproduce in on a `m7gd.xlarge` instance, pyarrow
25.0.0, ubuntu-24.04, 6.17.0-1017-aws.
Using @Klavinco 's test code:
```
import pyarrow as pa, pyarrow.parquet as pq
vals = list(range(100_000, 100_300))
pq.write_table(pa.table({"x": pa.array(vals, pa.int64())}),
"/tmp/min.parquet")
back = pq.read_table("/tmp/min.parquet")["x"].to_pylist()
print("match:", back == vals, "| unique:", len(set(back)), "of", len(back))
# Graviton3 + pyarrow 25.0.0: match: False | unique: 84 of 300
# same + ARROW_USER_SIMD_LEVEL=NONE: match: True | unique: 300 of 300
# same box + pyarrow 24.0.0: match: True | unique: 300 of 300
```
Surprisingly, `ARROW_USER_SIMD_LEVEL=SVE128` passes the test.
```
$ pip install pyarrow==25.0.0
# fail
$ python test.py
match: False | unique: 84 of 300
# fail
$ ARROW_USER_SIMD_LEVEL=SVE256 python test.py
match: False | unique: 84 of 300
# pass
$ ARROW_USER_SIMD_LEVEL=NONE python test.py
match: True | unique: 300 of 300
# pass
$ ARROW_USER_SIMD_LEVEL=SVE128 python test.py
match: True | unique: 300 of 300
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
