This is an automated email from the ASF dual-hosted git repository.
zhengruifeng pushed a commit to branch branch-4.x
in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/branch-4.x by this push:
new a98225cf6354 [SPARK-58765][PYTHON][TEST][FOLLOWUP] Override ARM
float-to-int results in the from_pandas type-scalar test
a98225cf6354 is described below
commit a98225cf63542c3b9046525b43c9eeac67829a44
Author: Spenser Sun <[email protected]>
AuthorDate: Fri Aug 21 18:04:28 2026 +0800
[SPARK-58765][PYTHON][TEST][FOLLOWUP] Override ARM float-to-int results in
the from_pandas type-scalar test
### What changes were proposed in this pull request?
- The `float64:infinity -> int8/int64` (unsafe) cell in
`PyArrowArrayFromPandasTypeScalarTests` (SPARK-58765) is platform-dependent:
`inf -> int` is C++ undefined behavior, so ARM's `FCVT` saturates (`inf ->
INT64_MAX`, truncated to `-1` for int8) while x86 returns the "integer
indefinite" `INT_MIN`. The x86-generated golden therefore fails on the
scheduled ARM (aarch64) / macOS (arm64) Python-only builds.
- Add an ARM override (`platform.machine() in ("aarch64", "arm64")`) for
those two cells, following the existing pattern in
`test_pyarrow_array_cast.py`. Only the `unsafe` method needs it -- `safe=True`
rejects `inf -> int` on every platform.
- Test-only; goldens unchanged (the override is inert on x86).
### Why are the changes needed?
- The scheduled ARM and macOS Python-only builds are red because the golden
pins x86 values for this undefined-behavior conversion.
### Does this PR introduce _any_ user-facing change?
No.
### How was this patch tested?
- Inert on x86 -- the existing goldens are unchanged and the scalar tests
still pass locally.
- The ARM/macOS override values match the output in the failing scheduled
builds.
### Was this patch authored or co-authored using generative AI tooling?
Generated-by: Claude Code (Opus 4.8)
Closes #58199 from Spenserrrr/scalar-arm-override.
Authored-by: Spenser Sun <[email protected]>
Signed-off-by: Ruifeng Zheng <[email protected]>
(cherry picked from commit dce742353f94b48e30cde926a0f2e1afaf83fd15)
Signed-off-by: Ruifeng Zheng <[email protected]>
---
.../pyarrow/test_pyarrow_array_from_pandas_non_default.py | 8 ++++++++
1 file changed, 8 insertions(+)
diff --git
a/python/pyspark/tests/upstream/pyarrow/test_pyarrow_array_from_pandas_non_default.py
b/python/pyspark/tests/upstream/pyarrow/test_pyarrow_array_from_pandas_non_default.py
index d9a2c88fc603..c41c0d8cd7f3 100644
---
a/python/pyspark/tests/upstream/pyarrow/test_pyarrow_array_from_pandas_non_default.py
+++
b/python/pyspark/tests/upstream/pyarrow/test_pyarrow_array_from_pandas_non_default.py
@@ -64,6 +64,7 @@ The committed golden files were generated with pandas 2.3.3,
pyarrow 24.0.0, and
numpy 2.4.1.
"""
+import platform
import unittest
from pyspark.loose_version import LooseVersion
@@ -272,6 +273,13 @@ class PyArrowArrayFromPandasTypeScalarTests(
overrides[("large_binary[pyarrow]:standard", col)] = (
"[b'hello', b'world']@large_binary"
)
+ # inf -> int is undefined behavior in C++; x86 (the golden's platform)
yields the
+ # "integer indefinite" INT_MIN, while ARM's FCVT saturates to INT_MAX.
Linux
+ # aarch64 and macOS arm64 agree on these two cells (safe=True rejects
inf -> int on
+ # every platform, so only this unsafe method needs the override).
+ if platform.machine() in ("aarch64", "arm64"):
+ overrides[("float64:infinity", "int8")] = "[-1, 1]@int8"
+ overrides[("float64:infinity", "int64")] = "[9223372036854775807,
1]@int64"
self._compare_type_matrix(
safe=False,
golden_file_prefix="golden_pyarrow_array_from_pandas_type_scalar_unsafe",
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
