This is an automated email from the ASF dual-hosted git repository.

zhengruifeng pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/spark.git


The following commit(s) were added to refs/heads/master by this push:
     new b24e4d7a2cab [SPARK-58469][PS] Use native expressions for NumPy rint
b24e4d7a2cab is described below

commit b24e4d7a2cabebc95f7ea664ca3d3b5f29c68b83
Author: Ruifeng Zheng <[email protected]>
AuthorDate: Mon Aug 3 16:43:42 2026 +0800

    [SPARK-58469][PS] Use native expressions for NumPy rint
    
    ### What changes were proposed in this pull request?
    
    Replace the pandas UDF mapping for NumPy rint on pandas-on-Spark objects 
with the native Spark `rint` expression after casting the input to `double`. 
Add coverage for halfway values, signed zero, NaN, and infinities.
    
    ### Why are the changes needed?
    
    The operation can be evaluated by Spark without crossing the Python worker 
boundary while preserving the existing double result type.
    
    ### Does this PR introduce _any_ user-facing change?
    
    No.
    
    ### How was this patch tested?
    
    - Added parity coverage in `NumPyCompatTests.test_np_math_functions`.
    - Passed Ruff format and lint checks for the changed Python files.
    - Focused PySpark tests were not run locally.
    
    ### Was this patch authored or co-authored using generative AI tooling?
    
    Generated-by: Codex (GPT-5)
    
    Closes #57672 from zhengruifeng/pandas-native-rint-dev2.
    
    Authored-by: Ruifeng Zheng <[email protected]>
    Signed-off-by: Ruifeng Zheng <[email protected]>
---
 python/pyspark/pandas/numpy_compat.py            | 2 +-
 python/pyspark/pandas/tests/test_numpy_compat.py | 1 +
 2 files changed, 2 insertions(+), 1 deletion(-)

diff --git a/python/pyspark/pandas/numpy_compat.py 
b/python/pyspark/pandas/numpy_compat.py
index 316e2e9e3afd..baa107660185 100644
--- a/python/pyspark/pandas/numpy_compat.py
+++ b/python/pyspark/pandas/numpy_compat.py
@@ -68,7 +68,7 @@ unary_np_spark_mappings = {
     "reciprocal": pandas_udf(  # type: ignore[call-overload]
         lambda s: np.reciprocal(s), DoubleType()
     ),
-    "rint": pandas_udf(lambda s: np.rint(s), DoubleType()),  # type: 
ignore[call-overload]
+    "rint": lambda c: F.rint(c.cast("double")),
     "sign": F.signum,
     "signbit": lambda c: F.when(c < 0, True).otherwise(False),
     "sin": F.sin,
diff --git a/python/pyspark/pandas/tests/test_numpy_compat.py 
b/python/pyspark/pandas/tests/test_numpy_compat.py
index bd7ba4c3e210..219f21a6efbc 100644
--- a/python/pyspark/pandas/tests/test_numpy_compat.py
+++ b/python/pyspark/pandas/tests/test_numpy_compat.py
@@ -104,6 +104,7 @@ class NumPyCompatTestsMixin:
             (np.negative, [-np.inf, -64.0, -2.0, 0.0, 2.0, 64.0, np.inf, 
np.nan]),
             (np.positive, [-np.inf, -64.0, -2.0, 0.0, 2.0, 64.0, np.inf, 
np.nan]),
             (np.rad2deg, [-np.inf, -64.0, -np.pi, 0.0, np.pi, 64.0, np.inf, 
np.nan]),
+            (np.rint, [-np.inf, -2.5, -1.5, -0.5, -0.0, 0.0, 0.5, 1.5, 2.5, 
np.inf, np.nan]),
             (np.sign, [-np.inf, -64.0, -2.0, -0.0, 0.0, 2.0, 64.0, np.inf, 
np.nan]),
             (np.sinh, [-np.inf, -64.0, -2.0, 0.0, 2.0, 64.0, np.inf, np.nan]),
             (np.square, [-np.inf, -64.0, -2.0, 0.0, 2.0, 64.0, np.inf, 
np.nan]),


---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Reply via email to