This is an automated email from the ASF dual-hosted git repository.

Yicong-Huang pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/spark.git


The following commit(s) were added to refs/heads/master by this push:
     new e0c6db19f1d4 [SPARK-60195][PYTHON] Improve type annotation for 
StatCounter callables
e0c6db19f1d4 is described below

commit e0c6db19f1d4583c25fb252efd586e02d920c623
Author: Tim Hsiung <[email protected]>
AuthorDate: Wed Nov 4 19:55:58 2026 +0000

    [SPARK-60195][PYTHON] Improve type annotation for StatCounter callables
    
    ### What changes were proposed in this pull request?
    
    Declare callable types for `maximum`, `minimum`, and `sqrt` in 
`StatCounter.__init__`, then remove the three assignment type ignores on the 
standard-library fallbacks.
    
    ### Why are the changes needed?
    
    The initial NumPy imports infer NumPy-specific callable types. Explicit 
scalar callable signatures describe both the NumPy implementations and their 
`max`, `min`, and `math.sqrt` fallbacks without suppressions.
    
    ### Does this PR introduce _any_ user-facing change?
    
    No. Function selection and statistical calculations are unchanged.
    
    ### How was this patch tested?
    
    - Existing `pyspark.tests.test_statcounter` suite passed.
    - Local probes produced identical baseline/modified results with NumPy and 
with its import forced to fail, including empty input, merging, self-merging, 
copying, and statistics output.
    - Mypy on `python/pyspark/statcounter.py`, Ruff lint/format checks, and 
`git diff --check` passed.
    
    No new tests were added; validation used the existing suite and local 
equivalence probes.
    
    ### Was this patch authored or co-authored using generative AI tooling?
    
    Generated-by: pi 1.1.0
    
    Closes #59396 from bearomorphism/typing-statcounter-callables.
    
    Authored-by: Tim Hsiung <[email protected]>
    Signed-off-by: Yicong-Huang <[email protected]>
---
 python/pyspark/statcounter.py | 11 +++++++----
 1 file changed, 7 insertions(+), 4 deletions(-)

diff --git a/python/pyspark/statcounter.py b/python/pyspark/statcounter.py
index 92bf0b11a3e1..05c93dfc8d5a 100644
--- a/python/pyspark/statcounter.py
+++ b/python/pyspark/statcounter.py
@@ -19,17 +19,20 @@
 
 import copy
 import math
-from typing import Dict, Iterable, Optional
+from typing import Callable, Dict, Iterable, Optional
 
 
 class StatCounter:
     def __init__(self, values: Optional[Iterable[float]] = None):
+        maximum: Callable[[float, float], float]
+        minimum: Callable[[float, float], float]
+        sqrt: Callable[[float], float]
         try:
             from numpy import maximum, minimum, sqrt
         except ImportError:
-            maximum = max  # type: ignore[assignment]
-            minimum = min  # type: ignore[assignment]
-            sqrt = math.sqrt  # type: ignore[assignment]
+            maximum = max
+            minimum = min
+            sqrt = math.sqrt
 
         self.maximum = maximum
         self.minimum = minimum


---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Reply via email to