This is an automated email from the ASF dual-hosted git repository.

gaogaotiantian pushed a commit to branch branch-4.x
in repository https://gitbox.apache.org/repos/asf/spark.git


The following commit(s) were added to refs/heads/branch-4.x by this push:
     new 9ddc37526aab [SPARK-60194][PYTHON] Improve type annotation for 
StreamingQueryListener sessions
9ddc37526aab is described below

commit 9ddc37526aabb1e5b8d4d9a000ede4b9a1aeaa72
Author: Tim Hsiung <[email protected]>
AuthorDate: Thu Nov 5 16:34:07 2026 -0800

    [SPARK-60194][PYTHON] Improve type annotation for StreamingQueryListener 
sessions
    
    ### What changes were proposed in this pull request?
    
    - Import the classic and Connect `SparkSession` types under `TYPE_CHECKING` 
and annotate the listener session with their union, removing three 
`name-defined` ignores and corresponding `F821` suppressions.
    - Declare `_sparkSession` with an optional type and a `None` default, 
replacing the getter's `getattr` with direct access.
    
    ### Why are the changes needed?
    
    `SparkSession` was an unresolved annotation. Listeners accept both classic 
and Connect sessions, which are separate types. The explicit default permits 
direct access before binding without requiring subclasses to call 
`super().__init__()`.
    
    ### Does this PR introduce _any_ user-facing change?
    
    Static type checkers can now resolve the listener session type. Runtime 
session-binding behavior is unchanged.
    
    ### How was this patch tested?
    
    - Mypy passed on `python/pyspark/sql/streaming/listener.py` and 
`python/pyspark/sql/connect/streaming/query.py`.
    - Local runtime probes using session stand-ins verified initial `None`, 
binding, repeated-binding no-overwrite behavior, direct setter assignment, 
instance isolation, and subclass initialization without `super()`.
    - Ruff lint/format checks and `git diff --check` passed.
    
    No new tests were added. The full streaming integration suite was not run; 
runtime validation was limited to session-binding logic.
    
    ### Was this patch authored or co-authored using generative AI tooling?
    
    Generated-by: pi 1.1.0
    
    Closes #59398 from bearomorphism/typing-streaming-listener-session.
    
    Authored-by: Tim Hsiung <[email protected]>
    Signed-off-by: Tian Gao <[email protected]>
    (cherry picked from commit 8faf7fd1f8b265d9be2b60cf98f12d1e14fc3c39)
    Signed-off-by: Tian Gao <[email protected]>
---
 python/pyspark/sql/streaming/listener.py | 15 ++++++++++-----
 1 file changed, 10 insertions(+), 5 deletions(-)

diff --git a/python/pyspark/sql/streaming/listener.py 
b/python/pyspark/sql/streaming/listener.py
index 76fec6cf159b..24ea2f2b43bb 100644
--- a/python/pyspark/sql/streaming/listener.py
+++ b/python/pyspark/sql/streaming/listener.py
@@ -17,7 +17,7 @@
 import json
 import uuid
 from abc import ABC, abstractmethod
-from typing import TYPE_CHECKING, Any, Dict, List, Optional, Set
+from typing import TYPE_CHECKING, Any, Dict, List, Optional, Set, Union
 
 from pyspark import cloudpickle
 from pyspark.sql.types import Row
@@ -27,6 +27,9 @@ __all__ = ["StreamingQueryListener"]
 if TYPE_CHECKING:
     from py4j.java_gateway import JavaObject
 
+    from pyspark.sql.connect.session import SparkSession as ConnectSparkSession
+    from pyspark.sql.session import SparkSession
+
 
 class StreamingQueryListener(ABC):
     """
@@ -63,19 +66,21 @@ class StreamingQueryListener(ABC):
     >>> spark.streams.addListener(MyListener())
     """
 
+    _sparkSession: Optional[Union["SparkSession", "ConnectSparkSession"]] = 
None
+
     def _set_spark_session(
         self,
-        session: "SparkSession",  # type: ignore[name-defined] # noqa: F821
+        session: Union["SparkSession", "ConnectSparkSession"],
     ) -> None:
         if self.spark is None:
             self.spark = session
 
     @property
-    def spark(self) -> Optional["SparkSession"]:  # type: ignore[name-defined] 
# noqa: F821
-        return getattr(self, "_sparkSession", None)
+    def spark(self) -> Optional[Union["SparkSession", "ConnectSparkSession"]]:
+        return self._sparkSession
 
     @spark.setter
-    def spark(self, session: "SparkSession") -> None:  # type: 
ignore[name-defined] # noqa: F821
+    def spark(self, session: Union["SparkSession", "ConnectSparkSession"]) -> 
None:
         # For backward compatibility
         self._sparkSession = session
 


---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Reply via email to