This is an automated email from the ASF dual-hosted git repository.

HyukjinKwon pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/spark-connect-rust.git


The following commit(s) were added to refs/heads/master by this push:
     new 281cecf  [SPARK-61122][CONNECT][PYTHON] Re-export the exceptions of 
pyspark.errors from pyspark.sql.utils, in the Python client as PySpark does
281cecf is described below

commit 281cecf25245350bdf3b6eb432fab5dd866e9844
Author: Hyukjin Kwon <[email protected]>
AuthorDate: Sun Nov 8 06:05:23 2026 +0900

    [SPARK-61122][CONNECT][PYTHON] Re-export the exceptions of pyspark.errors 
from pyspark.sql.utils, in the Python client as PySpark does
    
    ### What changes were proposed in this pull request?
    
    [SPARK-61122](https://issues.apache.org/jira/browse/SPARK-61122): 
`pyspark.sql.utils` re-exports the exceptions of `pyspark.errors`, as PySpark's 
does: `AnalysisException`, `ParseException`, `IllegalArgumentException`, 
`StreamingQueryException`, `QueryExecutionException`, `PythonException`, 
`UnknownException`, `SparkUpgradeException`, `PySparkImportError`, 
`PySparkNotImplementedError`, `PySparkRuntimeError` and `CapturedException`.
    
    ### Why are the changes needed?
    
    `pyspark.sql.utils` is the import path of these exceptions before 
`pyspark.errors`, and user code and Apache Spark's tests still use it, as in 
`from pyspark.sql.utils import AnalysisException`. The client re-exported none 
of them, so that import raised `ImportError`.
    
    ### Does this PR introduce _any_ user-facing change?
    
    Yes. `from pyspark.sql.utils import AnalysisException`, and the other 
exceptions, work as with PySpark 4.3.0.
    
    ### How was this patch tested?
    
    - A new offline test checks:
      - that each exception of `pyspark.sql.utils` is that of `pyspark.errors`
      - that the error of a mock Connect server is an instance of 
`pyspark.sql.utils.AnalysisException`
    - The lib tests of both crates (590 passed), the build of all tests, also 
with `wasm-udf`, the offline Python suites (2166 passed, 8 skipped), and the 
live drop-in and Arrow suites (173 passed) and live cases (10 passed) against a 
local Spark 4.3.0 server. Clippy reports no diagnostics in the changed lines.
    
    ### Was this patch authored or co-authored using generative AI tooling?
    
    Generated-by: Claude Opus 5.5
    
    This pull request and its description were written by Isaac.
    
    Closes #317 from HyukjinKwon/SPARK-61122-sql-utils-exceptions.
    
    Authored-by: Hyukjin Kwon <[email protected]>
    Signed-off-by: Hyukjin Kwon <[email protected]>
---
 python/pyspark/sql/utils.py         | 20 +++++++++++++++++++-
 python/tests/test_dropin_offline.py | 36 ++++++++++++++++++++++++++++++++++++
 2 files changed, 55 insertions(+), 1 deletion(-)

diff --git a/python/pyspark/sql/utils.py b/python/pyspark/sql/utils.py
index f8cc0bd..5e9b5b4 100644
--- a/python/pyspark/sql/utils.py
+++ b/python/pyspark/sql/utils.py
@@ -14,11 +14,29 @@
 # See the License for the specific language governing permissions and
 # limitations under the License.
 #
-"""Small subset of pyspark.sql.utils needed by the drop-in and vendored 
modules."""
+"""Small subset of pyspark.sql.utils needed by the drop-in and vendored 
modules, and the
+exceptions that PySpark's re-exports from ``pyspark.errors``, of their import 
path before it."""
 
 from enum import Enum
 from typing import Any, Union, TYPE_CHECKING, Sequence
 
+# Re-exported, as PySpark's pyspark.sql.utils does: `from pyspark.sql.utils 
import
+# AnalysisException` is their import path before pyspark.errors.
+from pyspark.errors import (  # noqa: F401
+    AnalysisException,
+    ParseException,
+    IllegalArgumentException,
+    StreamingQueryException,
+    QueryExecutionException,
+    PythonException,
+    UnknownException,
+    SparkUpgradeException,
+    PySparkImportError,
+    PySparkNotImplementedError,
+    PySparkRuntimeError,
+)
+from pyspark.errors.exceptions.captured import CapturedException  # noqa: F401
+
 if TYPE_CHECKING:
     from pyspark.pandas._typing import IndexOpsLike, SeriesOrIndex
 
diff --git a/python/tests/test_dropin_offline.py 
b/python/tests/test_dropin_offline.py
index a8666e6..514a859 100644
--- a/python/tests/test_dropin_offline.py
+++ b/python/tests/test_dropin_offline.py
@@ -944,6 +944,42 @@ def test_functions_exports_fresh_process():
     assert result.returncode == 0, result.stdout + result.stderr
 
 
+def test_sql_utils_reexports_the_exceptions_of_pyspark(mock_connect):
+    # As PySpark's pyspark.sql.utils: the exceptions of pyspark.errors, of 
their import path
+    # before it, which the errors of the server are instances of.
+    import pyspark.errors as errors
+    import pyspark.sql.utils as utils
+    from pyspark.errors.exceptions.captured import CapturedException
+
+    for name in [
+        "AnalysisException",
+        "ParseException",
+        "IllegalArgumentException",
+        "StreamingQueryException",
+        "QueryExecutionException",
+        "PythonException",
+        "UnknownException",
+        "SparkUpgradeException",
+        "PySparkImportError",
+        "PySparkNotImplementedError",
+        "PySparkRuntimeError",
+    ]:
+        assert getattr(utils, name) is getattr(errors, name), name
+    assert utils.CapturedException is CapturedException
+
+    spark, service = mock_connect
+    classes = ["org.apache.spark.sql.AnalysisException", "java.lang.Exception"]
+    message = "[TABLE_OR_VIEW_NOT_FOUND] The table or view `t` cannot be 
found."
+    service.execute_error = (message, classes[0], _error_info(classes))
+    service.confs = {
+        "spark.sql.connect.serverStacktrace.enabled": "false",
+        "spark.sql.pyspark.jvmStacktrace.enabled": "false",
+    }
+    with pytest.raises(utils.AnalysisException) as error:
+        spark.range(1).collect()
+    assert "TABLE_OR_VIEW_NOT_FOUND" in str(error.value)
+
+
 def test_functions_build_columns():
     from pyspark.sql import functions as F
     c = F.col("id")


---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Reply via email to