This is an automated email from the ASF dual-hosted git repository.
pan3793 pushed a commit to branch branch-4.x
in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/branch-4.x by this push:
new 1e4c6b96905b [SPARK-58684][PYTHON] Make PySpark error sub-conditions
inherit the parent condition error state
1e4c6b96905b is described below
commit 1e4c6b96905b5f09d978c511f8aabeb5cdf6172e
Author: Jiwon Park <[email protected]>
AuthorDate: Sat Aug 15 04:48:18 2026 +0800
[SPARK-58684][PYTHON] Make PySpark error sub-conditions inherit the parent
condition error state
### What changes were proposed in this pull request?
`ErrorClassesReader.get_sqlstate` looks the error state up under the
sub-condition, so a sub-condition reports `None` even when its condition
declares one. This makes the lookup resolve on the condition only, matching
`ErrorClassesJsonReader.getSqlState` on the JVM side.
Split out of #57831; the two are unrelated, as
`python/pyspark/errors/error-conditions.json` is a separate file and does not
contain the condition that PR is about.
### Why are the changes needed?
`getSqlState()` is public API on `PySparkException`. `NEAREST_BY_JOIN`
declares `42604`, but:
```python
>>> from pyspark.errors import PySparkRuntimeError
>>> e = PySparkRuntimeError(
... errorClass="NEAREST_BY_JOIN.UNSUPPORTED_MODE",
... messageParameters={"mode": "invalid", "supported": "nearest"},
... )
>>> print(e.getSqlState())
None
```
The JVM reader resolves on the condition here, so the two sides report
different error states for the same condition.
### Does this PR introduce _any_ user-facing change?
Yes. `getSqlState()` returns the condition's error state for a
sub-condition instead of `None`: `NEAREST_BY_JOIN.*` goes from `None` to
`42604`, the only condition affected today. Where the condition itself declares
no error state, it still returns `None`.
### How was this patch tested?
Added a case to `test_sqlstate` pinning `NEAREST_BY_JOIN.UNSUPPORTED_MODE`
to `42604`, written test-first and confirmed failing before the fix. Added
`test_sqlstate_is_taken_from_the_main_class` covering a sub-condition that
declares a different error state, an unknown sub-condition name, and an unknown
condition. All 7 tests in `pyspark.errors.tests.test_errors` pass.
### Was this patch authored or co-authored using generative AI tooling?
Generated-by: Claude Code (claude-fable-5, claude-opus-5)
Closes #57887 from j1wonpark/pyspark-get-sqlstate-fallback.
Authored-by: Jiwon Park <[email protected]>
Signed-off-by: Cheng Pan <[email protected]>
(cherry picked from commit 59a50f8c68c6ab1168a6e508a94d13959e8264aa)
Signed-off-by: Cheng Pan <[email protected]>
---
python/pyspark/errors/tests/test_errors.py | 23 +++++++++++++++++++++++
python/pyspark/errors/utils.py | 14 +++++---------
2 files changed, 28 insertions(+), 9 deletions(-)
diff --git a/python/pyspark/errors/tests/test_errors.py
b/python/pyspark/errors/tests/test_errors.py
index 17ca6a34c6dd..e477f4a99b0e 100644
--- a/python/pyspark/errors/tests/test_errors.py
+++ b/python/pyspark/errors/tests/test_errors.py
@@ -118,12 +118,35 @@ class ErrorsTest(unittest.TestCase):
error = PySparkRuntimeError(errorClass="APPLICATION_NAME_NOT_SET",
messageParameters={})
self.assertIsNone(error.getSqlState())
+ # Neither the sub-class nor the main class declares a sqlState.
error = PySparkRuntimeError(
errorClass="SESSION_MUTATION_IN_DECLARATIVE_PIPELINE.SET_RUNTIME_CONF",
messageParameters={"method": "set"},
)
self.assertIsNone(error.getSqlState())
+ # A sub-class inherits the main class's sqlState.
+ error = PySparkRuntimeError(
+ errorClass="NEAREST_BY_JOIN.UNSUPPORTED_MODE",
+ messageParameters={"mode": "invalid", "supported": "nearest"},
+ )
+ self.assertEqual(error.getSqlState(), "42604")
+
+ def test_sqlstate_is_taken_from_the_main_class(self):
+ # The sqlState is looked up on the main class only, so a sub-class
entry never
+ # takes precedence and an unknown sub-class name still resolves.
+ error_reader = ErrorClassesReader()
+ error_reader.error_info_map = {
+ "TEST_ERROR": {
+ "message": ["Error message."],
+ "sqlState": "42604",
+ "sub_class": {"SUBCLASS": {"message": ["Subclass message."],
"sqlState": "08003"}},
+ },
+ }
+ self.assertEqual(error_reader.get_sqlstate("TEST_ERROR.SUBCLASS"),
"42604")
+
self.assertEqual(error_reader.get_sqlstate("TEST_ERROR.NON_EXISTENT_SUB"),
"42604")
+
self.assertIsNone(error_reader.get_sqlstate("NON_EXISTENT_ERROR.SUBCLASS"))
+
if __name__ == "__main__":
from pyspark.testing import main
diff --git a/python/pyspark/errors/utils.py b/python/pyspark/errors/utils.py
index c796e37a28c5..53a5e18901e6 100644
--- a/python/pyspark/errors/utils.py
+++ b/python/pyspark/errors/utils.py
@@ -98,20 +98,16 @@ class ErrorClassesReader:
def get_sqlstate(self, errorClass: Optional[str]) -> Optional[str]:
"""
Returns the SQL state for the given error class.
+
+ The SQL state is declared on the main class, so a sub-class inherits
it.
"""
if errorClass is None:
return None
- error_classes = errorClass.split(".")
- try:
- if len(error_classes) == 1:
- return self.error_info_map[errorClass]["sqlState"]
- else:
- return
self.error_info_map[error_classes[0]]["sub_class"][error_classes[1]][
- "sqlState"
- ]
- except KeyError:
+ main_class_info = self.error_info_map.get(errorClass.split(".")[0])
+ if main_class_info is None:
return None
+ return main_class_info.get("sqlState")
def get_error_message(self, errorClass: str, messageParameters: Dict[str,
str]) -> str:
"""
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
