j1wonpark opened a new pull request, #57831: URL: https://github.com/apache/spark/pull/57831
### What changes were proposed in this pull request? This PR makes the error condition framework support assigning a SQLSTATE at the sub-condition level, and uses it to fix the SQLSTATE of the `INVALID_HANDLE` session sub-conditions. Three changes, one per commit: 1. **Framework (`[CORE]`)**: add an optional `sqlState` field to `ErrorSubInfo`, and make `ErrorClassesJsonReader.getSqlState` resolve the sub-condition's SQLSTATE first, falling back to the main condition's. This is the same sub-condition-first resolution that `getBreakingChangeInfo` already uses; degenerate inputs (null, malformed names) keep `getSqlState`'s pre-existing lenient behavior. Sub-condition SQLSTATEs are validated against `error-states.json` like top-level ones, the rule is documented in the error README, and the error documentation generator renders them in sub-condition rows. 2. **`error-conditions.json` (`[CONNECT]`)**: assign SQLSTATE `08003` (connection does not exist) to `INVALID_HANDLE.SESSION_CHANGED`, `SESSION_CLOSED`, and `SESSION_NOT_FOUND`. The operation-level sub-conditions (`FORMAT`, `OPERATION_*`) concern a single operation on a healthy session and keep inheriting `HY000`. 3. **PySpark (`[PYTHON]`)**: make `get_sqlstate` fall back to the main condition's SQLSTATE when the sub-condition does not declare one, matching the JVM framework and `get_breaking_change_info`. This also fixes `NEAREST_BY_JOIN.*` reporting no SQLSTATE despite `NEAREST_BY_JOIN` declaring `42604`. ### Why are the changes needed? A SQLSTATE can currently only be assigned to a top-level error condition, so every sub-condition unconditionally inherits its parent's value. Giving one sub-condition a more accurate SQLSTATE requires a compatibility-breaking split into a new top-level condition. `INVALID_HANDLE.SESSION_*` shows why this matters: these errors mean the server-side session backing a Connect client is gone — exactly what SQL standard `08003` describes — but they inherit `INVALID_HANDLE`'s generic `HY000`, so tools that detect dead connections by SQLSTATE class `08` cannot recognize them. `HY000` was never a deliberate choice here: `INVALID_HANDLE` received it when it only covered handle-validity errors (SPARK-44422), and the `SESSION_*` sub-conditions were folded in later (SPARK-44421, SPARK-45680, SPARK-47380), inheriting it automatically. This was raised in the SPARK-57933 review (https://github.com/apache/spark/pull/57009#discussion_r3674458143): the Connect JDBC driver has to hard-code a `SESSION_*` → `08003` remapping because the framework cannot express it. Splitting `SESSION_*` into new top-level conditions is not viable, because deployed Connect clients match the `INVALID_HANDLE.SESSION_*` condition names to drive transparent reattach and session invalidation (`ResponseValidator`, `ExecutePlanResponseReattachableIterator`, and their Python counterparts). The framework change also benefits error JSON files loaded through the same `ErrorClassesJsonReader` outside the main file (e.g. `kafka-error-conditions.json`), which can now declare sub-condition SQLSTATEs the same way. ### Does this PR introduce _any_ user-facing change? Yes. - `SparkThrowable.getSqlState` (and the SQLSTATE surfaced to Connect clients) for `INVALID_HANDLE.SESSION_CHANGED`/`SESSION_CLOSED`/`SESSION_NOT_FOUND` changes from `HY000` to `08003`. Noted in `sql-migration-guide.md`. - PySpark's `pyspark.errors.exceptions.base.PySparkException.getSqlState` for a sub-condition without its own SQLSTATE now returns the main condition's SQLSTATE instead of `None` (e.g. `NEAREST_BY_JOIN.UNSUPPORTED_MODE`: `None` → `42604`). ### How was this patch tested? Each commit was written test-first (failing test confirmed before the fix). - `SparkThrowableSuite`: sub-condition SQLSTATE resolution (override, inheritance, degenerate inputs), the error-states.json invariant extended to sub-condition SQLSTATEs, golden-file round-trip for the new JSON field, and pins for `SESSION_*` → `08003` plus the derived expectation that all other `INVALID_HANDLE` sub-conditions stay `HY000`. 36 tests pass. - `python/pyspark/errors/tests/test_errors.py`: main-class fallback regression (`NEAREST_BY_JOIN.UNSUPPORTED_MODE` now reports `42604`), alongside the existing cases where neither level declares a sqlState. All 6 tests pass. - `JdbcErrorUtilsSuite` (13 tests) re-run to confirm the JDBC driver's existing hard-coded mapping stays consistent. ### Was this patch authored or co-authored using generative AI tooling? Generated-by: Claude Code (claude-fable-5) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
