This is an automated email from the ASF dual-hosted git repository.
dongjoon-hyun pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/spark-connect-swift.git
The following commit(s) were added to refs/heads/main by this push:
new b3e3f1b [SPARK-58539] Support error details in `SparkConnectError`
using gRPC `ErrorInfo`
b3e3f1b is described below
commit b3e3f1bd530338d6716c747410ed2815617956b8
Author: Dongjoon Hyun <[email protected]>
AuthorDate: Mon Aug 3 22:28:31 2026 -0700
[SPARK-58539] Support error details in `SparkConnectError` using gRPC
`ErrorInfo`
### What changes were proposed in this pull request?
This PR improves `SparkConnectError` to carry error details and replaces
message-substring-based
error classification with `google.rpc.ErrorInfo` parsing.
1. Each `SparkConnectError` case now carries a `Details` payload
(`message`, `errorClass`,
`sqlState`, `messageParameters`). The cases are renamed to
`lowerCamelCase` while the original
spellings are kept as compatibility static properties, so the existing
`throw`/`catch`/
`#expect(throws:)` call sites continue to compile unchanged.
2. A new `GrpcErrorConverter` classifies errors by the Spark error class
from `ErrorInfo` of the
gRPC status details, like the PySpark and Scala clients.
Message-substring matching remains only
as a fallback.
3. The three duplicated substring-matching sites
(`SparkConnectClient.withGPRC`,
`DataFrame.withGPRC`, `ddlParse`) are unified into the converter.
### Why are the changes needed?
Users could not access the server error message, error class, or SQLSTATE
from a caught error, and
the classification relied on fragile substring matching duplicated in three
places.
### Does this PR introduce _any_ user-facing change?
Yes, `SparkConnectError` provides detailed error information now.
```swift
} catch SparkConnectError.tableOrViewNotFound(let details) {
print(details.errorClass) // Optional("TABLE_OR_VIEW_NOT_FOUND")
print(details.sqlState) // Optional("42P01")
}
```
The existing code using the original case spellings continues to compile
and behave the same.
### How was this patch tested?
Pass the CIs with the newly added `SparkConnectErrorTests` test suite. All
existing test suites pass
unmodified, which proves the source compatibility of this change.
### Was this patch authored or co-authored using generative AI tooling?
Generated-by: Claude Fable 5
Closes #474 from dongjoon-hyun/SPARK-58539.
Authored-by: Dongjoon Hyun <[email protected]>
Signed-off-by: Dongjoon Hyun <[email protected]>
---
Sources/SparkConnect/DataFrame.swift | 15 +-
Sources/SparkConnect/GrpcErrorConverter.swift | 127 ++++++++++++++++
Sources/SparkConnect/SparkConnectClient.swift | 32 +---
Sources/SparkConnect/SparkConnectError.swift | 161 ++++++++++++++++++---
.../SparkConnectTests/SparkConnectErrorTests.swift | 83 +++++++++++
5 files changed, 357 insertions(+), 61 deletions(-)
diff --git a/Sources/SparkConnect/DataFrame.swift
b/Sources/SparkConnect/DataFrame.swift
index 98b6fda..f174139 100644
--- a/Sources/SparkConnect/DataFrame.swift
+++ b/Sources/SparkConnect/DataFrame.swift
@@ -317,20 +317,7 @@ public actor DataFrame: Sendable {
do {
return try await f(client)
} catch let error as RPCError where error.code == .internalError {
- switch error.message {
- case let m where m.contains("CATALOG_NOT_FOUND"):
- throw SparkConnectError.CatalogNotFound
- case let m where m.contains("SCHEMA_NOT_FOUND"):
- throw SparkConnectError.SchemaNotFound
- case let m where m.contains("TABLE_OR_VIEW_NOT_FOUND"):
- throw SparkConnectError.TableOrViewNotFound
- case let m where m.contains("UNRESOLVED_COLUMN.WITH_SUGGESTION"):
- throw SparkConnectError.ColumnNotFound
- case let m where m.contains("PARSE_SYNTAX_ERROR"):
- throw SparkConnectError.ParseSyntaxError
- default:
- throw error
- }
+ throw GrpcErrorConverter.convert(error) ?? error
}
}
}
diff --git a/Sources/SparkConnect/GrpcErrorConverter.swift
b/Sources/SparkConnect/GrpcErrorConverter.swift
new file mode 100644
index 0000000..b6cb2f4
--- /dev/null
+++ b/Sources/SparkConnect/GrpcErrorConverter.swift
@@ -0,0 +1,127 @@
+//
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+//
+#if canImport(FoundationEssentials)
+import FoundationEssentials
+#else
+import Foundation
+#endif
+import GRPCCore
+import GRPCProtobuf
+
+/// A converter from gRPC ``RPCError``s to ``SparkConnectError``s
+/// like `org.apache.spark.sql.connect.client.GrpcExceptionConverter`.
+///
+/// It parses the `google.rpc.ErrorInfo` from the gRPC status details to
classify errors by
+/// the Spark error class and to enrich
``SparkConnectError/Details-swift.struct`` with
+/// the error class, SQLSTATE, and message parameters. When the status details
are missing,
+/// it falls back to classifying by the error message.
+enum GrpcErrorConverter {
+ /// Convert an ``RPCError`` into a ``SparkConnectError``.
+ /// - Parameter error: An ``RPCError`` from the server.
+ /// - Returns: A ``SparkConnectError`` or nil if unclassifiable.
+ static func convert(_ error: RPCError) -> SparkConnectError? {
+ var details = SparkConnectError.Details(message: error.message)
+ if let status = try? error.unpackGoogleRPCStatus(),
+ let info = status.details.compactMap({ $0.errorInfo }).first
+ {
+ details = SparkConnectError.Details(
+ message: status.message.isEmpty ? error.message : status.message,
+ errorClass: info.metadata["errorClass"],
+ sqlState: info.metadata["sqlState"],
+ messageParameters: messageParameters(from:
info.metadata["messageParameters"]))
+ }
+ if let errorClass = details.errorClass, let converted =
convert(errorClass, details) {
+ return converted
+ }
+ return convert(byMessage: details.message, details)
+ }
+
+ /// Classify by the Spark error class from `ErrorInfo`.
+ private static func convert(
+ _ errorClass: String, _ details: SparkConnectError.Details
+ ) -> SparkConnectError? {
+ switch errorClass {
+ case "CATALOG_NOT_FOUND":
+ return .catalogNotFound(details)
+ case "DATA_SOURCE_NOT_FOUND":
+ return .dataSourceNotFound(details)
+ case "INVALID_HANDLE.SESSION_CLOSED":
+ return .sessionClosed(details)
+ case "INVALID_IDENTIFIER", "UNSUPPORTED_DATATYPE":
+ return .invalidType(details)
+ case "OUTPUT_TYPE_UNSPECIFIED":
+ return .outputTypeUnspecified(details)
+ case "PARSE_SYNTAX_ERROR":
+ return .parseSyntaxError(details)
+ case "SCHEMA_NOT_FOUND":
+ return .schemaNotFound(details)
+ case "SQL_CONF_NOT_FOUND":
+ return .sqlConfNotFound(details)
+ case "TABLE_OR_VIEW_ALREADY_EXISTS":
+ return .tableOrViewAlreadyExists(details)
+ case "TABLE_OR_VIEW_NOT_FOUND":
+ return .tableOrViewNotFound(details)
+ case "UNRESOLVED_COLUMN.WITH_SUGGESTION":
+ return .columnNotFound(details)
+ default:
+ return nil
+ }
+ }
+
+ /// Classify by the error message for legacy error classes and
+ /// servers that don't provide `ErrorInfo`.
+ private static func convert(
+ byMessage message: String, _ details: SparkConnectError.Details
+ ) -> SparkConnectError? {
+ switch message {
+ case let m where m.contains("CATALOG_NOT_FOUND"):
+ return .catalogNotFound(details)
+ case let m where m.contains("DATA_SOURCE_NOT_FOUND"):
+ return .dataSourceNotFound(details)
+ case let m where m.contains("INVALID_HANDLE.SESSION_CLOSED"):
+ return .sessionClosed(details)
+ case let m where m.contains("UNSUPPORTED_DATATYPE") ||
m.contains("INVALID_IDENTIFIER"):
+ return .invalidType(details)
+ case let m where m.contains("Invalid view name:"):
+ return .invalidViewName(details)
+ case let m where m.contains("OUTPUT_TYPE_UNSPECIFIED"):
+ return .outputTypeUnspecified(details)
+ case let m where m.contains("PARSE_SYNTAX_ERROR"):
+ return .parseSyntaxError(details)
+ case let m where m.contains("SCHEMA_NOT_FOUND"):
+ return .schemaNotFound(details)
+ case let m where m.contains("SQL_CONF_NOT_FOUND"):
+ return .sqlConfNotFound(details)
+ case let m where m.contains("TABLE_OR_VIEW_ALREADY_EXISTS"):
+ return .tableOrViewAlreadyExists(details)
+ case let m where m.contains("TABLE_OR_VIEW_NOT_FOUND"):
+ return .tableOrViewNotFound(details)
+ case let m where m.contains("UNRESOLVED_COLUMN.WITH_SUGGESTION"):
+ return .columnNotFound(details)
+ default:
+ return nil
+ }
+ }
+
+ /// Decode the JSON-encoded `messageParameters` of `ErrorInfo` metadata.
+ private static func messageParameters(from json: String?) -> [String:
String] {
+ guard let json, let data = json.data(using: .utf8) else { return [:] }
+ return (try? JSONDecoder().decode([String: String].self, from: data)) ??
[:]
+ }
+}
diff --git a/Sources/SparkConnect/SparkConnectClient.swift
b/Sources/SparkConnect/SparkConnectClient.swift
index 7f43579..878180b 100644
--- a/Sources/SparkConnect/SparkConnectClient.swift
+++ b/Sources/SparkConnect/SparkConnectClient.swift
@@ -145,24 +145,7 @@ public actor SparkConnectClient {
do {
return try await f(client)
} catch let error as RPCError where error.code == .internalError {
- switch error.message {
- case let m where m.contains("INVALID_HANDLE.SESSION_CLOSED"):
- throw SparkConnectError.SessionClosed
- case let m where m.contains("SQL_CONF_NOT_FOUND"):
- throw SparkConnectError.SqlConfNotFound
- case let m where m.contains("TABLE_OR_VIEW_ALREADY_EXISTS"):
- throw SparkConnectError.TableOrViewAlreadyExists
- case let m where m.contains("TABLE_OR_VIEW_NOT_FOUND"):
- throw SparkConnectError.TableOrViewNotFound
- case let m where m.contains("Invalid view name:"):
- throw SparkConnectError.InvalidViewName
- case let m where m.contains("DATA_SOURCE_NOT_FOUND"):
- throw SparkConnectError.DataSourceNotFound
- case let m where m.contains("OUTPUT_TYPE_UNSPECIFIED"):
- throw SparkConnectError.OutputTypeUnspecified
- default:
- throw error
- }
+ throw GrpcErrorConverter.convert(error) ?? error
}
}
}
@@ -1009,17 +992,8 @@ public actor SparkConnectClient {
ddlParse.ddlString = ddlString
return OneOf_Analyze.ddlParse(ddlParse)
})
- do {
- let response = try await service.analyzePlan(request)
- return response.ddlParse.parsed
- } catch let error as RPCError where error.code == .internalError {
- switch error.message {
- case let m where m.contains("UNSUPPORTED_DATATYPE") ||
m.contains("INVALID_IDENTIFIER"):
- throw SparkConnectError.InvalidType
- default:
- throw error
- }
- }
+ let response = try await service.analyzePlan(request)
+ return response.ddlParse.parsed
}
}
diff --git a/Sources/SparkConnect/SparkConnectError.swift
b/Sources/SparkConnect/SparkConnectError.swift
index 5fec560..f45bb62 100644
--- a/Sources/SparkConnect/SparkConnectError.swift
+++ b/Sources/SparkConnect/SparkConnectError.swift
@@ -18,22 +18,147 @@
//
/// A enum for ``SparkConnect`` package errors
-public enum SparkConnectError: Error {
- case CatalogNotFound
- case ColumnNotFound
- case DataSourceNotFound
- case InvalidArgument
- case InvalidArrowData
- case InvalidSessionID
- case InvalidType
- case InvalidViewName
- case LocalRelationTooLarge
- case OutputTypeUnspecified
- case ParseSyntaxError
- case SchemaNotFound
- case SessionClosed
- case SqlConfNotFound
- case TableOrViewAlreadyExists
- case TableOrViewNotFound
- case UnsupportedOperation
+///
+/// Each case carries a ``SparkConnectError/Details-swift.struct`` payload
with the error message
+/// and, for server-side errors, the Spark error class and SQLSTATE.
+///
+/// Note that the equality of ``SparkConnectError`` compares only the error
kind and
+/// ignores the attached ``SparkConnectError/Details-swift.struct``.
+public enum SparkConnectError: Error, Sendable, Equatable {
+ /// Detailed error information.
+ public struct Details: Sendable, Equatable {
+ /// An error message.
+ public let message: String
+ /// A Spark error class like `TABLE_OR_VIEW_NOT_FOUND` if the server
provides it.
+ public let errorClass: String?
+ /// A SQLSTATE like `42P01` if the server provides it.
+ public let sqlState: String?
+ /// The message parameters of the error class.
+ public let messageParameters: [String: String]
+
+ public init(
+ message: String = "",
+ errorClass: String? = nil,
+ sqlState: String? = nil,
+ messageParameters: [String: String] = [:]
+ ) {
+ self.message = message
+ self.errorClass = errorClass
+ self.sqlState = sqlState
+ self.messageParameters = messageParameters
+ }
+ }
+
+ case catalogNotFound(Details)
+ case columnNotFound(Details)
+ case dataSourceNotFound(Details)
+ case invalidArgument(Details)
+ case invalidArrowData(Details)
+ case invalidSessionID(Details)
+ case invalidType(Details)
+ case invalidViewName(Details)
+ case localRelationTooLarge(Details)
+ case outputTypeUnspecified(Details)
+ case parseSyntaxError(Details)
+ case schemaNotFound(Details)
+ case sessionClosed(Details)
+ case sqlConfNotFound(Details)
+ case tableOrViewAlreadyExists(Details)
+ case tableOrViewNotFound(Details)
+ case unsupportedOperation(Details)
+
+ // Compatibility members to keep the original case spellings working
+ // for `throw`/`catch`/`#expect(throws:)` call sites.
+ public static var CatalogNotFound: SparkConnectError {
.catalogNotFound(Details()) }
+ public static var ColumnNotFound: SparkConnectError {
.columnNotFound(Details()) }
+ public static var DataSourceNotFound: SparkConnectError {
.dataSourceNotFound(Details()) }
+ public static var InvalidArgument: SparkConnectError {
.invalidArgument(Details()) }
+ public static var InvalidArrowData: SparkConnectError {
.invalidArrowData(Details()) }
+ public static var InvalidSessionID: SparkConnectError {
.invalidSessionID(Details()) }
+ public static var InvalidType: SparkConnectError { .invalidType(Details()) }
+ public static var InvalidViewName: SparkConnectError {
.invalidViewName(Details()) }
+ public static var LocalRelationTooLarge: SparkConnectError {
.localRelationTooLarge(Details()) }
+ public static var OutputTypeUnspecified: SparkConnectError {
.outputTypeUnspecified(Details()) }
+ public static var ParseSyntaxError: SparkConnectError {
.parseSyntaxError(Details()) }
+ public static var SchemaNotFound: SparkConnectError {
.schemaNotFound(Details()) }
+ public static var SessionClosed: SparkConnectError {
.sessionClosed(Details()) }
+ public static var SqlConfNotFound: SparkConnectError {
.sqlConfNotFound(Details()) }
+ public static var TableOrViewAlreadyExists: SparkConnectError {
+ .tableOrViewAlreadyExists(Details())
+ }
+ public static var TableOrViewNotFound: SparkConnectError {
.tableOrViewNotFound(Details()) }
+ public static var UnsupportedOperation: SparkConnectError {
.unsupportedOperation(Details()) }
+
+ /// The detailed error information.
+ public var details: Details {
+ switch self {
+ case .catalogNotFound(let details), .columnNotFound(let details),
+ .dataSourceNotFound(let details), .invalidArgument(let details),
+ .invalidArrowData(let details), .invalidSessionID(let details),
+ .invalidType(let details), .invalidViewName(let details),
+ .localRelationTooLarge(let details), .outputTypeUnspecified(let details),
+ .parseSyntaxError(let details), .schemaNotFound(let details),
+ .sessionClosed(let details), .sqlConfNotFound(let details),
+ .tableOrViewAlreadyExists(let details), .tableOrViewNotFound(let
details),
+ .unsupportedOperation(let details):
+ return details
+ }
+ }
+
+ /// An error message.
+ public var message: String { details.message }
+
+ /// A Spark error class like `TABLE_OR_VIEW_NOT_FOUND` if the server
provides it.
+ public var errorClass: String? { details.errorClass }
+
+ /// A SQLSTATE like `42P01` if the server provides it.
+ public var sqlState: String? { details.sqlState }
+
+ /// The message parameters of the error class.
+ public var messageParameters: [String: String] { details.messageParameters }
+
+ private var caseName: String {
+ switch self {
+ case .catalogNotFound: return "catalogNotFound"
+ case .columnNotFound: return "columnNotFound"
+ case .dataSourceNotFound: return "dataSourceNotFound"
+ case .invalidArgument: return "invalidArgument"
+ case .invalidArrowData: return "invalidArrowData"
+ case .invalidSessionID: return "invalidSessionID"
+ case .invalidType: return "invalidType"
+ case .invalidViewName: return "invalidViewName"
+ case .localRelationTooLarge: return "localRelationTooLarge"
+ case .outputTypeUnspecified: return "outputTypeUnspecified"
+ case .parseSyntaxError: return "parseSyntaxError"
+ case .schemaNotFound: return "schemaNotFound"
+ case .sessionClosed: return "sessionClosed"
+ case .sqlConfNotFound: return "sqlConfNotFound"
+ case .tableOrViewAlreadyExists: return "tableOrViewAlreadyExists"
+ case .tableOrViewNotFound: return "tableOrViewNotFound"
+ case .unsupportedOperation: return "unsupportedOperation"
+ }
+ }
+
+ public static func == (lhs: SparkConnectError, rhs: SparkConnectError) ->
Bool {
+ return lhs.caseName == rhs.caseName
+ }
+}
+
+extension SparkConnectError: CustomStringConvertible {
+ public var description: String {
+ var result = self.caseName
+ if !self.message.isEmpty {
+ result += ": \(self.message)"
+ } else if let errorClass = self.errorClass {
+ result += ": [\(errorClass)]"
+ }
+ return result
+ }
+}
+
+/// Supports the compatibility members in `catch` expression patterns
+/// like `catch SparkConnectError.TableOrViewNotFound`.
+public func ~= (pattern: SparkConnectError, value: any Error) -> Bool {
+ guard let error = value as? SparkConnectError else { return false }
+ return error == pattern
}
diff --git a/Tests/SparkConnectTests/SparkConnectErrorTests.swift
b/Tests/SparkConnectTests/SparkConnectErrorTests.swift
new file mode 100644
index 0000000..b5e3cce
--- /dev/null
+++ b/Tests/SparkConnectTests/SparkConnectErrorTests.swift
@@ -0,0 +1,83 @@
+//
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+//
+
+import Testing
+
+@testable import SparkConnect
+
+/// A test suite for `SparkConnectError`
+@Suite(.serialized)
+struct SparkConnectErrorTests {
+ @Test
+ func tableOrViewNotFound() async throws {
+ let spark = try await SparkSession.builder.getOrCreate()
+ let error = try await #require(throws: SparkConnectError.self) {
+ try await spark.sql("SELECT * FROM
nonexistent_table_for_error_tests").count()
+ }
+ #expect(error == .TableOrViewNotFound)
+ #expect(error.errorClass == "TABLE_OR_VIEW_NOT_FOUND")
+ #expect(error.sqlState == "42P01")
+ #expect(error.message.contains("nonexistent_table_for_error_tests"))
+ #expect(
+
error.messageParameters["relationName"]?.contains("nonexistent_table_for_error_tests")
+ ?? false)
+ await spark.stop()
+ }
+
+ @Test
+ func parseSyntaxError() async throws {
+ let spark = try await SparkSession.builder.getOrCreate()
+ let error = try await #require(throws: SparkConnectError.self) {
+ try await spark.sql("SELECT 1 +").count()
+ }
+ #expect(error == .ParseSyntaxError)
+ #expect(error.errorClass == "PARSE_SYNTAX_ERROR")
+ #expect(error.sqlState == "42601")
+ await spark.stop()
+ }
+
+ @Test
+ func compatibilityPatterns() async throws {
+ let spark = try await SparkSession.builder.getOrCreate()
+ // The original case spellings continue to work in `#expect(throws:)` and
`catch`.
+ try await #require(throws: SparkConnectError.TableOrViewNotFound) {
+ try await spark.sql("SELECT * FROM
nonexistent_table_for_error_tests").count()
+ }
+ do {
+ _ = try await spark.sql("SELECT * FROM
nonexistent_table_for_error_tests").count()
+ Issue.record("Expected SparkConnectError.TableOrViewNotFound")
+ } catch SparkConnectError.TableOrViewNotFound {
+ // Expected.
+ }
+ await spark.stop()
+ }
+
+ @Test
+ func detailsPatternMatching() async throws {
+ let spark = try await SparkSession.builder.getOrCreate()
+ do {
+ _ = try await spark.sql("SELECT * FROM
nonexistent_table_for_error_tests").count()
+ Issue.record("Expected SparkConnectError.tableOrViewNotFound")
+ } catch SparkConnectError.tableOrViewNotFound(let details) {
+ #expect(details.errorClass == "TABLE_OR_VIEW_NOT_FOUND")
+ #expect(details.sqlState == "42P01")
+ }
+ await spark.stop()
+ }
+}
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]