This is an automated email from the ASF dual-hosted git repository.
dongjoon-hyun pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/spark-connect-swift.git
The following commit(s) were added to refs/heads/main by this push:
new 4372aad [SPARK-60063] Fix
Catalog.listDatabases/getDatabase/getFunction to not crash on null locationUri
and className
4372aad is described below
commit 4372aad10988b65e3cd925eab2223b41b947b919
Author: Dongjoon Hyun <[email protected]>
AuthorDate: Wed Oct 7 16:15:28 2026 -0700
[SPARK-60063] Fix Catalog.listDatabases/getDatabase/getFunction to not
crash on null locationUri and className
### What changes were proposed in this pull request?
This PR aims to fix `Catalog.listDatabases`, `Catalog.getDatabase`, and
`Catalog.getFunction` to handle nullable `locationUri` and `className` columns
gracefully by falling back to an empty string, like `Catalog.listFunctions`
already does.
### Why are the changes needed?
Previously, these APIs force-cast the `locationUri` and `className` columns
with `as! String`, which crashes the client process when the server returns
`null`.
- Apache Spark returns `locationUri = null` for V2 catalog namespaces
without a location property (e.g., `JDBCTableCatalog`).
- Apache Spark returns `className = null` for virtual operators such as
`case`, `<>`, `!=`, and `||`.
```
Could not cast value of type 'Swift.Optional<Any>' to 'Swift.String'.
```
### Does this PR introduce _any_ user-facing change?
Yes. These APIs now return an empty string instead of crashing. There is no
API change because `Database.locationUri` and `Function.className` remain
`String`.
### How was this patch tested?
Pass the CIs with the newly added test cases. I also verified them manually
against an Apache Spark 4.2.0 Connect server.
```
$ swift test --no-parallel --filter CatalogTests
...
Test run with 41 tests in 1 suite passed
```
### Was this patch authored or co-authored using generative AI tooling?
Generated-by: Claude Opus 5.5
Closes #589 from dongjoon-hyun/SPARK-60063.
Authored-by: Dongjoon Hyun <[email protected]>
Signed-off-by: Dongjoon Hyun <[email protected]>
---
Sources/SparkConnect/Catalog.swift | 8 +++----
Tests/SparkConnectTests/CatalogTests.swift | 34 ++++++++++++++++++++++++++++++
2 files changed, 38 insertions(+), 4 deletions(-)
diff --git a/Sources/SparkConnect/Catalog.swift
b/Sources/SparkConnect/Catalog.swift
index a36789c..cbd44e5 100644
--- a/Sources/SparkConnect/Catalog.swift
+++ b/Sources/SparkConnect/Catalog.swift
@@ -229,7 +229,7 @@ public actor Catalog: Sendable {
return try await df.collect().map {
try Database(
name: $0[0] as! String, catalog: $0[1] as? String, description: $0[2]
as? String,
- locationUri: $0[3] as! String)
+ locationUri: ($0[3] as? String) ?? "")
}
}
@@ -247,7 +247,7 @@ public actor Catalog: Sendable {
return try await df.collect().map {
try Database(
name: $0[0] as! String, catalog: $0[1] as? String, description: $0[2]
as? String,
- locationUri: $0[3] as! String)
+ locationUri: ($0[3] as? String) ?? "")
}.firstOrThrow()
}
@@ -538,7 +538,7 @@ public actor Catalog: Sendable {
catalog: $0[1] as? String,
namespace: $0[2] as? [String],
description: $0[3] as? String,
- className: $0[4] as! String,
+ className: ($0[4] as? String) ?? "",
isTemporary: $0[5] as! Bool)
}.firstOrThrow()
}
@@ -563,7 +563,7 @@ public actor Catalog: Sendable {
catalog: $0[1] as? String,
namespace: $0[2] as? [String],
description: $0[3] as? String,
- className: $0[4] as! String,
+ className: ($0[4] as? String) ?? "",
isTemporary: $0[5] as! Bool)
}.firstOrThrow()
}
diff --git a/Tests/SparkConnectTests/CatalogTests.swift
b/Tests/SparkConnectTests/CatalogTests.swift
index e05a9f4..8927022 100644
--- a/Tests/SparkConnectTests/CatalogTests.swift
+++ b/Tests/SparkConnectTests/CatalogTests.swift
@@ -109,6 +109,30 @@ struct CatalogTests {
await spark.stop()
}
+ @Test
+ func listAndGetDatabaseWithoutLocation() async throws {
+ let spark = try await SparkSession.builder.getOrCreate()
+ let catalog = "derby"
+ try await spark.conf.set(
+ "spark.sql.catalog.\(catalog)",
+ "org.apache.spark.sql.execution.datasources.v2.jdbc.JDBCTableCatalog")
+ try await spark.conf.set(
+ "spark.sql.catalog.\(catalog).url",
"jdbc:derby:memory:\(UUID().uuidString);create=true")
+ try await spark.conf.set(
+ "spark.sql.catalog.\(catalog).driver",
"org.apache.derby.jdbc.EmbeddedDriver")
+ try await spark.catalog.setCurrentCatalog(catalog)
+
+ let dbs = try await spark.catalog.listDatabases(pattern: "APP")
+ #expect(dbs.count == 1)
+ #expect(dbs[0].name == "APP")
+ #expect(dbs[0].catalog == catalog)
+ #expect(dbs[0].locationUri == "")
+
+ let db = try await spark.catalog.getDatabase("APP")
+ #expect(db == dbs[0])
+ await spark.stop()
+ }
+
@Test
func createDatabase() async throws {
let spark = try await SparkSession.builder.getOrCreate()
@@ -380,6 +404,16 @@ struct CatalogTests {
await spark.stop()
}
+ @Test
+ func getFunctionWithoutClassName() async throws {
+ let spark = try await SparkSession.builder.getOrCreate()
+ let function = try await spark.catalog.getFunction("case")
+ #expect(function.name == "case")
+ #expect(function.isTemporary)
+ #expect(function.className == "")
+ await spark.stop()
+ }
+
@Test
func listFunctions() async throws {
let spark = try await SparkSession.builder.getOrCreate()
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
