This is an automated email from the ASF dual-hosted git repository.
dongjoon-hyun pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/spark-connect-swift.git
The following commit(s) were added to refs/heads/main by this push:
new c63d92f [SPARK-57392] Use Apache Spark 4.0.3 in integration tests
c63d92f is described below
commit c63d92f28274ec786d1363726e00f4265aac1ae1
Author: Dongjoon Hyun <[email protected]>
AuthorDate: Thu Jun 11 10:05:22 2026 -0700
[SPARK-57392] Use Apache Spark 4.0.3 in integration tests
### What changes were proposed in this pull request?
This PR aims to use Apache Spark `4.0.3` instead of `4.0.3 RC1` and `4.0.2`
in integration tests.
### Why are the changes needed?
Apache Spark 4.0.3 is officially released.
- https://spark.apache.org/news/spark-4-0-3-released.html
- https://github.com/apache/spark/releases/tag/v4.0.3
### Does this PR introduce _any_ user-facing change?
No, this is a test-only change.
### How was this patch tested?
Pass the CIs.
### Was this patch authored or co-authored using generative AI tooling?
Generated-by: Claude Code (Fable 5)
Closes #420 from dongjoon-hyun/SPARK-57392.
Authored-by: Dongjoon Hyun <[email protected]>
Signed-off-by: Dongjoon Hyun <[email protected]>
---
.github/workflows/build_and_test.yml | 12 ++++++------
AGENTS.md | 2 +-
2 files changed, 7 insertions(+), 7 deletions(-)
diff --git a/.github/workflows/build_and_test.yml
b/.github/workflows/build_and_test.yml
index 52d9426..f55ff5b 100644
--- a/.github/workflows/build_and_test.yml
+++ b/.github/workflows/build_and_test.yml
@@ -73,7 +73,7 @@ jobs:
SPARK_REMOTE: "sc://localhost:15003"
services:
spark:
- image: apache/spark:4.0.2-scala
+ image: apache/spark:4.0.3-scala
env:
SPARK_NO_DAEMONIZE: 1
ports:
@@ -176,7 +176,7 @@ jobs:
run: swift test --filter NOTHING -c release
- name: Test
run: |
- curl -LO
https://dist.apache.org/repos/dist/dev/spark/v4.0.3-rc1-bin/spark-4.0.3-bin-hadoop3.tgz
+ curl -LO
https://www.apache.org/dyn/closer.lua/spark/spark-4.0.3/spark-4.0.3-bin-hadoop3.tgz?action=download
tar xvfz spark-4.0.3-bin-hadoop3.tgz && rm spark-4.0.3-bin-hadoop3.tgz
mv spark-4.0.3-bin-hadoop3 /tmp/spark
cd /tmp/spark/sbin
@@ -225,11 +225,11 @@ jobs:
run: swift test --filter NOTHING -c release
- name: Test
run: |
- curl -LO
https://www.apache.org/dyn/closer.lua/spark/spark-4.0.2/spark-4.0.2-bin-hadoop3.tgz?action=download
- tar xvfz spark-4.0.2-bin-hadoop3.tgz && rm spark-4.0.2-bin-hadoop3.tgz
- mv spark-4.0.2-bin-hadoop3 /tmp/spark
+ curl -LO
https://www.apache.org/dyn/closer.lua/spark/spark-4.0.3/spark-4.0.3-bin-hadoop3.tgz?action=download
+ tar xvfz spark-4.0.3-bin-hadoop3.tgz && rm spark-4.0.3-bin-hadoop3.tgz
+ mv spark-4.0.3-bin-hadoop3 /tmp/spark
cd /tmp/spark/sbin
- ./start-connect-server.sh --packages
org.apache.spark:spark-connect_2.13:4.0.2,org.apache.iceberg:iceberg-spark-runtime-4.0_2.13:1.11.0
-c spark.sql.catalog.local=org.apache.iceberg.spark.SparkCatalog -c
spark.sql.catalog.local.type=hadoop -c
spark.sql.catalog.local.warehouse=/tmp/spark/warehouse -c
spark.sql.defaultCatalog=local
+ ./start-connect-server.sh --packages
org.apache.spark:spark-connect_2.13:4.0.3,org.apache.iceberg:iceberg-spark-runtime-4.0_2.13:1.11.0
-c spark.sql.catalog.local=org.apache.iceberg.spark.SparkCatalog -c
spark.sql.catalog.local.type=hadoop -c
spark.sql.catalog.local.warehouse=/tmp/spark/warehouse -c
spark.sql.defaultCatalog=local
# Iceberg jars are resolved at startup, so wait until the server is
ready.
for i in $(seq 1 12); do
grep -q "Spark Connect server started at" /tmp/spark/logs/*.out
2>/dev/null && break
diff --git a/AGENTS.md b/AGENTS.md
index b3924fb..4ed5f4b 100644
--- a/AGENTS.md
+++ b/AGENTS.md
@@ -21,7 +21,7 @@ catalog, ML) over gRPC, exchanging results in Apache Arrow
format.
- **Swift 6.3.2+** with Swift Package Manager (`swift-tools-version: 6.3.2`).
- **Platforms**: macOS 15+, iOS 18+, watchOS 11+, tvOS 18+, and Linux
(built and tested on Ubuntu x86_64 & arm64 with the Swift 6.3.2 toolchain).
-- **Server**: Apache Spark 4.x Connect server (tested against 4.0.2 / 4.1.2 /
4.2.0-preview).
+- **Server**: Apache Spark 4.x Connect server (tested against 4.0.3 / 4.1.2 /
4.2.0-preview).
- **Dependencies** (all pinned with `exact` in [Package.swift](Package.swift)):
`grpc-swift-2`, `grpc-swift-protobuf`, `grpc-swift-nio-transport`,
`flatbuffers`, `swift-system`. Keep version bumps as `exact` pins, never
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
