This is an automated email from the ASF dual-hosted git repository.
dongjoon-hyun pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/spark-connect-swift.git
The following commit(s) were added to refs/heads/main by this push:
new 53e27ce [SPARK-57357] Remove `web` example
53e27ce is described below
commit 53e27ceef0dfff660303c22ec0950088bf23c666
Author: Dongjoon Hyun <[email protected]>
AuthorDate: Tue Jun 9 17:38:46 2026 -0700
[SPARK-57357] Remove `web` example
### What changes were proposed in this pull request?
This PR aims to remove `web` example.
### Why are the changes needed?
To simplify the repository.
### Does this PR introduce _any_ user-facing change?
No behavior change because this is a removal of example.
### How was this patch tested?
Manual review.
### Was this patch authored or co-authored using generative AI tooling?
Generated-by: Claude Code (Claude Opus 4.8)
Closes #418 from dongjoon-hyun/SPARK-57357.
Authored-by: Dongjoon Hyun <[email protected]>
Signed-off-by: Dongjoon Hyun <[email protected]>
---
.github/workflows/publish_image.yml | 4 +-
AGENTS.md | 2 +-
Examples/web/Dockerfile | 41 -------
Examples/web/Package.swift | 52 ---------
Examples/web/README.md | 126 ---------------------
.../Sources/SparkConnectSwiftWeb/configure.swift | 27 -----
.../Sources/SparkConnectSwiftWeb/entrypoint.swift | 50 --------
.../web/Sources/SparkConnectSwiftWeb/routes.swift | 40 -------
README.md | 2 +-
.../SparkConnect/Documentation.docc/Examples.md | 36 ------
10 files changed, 4 insertions(+), 376 deletions(-)
diff --git a/.github/workflows/publish_image.yml
b/.github/workflows/publish_image.yml
index a53f953..dfdafc6 100644
--- a/.github/workflows/publish_image.yml
+++ b/.github/workflows/publish_image.yml
@@ -7,7 +7,7 @@ on:
description: 'list of examples to publish (JSON)'
required: true
# keep in sync with default value of strategy matrix 'example'
- default: '["app", "pi", "pyspark-connect", "spark-sql", "stream",
"web"]'
+ default: '["app", "pi", "pyspark-connect", "spark-sql", "stream"]'
jobs:
publish-image:
@@ -18,7 +18,7 @@ jobs:
max-parallel: 20
matrix:
# keep in sync with default value of workflow_dispatch input 'example'
- example: ${{ fromJSON( inputs.example || '["app", "pi",
"pyspark-connect", "spark-sql", "stream", "web"]' ) }}
+ example: ${{ fromJSON( inputs.example || '["app", "pi",
"pyspark-connect", "spark-sql", "stream"]' ) }}
steps:
- name: Set up QEMU
uses: docker/setup-qemu-action@ce360397dd3f832beb865e1373c09c0e9f86d70a
diff --git a/AGENTS.md b/AGENTS.md
index c15cc00..b3924fb 100644
--- a/AGENTS.md
+++ b/AGENTS.md
@@ -43,7 +43,7 @@ catalog, ML) over gRPC, exchanging results in Apache Arrow
format.
- `Documentation.docc/` — DocC docs (published to Swift Package Index).
- `Tests/SparkConnectTests/` — test suite (Swift Testing). `Resources/queries/`
holds golden SQL result files.
-- `Examples/` — runnable sample apps (`pi`, `spark-sql`, `stream`, `web`,
`app`,
+- `Examples/` — runnable sample apps (`pi`, `spark-sql`, `stream`, `app`,
`pyspark-connect`), each with its own `Package.swift` and `Dockerfile`.
- `dev/` — Python maintainer scripts (JIRA + PR merge tooling).
- `.github/workflows/build_main.yml` — CI entry point: runs on push to all
diff --git a/Examples/web/Dockerfile b/Examples/web/Dockerfile
deleted file mode 100644
index 2aab7bb..0000000
--- a/Examples/web/Dockerfile
+++ /dev/null
@@ -1,41 +0,0 @@
-#
-# Licensed to the Apache Software Foundation (ASF) under one or more
-# contributor license agreements. See the NOTICE file distributed with
-# this work for additional information regarding copyright ownership.
-# The ASF licenses this file to You under the Apache License, Version 2.0
-# (the "License"); you may not use this file except in compliance with
-# the License. You may obtain a copy of the License at
-#
-# http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing, software
-# distributed under the License is distributed on an "AS IS" BASIS,
-# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-# See the License for the specific language governing permissions and
-# limitations under the License.
-#
-FROM swift:6.3.2 AS builder
-
-WORKDIR /app
-
-ADD
https://download.swift.org/swift-6.3.2-release/static-sdk/swift-6.3.2-RELEASE/swift-6.3.2-RELEASE_static-linux-0.1.0.artifactbundle.tar.gz
/tmp/swift-static-sdk.tar.gz
-RUN swift sdk install /tmp/swift-static-sdk.tar.gz && \
- rm /tmp/swift-static-sdk.tar.gz
-
-COPY . .
-
-RUN arch=$(uname -m) && \
- swift build -c release --swift-sdk ${arch}-swift-linux-musl && \
- strip .build/*-swift-linux-musl/release/SparkConnectSwiftWeb
-
-FROM gcr.io/distroless/static-debian12:nonroot
-
-LABEL org.opencontainers.image.authors="Apache Spark project
<[email protected]>"
-LABEL org.opencontainers.image.licenses="Apache-2.0"
-LABEL org.opencontainers.image.ref.name="Apache Spark Connect for Swift"
-
-COPY --from=builder --chown=65532:65532
/app/.build/*-swift-linux-musl/release/SparkConnectSwiftWeb /opt/spark-app/
-
-EXPOSE 8080
-
-ENTRYPOINT ["/opt/spark-app/SparkConnectSwiftWeb"]
diff --git a/Examples/web/Package.swift b/Examples/web/Package.swift
deleted file mode 100644
index 26b6e72..0000000
--- a/Examples/web/Package.swift
+++ /dev/null
@@ -1,52 +0,0 @@
-// swift-tools-version: 6.3.2
-//
-// Licensed to the Apache Software Foundation (ASF) under one
-// or more contributor license agreements. See the NOTICE file
-// distributed with this work for additional information
-// regarding copyright ownership. The ASF licenses this file
-// to you under the Apache License, Version 2.0 (the
-// "License"); you may not use this file except in compliance
-// with the License. You may obtain a copy of the License at
-//
-// http://www.apache.org/licenses/LICENSE-2.0
-//
-// Unless required by applicable law or agreed to in writing,
-// software distributed under the License is distributed on an
-// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
-// KIND, either express or implied. See the License for the
-// specific language governing permissions and limitations
-// under the License.
-//
-import PackageDescription
-
-let package = Package(
- name: "SparkConnectSwiftWeb",
- platforms: [
- .macOS(.v15)
- ],
- dependencies: [
- // 💧 A server-side Swift web framework.
- .package(url: "https://github.com/vapor/vapor.git";, from: "4.121.3"),
- // 🔵 Non-blocking, event-driven networking for Swift. Used for custom
executors
- .package(url: "https://github.com/apple/swift-nio.git";, from: "2.65.0"),
- .package(url: "https://github.com/apache/spark-connect-swift.git";, branch:
"main"),
- ],
- targets: [
- .executableTarget(
- name: "SparkConnectSwiftWeb",
- dependencies: [
- .product(name: "Vapor", package: "vapor"),
- .product(name: "NIOCore", package: "swift-nio"),
- .product(name: "NIOPosix", package: "swift-nio"),
- .product(name: "SparkConnect", package: "spark-connect-swift"),
- ],
- swiftSettings: swiftSettings
- )
- ]
-)
-
-var swiftSettings: [SwiftSetting] {
- [
- .enableUpcomingFeature("ExistentialAny")
- ]
-}
diff --git a/Examples/web/README.md b/Examples/web/README.md
deleted file mode 100644
index d7a3f6d..0000000
--- a/Examples/web/README.md
+++ /dev/null
@@ -1,126 +0,0 @@
-# A Swift Application with Apache Spark Connect Swift Client
-
-This project is designed to illustrate a Swift-based HTTP WebServer with
Apache Spark Connect.
-
-- <https://swiftpackageindex.com/vapor/vapor>
-
-## Create a Swift project
-
-```bash
-brew install vapor
-vapor new spark-connect-swift-web -n
-```
-
-## Use `Apache Spark Connect Swift Client` package
-
-```bash
-$ git diff HEAD
-diff --git a/Package.swift b/Package.swift
-index 2edcc8f..dd918a9 100644
---- a/Package.swift
-+++ b/Package.swift
-@@ -4,13 +4,14 @@ import PackageDescription
- let package = Package(
- name: "SparkConnectSwiftWebapp",
- platforms: [
-- .macOS(.v13)
-+ .macOS(.v15)
- ],
- dependencies: [
- // 💧 A server-side Swift web framework.
- .package(url: "https://github.com/vapor/vapor.git";, from: "4.115.0"),
- // 🔵 Non-blocking, event-driven networking for Swift. Used for custom
executors
- .package(url: "https://github.com/apple/swift-nio.git";, from:
"2.65.0"),
-+ .package(url: "https://github.com/apache/spark-connect-swift.git";,
branch: "main"),
- ],
- targets: [
- .executableTarget(
-@@ -19,6 +20,7 @@ let package = Package(
- .product(name: "Vapor", package: "vapor"),
- .product(name: "NIOCore", package: "swift-nio"),
- .product(name: "NIOPosix", package: "swift-nio"),
-+ .product(name: "SparkConnect", package:
"spark-connect-swift"),
- ],
- swiftSettings: swiftSettings
- ),
-diff --git a/Sources/SparkConnectSwiftWeb/configure.swift
b/Sources/SparkConnectSwiftWeb/configure.swift
-index 7715d7c..eea2f95 100644
---- a/Sources/SparkConnectSwiftWeb/configure.swift
-+++ b/Sources/SparkConnectSwiftWeb/configure.swift
-@@ -2,6 +2,7 @@ import Vapor
-
- // configures your application
- public func configure(_ app: Application) async throws {
-+ app.http.server.configuration.hostname = "0.0.0.0"
- // uncomment to serve files from /Public folder
- // app.middleware.use(FileMiddleware(publicDirectory:
app.directory.publicDirectory))
-
-diff --git a/Sources/SparkConnectSwiftWeb/routes.swift
b/Sources/SparkConnectSwiftWeb/routes.swift
-index 2edcc8f..dd918a9 100644
---- a/Sources/SparkConnectSwiftWeb/routes.swift
-+++ b/Sources/SparkConnectSwiftWeb/routes.swift
-@@ -1,11 +1,21 @@
- import Vapor
-+import SparkConnect
-
- func routes(_ app: Application) throws {
- app.get { req async in
-- "It works!"
-+ "Welcome to the Swift world. Say hello!"
- }
-
- app.get("hello") { req async -> String in
-- "Hello, world!"
-+ return await Task {
-+ do {
-+ let spark = try await SparkSession.builder.getOrCreate()
-+ let response = "Hi, this is powered by the Apache Spark
\(await spark.version)."
-+ await spark.stop()
-+ return response
-+ } catch {
-+ return "Fail to connect: \(error)"
-+ }
-+ }.value
- }
- }
-```
-
-## How to run
-
-Prepare `Spark Connect Server` via running Docker image.
-
-```bash
-docker run --rm -p 15002:15002 apache/spark:4.1.2 bash -c
"/opt/spark/sbin/start-connect-server.sh --wait"
-```
-
-Build an application Docker image.
-
-```bash
-$ docker build -t apache/spark-connect-swift:web .
-$ docker images apache/spark-connect-swift:web
-IMAGE ID DISK USAGE CONTENT SIZE
EXTRA
-apache/spark-connect-swift:web 240d2adab124 130MB 37.1MB
-```
-
-Run `web` docker image
-
-```bash
-$ docker run -it --rm -p 8080:8080 -e
SPARK_REMOTE=sc://host.docker.internal:15002 apache/spark-connect-swift:web
-[ NOTICE ] Server started on http://127.0.0.1:8080
-```
-
-Connect to the Swift Web Server to talk with `Apache Spark`.
-
-```bash
-$ curl http://127.0.0.1:8080/
-Welcome to the Swift world. Say hello!%
-
-$ curl http://127.0.0.1:8080/hello
-Hi, this is powered by the Apache Spark 4.1.2.%
-```
-
-Run from source code.
-
-```bash
-swift run
-```
diff --git a/Examples/web/Sources/SparkConnectSwiftWeb/configure.swift
b/Examples/web/Sources/SparkConnectSwiftWeb/configure.swift
deleted file mode 100644
index f70192a..0000000
--- a/Examples/web/Sources/SparkConnectSwiftWeb/configure.swift
+++ /dev/null
@@ -1,27 +0,0 @@
-//
-// Licensed to the Apache Software Foundation (ASF) under one
-// or more contributor license agreements. See the NOTICE file
-// distributed with this work for additional information
-// regarding copyright ownership. The ASF licenses this file
-// to you under the Apache License, Version 2.0 (the
-// "License"); you may not use this file except in compliance
-// with the License. You may obtain a copy of the License at
-//
-// http://www.apache.org/licenses/LICENSE-2.0
-//
-// Unless required by applicable law or agreed to in writing,
-// software distributed under the License is distributed on an
-// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
-// KIND, either express or implied. See the License for the
-// specific language governing permissions and limitations
-// under the License.
-//
-
-import Vapor
-
-// configures your application
-public func configure(_ app: Application) async throws {
- app.http.server.configuration.hostname = "0.0.0.0"
- // register routes
- try routes(app)
-}
diff --git a/Examples/web/Sources/SparkConnectSwiftWeb/entrypoint.swift
b/Examples/web/Sources/SparkConnectSwiftWeb/entrypoint.swift
deleted file mode 100644
index 0be116f..0000000
--- a/Examples/web/Sources/SparkConnectSwiftWeb/entrypoint.swift
+++ /dev/null
@@ -1,50 +0,0 @@
-//
-// Licensed to the Apache Software Foundation (ASF) under one
-// or more contributor license agreements. See the NOTICE file
-// distributed with this work for additional information
-// regarding copyright ownership. The ASF licenses this file
-// to you under the Apache License, Version 2.0 (the
-// "License"); you may not use this file except in compliance
-// with the License. You may obtain a copy of the License at
-//
-// http://www.apache.org/licenses/LICENSE-2.0
-//
-// Unless required by applicable law or agreed to in writing,
-// software distributed under the License is distributed on an
-// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
-// KIND, either express or implied. See the License for the
-// specific language governing permissions and limitations
-// under the License.
-//
-
-import Logging
-import NIOCore
-import NIOPosix
-import Vapor
-
-@main
-enum Entrypoint {
- static func main() async throws {
- var env = try Environment.detect()
- try LoggingSystem.bootstrap(from: &env)
-
- let app = try await Application.make(env)
-
- // This attempts to install NIO as the Swift Concurrency global executor.
- // You can enable it if you'd like to reduce the amount of context
switching between NIO and Swift Concurrency.
- // Note: this has caused issues with some libraries that use `.wait()` and
cleanly shutting down.
- // If enabled, you should be careful about calling async functions before
this point as it can cause assertion failures.
- // let executorTakeoverSuccess =
NIOSingletons.unsafeTryInstallSingletonPosixEventLoopGroupAsConcurrencyGlobalExecutor()
- // app.logger.debug("Tried to install SwiftNIO's EventLoopGroup as Swift's
global concurrency executor", metadata: ["success":
.stringConvertible(executorTakeoverSuccess)])
-
- do {
- try await configure(app)
- try await app.execute()
- } catch {
- app.logger.report(error: error)
- try? await app.asyncShutdown()
- throw error
- }
- try await app.asyncShutdown()
- }
-}
diff --git a/Examples/web/Sources/SparkConnectSwiftWeb/routes.swift
b/Examples/web/Sources/SparkConnectSwiftWeb/routes.swift
deleted file mode 100644
index c6a1d91..0000000
--- a/Examples/web/Sources/SparkConnectSwiftWeb/routes.swift
+++ /dev/null
@@ -1,40 +0,0 @@
-//
-// Licensed to the Apache Software Foundation (ASF) under one
-// or more contributor license agreements. See the NOTICE file
-// distributed with this work for additional information
-// regarding copyright ownership. The ASF licenses this file
-// to you under the Apache License, Version 2.0 (the
-// "License"); you may not use this file except in compliance
-// with the License. You may obtain a copy of the License at
-//
-// http://www.apache.org/licenses/LICENSE-2.0
-//
-// Unless required by applicable law or agreed to in writing,
-// software distributed under the License is distributed on an
-// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
-// KIND, either express or implied. See the License for the
-// specific language governing permissions and limitations
-// under the License.
-//
-
-import SparkConnect
-import Vapor
-
-func routes(_ app: Application) throws {
- app.get { req async in
- "Welcome to the Swift world. Say hello!"
- }
-
- app.get("hello") { req async -> String in
- return await Task {
- do {
- let spark = try await SparkSession.builder.getOrCreate()
- let response = "Hi, this is powered by the Apache Spark \(await
spark.version)."
- await spark.stop()
- return response
- } catch {
- return "Fail to connect: \(error)"
- }
- }.value
- }
-}
diff --git a/README.md b/README.md
index 9b45b5e..d8fed51 100644
--- a/README.md
+++ b/README.md
@@ -124,6 +124,6 @@ SELECT * FROM t
+---+
```
-You can find more complete examples including `Spark SQL REPL`, `Web Server`
and `Streaming` applications in the
[Examples](https://github.com/apache/spark-connect-swift/tree/main/Examples)
directory.
+You can find more complete examples including `Spark SQL REPL` and `Streaming`
applications in the
[Examples](https://github.com/apache/spark-connect-swift/tree/main/Examples)
directory.
This library also supports `SPARK_REMOTE` environment variable to specify the
[Spark Connect connection
string](https://spark.apache.org/docs/latest/spark-connect-overview.html#set-sparkremote-environment-variable)
in order to provide more options.
diff --git a/Sources/SparkConnect/Documentation.docc/Examples.md
b/Sources/SparkConnect/Documentation.docc/Examples.md
index 7260e0b..a814d5e 100644
--- a/Sources/SparkConnect/Documentation.docc/Examples.md
+++ b/Sources/SparkConnect/Documentation.docc/Examples.md
@@ -121,42 +121,6 @@ swift run
Type text into the Netcat terminal to see real-time word counting in the
`Spark Connect Server` container output.
-## Web Application Example
-
-The web application example showcases how to integrate Spark Connect Swift
with a web server using the Vapor framework.
-
-### Key Features
-
-- HTTP server integration with Vapor
-- REST API endpoints
-- Spark session management within web requests
-- Version information retrieval
-
-### How to Run
-
-Build and run the application:
-
-```bash
-# Using Docker
-docker build -t apache/spark-connect-swift:web .
-docker run -it --rm -p 8080:8080 -e
SPARK_REMOTE=sc://host.docker.internal:15002 apache/spark-connect-swift:web
-
-# From source code
-swift run
-```
-
-Access the web application:
-
-```bash
-# Root endpoint
-$ curl http://127.0.0.1:8080/
-Welcome to the Swift world. Say hello!%
-
-# Spark-powered endpoint
-curl http://127.0.0.1:8080/hello
-Hi, this is powered by the Apache Spark 4.1.2.%
-```
-
## Development Environment
All examples include:
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
