This is an automated email from the ASF dual-hosted git repository.
agrove pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/datafusion-ballista.git
The following commit(s) were added to refs/heads/main by this push:
new 36af974a doc: update docker related documentation (#1204)
36af974a is described below
commit 36af974abe215a48de10c5c3164cff98cd792464
Author: Marko Milenković <[email protected]>
AuthorDate: Mon Mar 17 21:03:40 2025 +0000
doc: update docker related documentation (#1204)
---
docker-compose.yml | 9 +----
.../source/user-guide/deployment/docker-compose.md | 16 ++++----
docs/source/user-guide/deployment/docker.md | 44 +++++++---------------
3 files changed, 22 insertions(+), 47 deletions(-)
diff --git a/docker-compose.yml b/docker-compose.yml
index 57556e69..c21230ec 100644
--- a/docker-compose.yml
+++ b/docker-compose.yml
@@ -16,25 +16,18 @@
# under the License.
version: '3.3'
services:
- etcd:
- image: quay.io/coreos/etcd:v3.4.9
- command: "etcd -advertise-client-urls http://etcd:2379 -listen-client-urls
http://0.0.0.0:2379";
- ports:
- - "2379:2379"
ballista-scheduler:
image: ballista-scheduler
build:
dockerfile: dev/docker/ballista-scheduler.Dockerfile
context: .
- command: "--cluster-backend etcd --etcd-urls etcd:2379 --bind-host 0.0.0.0"
+ command: "--bind-host 0.0.0.0"
ports:
- "50050:50050"
environment:
- RUST_LOG=ballista=info
volumes:
- ./benchmarks/data:/data
- depends_on:
- - etcd
healthcheck:
test: ["CMD", "nc", "-z", "ballista-scheduler", "50050"]
interval: 5s
diff --git a/docs/source/user-guide/deployment/docker-compose.md
b/docs/source/user-guide/deployment/docker-compose.md
index 7782b03f..3c7eb4f7 100644
--- a/docs/source/user-guide/deployment/docker-compose.md
+++ b/docs/source/user-guide/deployment/docker-compose.md
@@ -26,24 +26,24 @@ Docker Compose is a convenient way to launch a cluster when
testing locally.
Run the following commands to download the [official Docker
image](https://github.com/apache/datafusion-ballista/pkgs/container/datafusion-ballista-standalone):
```bash
-docker pull ghcr.io/apache/datafusion-ballista-standalone:0.12.0-rc4
+docker pull ghcr.io/apache/datafusion-ballista-standalone:latest
```
Altenatively run the following commands to clone the source repository and
build the Docker images from source:
```bash
-git clone [email protected]:apache/datafusion-ballista.git -b 0.12.0
+git clone [email protected]:apache/datafusion-ballista.git -b latest
cd datafusion-ballista
./dev/build-ballista-docker.sh
```
This will create the following images:
-- `apache/datafusion-ballista-benchmarks:0.12.0`
-- `apache/datafusion-ballista-cli:0.12.0`
-- `apache/datafusion-ballista-executor:0.12.0`
-- `apache/datafusion-ballista-scheduler:0.12.0`
-- `apache/datafusion-ballista-standalone:0.12.0`
+- `apache/datafusion-ballista-benchmarks:latest`
+- `apache/datafusion-ballista-cli:latest`
+- `apache/datafusion-ballista-executor:latest`
+- `apache/datafusion-ballista-scheduler:latest`
+- `apache/datafusion-ballista-standalone:latest`
## Start a Cluster
@@ -75,5 +75,5 @@ The scheduler listens on port 50050 and this is the port that
clients will need
## Connect from the Ballista CLI
```shell
-docker run --network=host -it apache/datafusion-ballista-cli:0.12.0 --host
localhost --port 50050
+docker run --network=host -it apache/datafusion-ballista-cli:latest --host
localhost --port 50050
```
diff --git a/docs/source/user-guide/deployment/docker.md
b/docs/source/user-guide/deployment/docker.md
index 3e15fd3f..971d1780 100644
--- a/docs/source/user-guide/deployment/docker.md
+++ b/docs/source/user-guide/deployment/docker.md
@@ -24,24 +24,24 @@
Run the following commands to download the [official Docker
image](https://github.com/apache/datafusion-ballista/pkgs/container/datafusion-ballista-standalone):
```bash
-docker pull ghcr.io/apache/datafusion-ballista-standalone:0.12.0-rc4
+docker pull ghcr.io/apache/datafusion-ballista-standalone:latest
```
Altenatively run the following commands to clone the source repository and
build the Docker images from source:
```bash
-git clone [email protected]:apache/datafusion-ballista.git -b 0.12.0
+git clone [email protected]:apache/datafusion-ballista.git
cd datafusion-ballista
./dev/build-ballista-docker.sh
```
This will create the following images:
-- `apache/datafusion-ballista-benchmarks:0.12.0`
-- `apache/datafusion-ballista-cli:0.12.0`
-- `apache/datafusion-ballista-executor:0.12.0`
-- `apache/datafusion-ballista-scheduler:0.12.0`
-- `apache/datafusion-ballista-standalone:0.12.0`
+- `apache/datafusion-ballista-benchmarks:latest`
+- `apache/datafusion-ballista-cli:latest`
+- `apache/datafusion-ballista-executor:latest`
+- `apache/datafusion-ballista-scheduler:latest`
+- `apache/datafusion-ballista-standalone:latest`
## Start a Cluster
@@ -51,7 +51,7 @@ Start a scheduler using the following syntax:
```bash
docker run --network=host \
- -d apache/datafusion-ballista-scheduler:0.12.0 \
+ -d apache/datafusion-ballista-scheduler:latest \
--bind-port 50050
```
@@ -60,7 +60,7 @@ Run `docker ps` to check that the process is running:
```
$ docker ps
CONTAINER ID IMAGE COMMAND
CREATED STATUS PORTS NAMES
-a756055576f3 apache/datafusion-ballista-scheduler:0.12.0
"/root/scheduler-ent…" 8 seconds ago Up 8 seconds
xenodochial_carson
+a756055576f3 apache/datafusion-ballista-scheduler:latest
"/root/scheduler-ent…" 8 seconds ago Up 8 seconds
xenodochial_carson
```
Run `docker logs CONTAINER_ID` to check the output from the process:
@@ -82,7 +82,7 @@ Start one or more executor processes. Each executor process
will need to listen
```bash
docker run --network=host \
- -d apache/datafusion-ballista-executor:0.12.0 \
+ -d apache/datafusion-ballista-executor:latest \
--external-host localhost --bind-port 50051
```
@@ -91,8 +91,8 @@ Use `docker ps` to check that both the scheduler and
executor(s) are now running
```
$ docker ps
CONTAINER ID IMAGE COMMAND
CREATED STATUS PORTS NAMES
-fb8b530cee6d apache/datafusion-ballista-executor:0.12.0
"/root/executor-entr…" 2 seconds ago Up 1 second gallant_galois
-a756055576f3 apache/datafusion-ballista-scheduler:0.12.0
"/root/scheduler-ent…" 8 seconds ago Up 8 seconds
xenodochial_carson
+fb8b530cee6d apache/datafusion-ballista-executor:latest
"/root/executor-entr…" 2 seconds ago Up 1 second gallant_galois
+a756055576f3 apache/datafusion-ballista-scheduler:latest
"/root/scheduler-ent…" 8 seconds ago Up 8 seconds
xenodochial_carson
```
Use `docker logs CONTAINER_ID` to check the output from the executor(s):
@@ -106,26 +106,8 @@ $ docker logs fb8b530cee6d
2024-02-03T14:50:24.063281Z INFO tokio-runtime-worker ThreadId(47)
ballista_executor::execution_loop: Starting poll work loop with scheduler
```
-### Using etcd as a Backing Store
-
-_NOTE: This functionality is currently experimental_
-
-Ballista can optionally use [etcd](https://etcd.io/) as a backing store for
the scheduler. Use the following commands
-to launch the scheduler with this option enabled.
-
-```bash
-docker run --network=host \
- -d apache/datafusion-ballista-scheduler:0.12.0 \
- --bind-port 50050 \
- --config-backend etcd \
- --etcd-urls etcd:2379
-```
-
-Please refer to the [etcd](https://etcd.io/) website for installation
instructions. Etcd version 3.4.9 or later is
-recommended.
-
## Connect from the CLI
```shell
-docker run --network=host -it apache/datafusion-ballista-cli:0.12.0 --host
localhost --port 50050
+docker run --network=host -it apache/datafusion-ballista-cli:latest --host
localhost --port 50050
```
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
