This is an automated email from the ASF dual-hosted git repository.
morningman pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/doris.git
The following commit(s) were added to refs/heads/master by this push:
new b10f3e08114 [feat](docker) run the 4.1 all-in-one image as multi-node
and cloud compose clusters (#67875)
b10f3e08114 is described below
commit b10f3e08114803d1d724553320e2f356bb620910
Author: Mingyu Chen (Rayner) <[email protected]>
AuthorDate: Fri Sep 11 22:01:30 2026 +0800
[feat](docker) run the 4.1 all-in-one image as multi-node and cloud compose
clusters (#67875)
## Proposed changes
#66983 added a single-container Doris for downstream CI. The next things
one
wants from the same box are a cluster with several FEs and BEs, and a
compute-storage separated (cloud) cluster — for local feature work and
demos,
not for the regression pipelines. This PR makes the **same image** serve
both,
via two compose files under `docker/runtime/all-in-one/4.1/compose/`. No
second image, no Python, no cloud credentials.
```shell
cd docker/runtime/all-in-one/4.1/compose
docker compose -f multi-node.yml up --wait # 3 FE + 3 BE, ~35 s
to healthy
docker compose -f cloud.yml up --wait # fdb + ms + recycler
+ minio + FE + 3 BE, ~50 s
docker compose -f cloud.yml --profile ha up --wait # ... with 3 FEs
docker compose -f cloud.yml exec client mysql -uroot -hfe-1 -P9030
```
### One image, three ways to run it
The entrypoint now dispatches on `DORIS_ROLE`: `all` (the single
container,
unchanged), `fe`, `be`, `ms`, `recycler`, `cloud-init` (one-shot,
creates the
instance) and `client` (waits for the expected nodes, then idles with
`mysql`
and `curl` inside the network). Each role is plain Doris under
`--console`
with the same start → probe → register → ready-flag → fail-fast shape as
before, so `docker compose up --wait` blocks until the whole cluster is
formed
and a dead process still turns into a non-zero exit.
Topology-dependent settings — `priority_networks`, replica count,
balancing,
the cloud keys — moved from the conf files baked into the image to the
entrypoint, written per role at container start. The baked conf keeps
only
the size-related defaults.
### Cloud mode with nothing external
`cloud.yml` is FoundationDB + meta-service + recycler + MinIO + one to
three
FEs + three BEs in two compute groups (`cg_a` × 2, `cg_b` × 1), so
`use @cg_b`, `SHOW COMPUTE GROUPS`, storage vaults and FE failover in
cloud
mode all have something to run against.
- Nodes are managed by SQL: the FE gets `cluster_id = <instance id>` and
derives its `cloud_unique_id` itself; BEs are added with
`ALTER SYSTEM ADD BACKEND ... ("tag.compute_group_name" = ...)`.
- `cloud-init` creates the instance in storage-vault mode with MinIO as
the
vault — `use_path_style` in the `create_instance` payload is enough, no
virtual-host aliasing — and the first FE marks `built_in_storage_vault`
as
default, so `CREATE TABLE` works as is. Both steps are idempotent.
- The meta-service payload joins both tags. It is 664 MB as shipped, 456
MB
of which is an unstripped `libfdb_c.so`; `strip --strip-debug` brings
the
whole thing to 183 MB, so the tags grow from 2.46 / 2.99 GB to 2.68 /
3.21 GB.
### Also in here
- `FE_MASTER` may list several FEs (`fe-1,fe-2,fe-3`): the first one
that
answers is used, so a BE or follower restart does not wait on the one FE
that is down. An FE that already has metadata rejoins on its own.
- A local `./output` or an extracted release tarball is passed to the
build
as BuildKit named contexts. The previous `--tarball-dir` path could not
work: `Dockerfile.dockerignore` only let `output/fe` and `output/be`
into
the context. This also means a tarball can build the image before the
component images of a release reach Docker Hub.
- Node probes use `SHOW FRONTENDS` / `SHOW BACKENDS` with header-based
column
lookup. The `frontends()` / `backends()` table functions look handier
but
are queries, and a query needs a live BE — which is what the FE probe
runs
before.
### Things a user should know (all in the README)
- Nodes get fixed IPs on a private subnet so a restart keeps their
identity.
From the host, use the published ports; on Docker Desktop the container
addresses are not routable, so a stream load from the host cannot follow
FE's redirect to a BE — run it from the `client` service. The
meta-service
HTTP API is published on `15000`, since `5000` is AirPlay on macOS.
- `foundationdb/foundationdb:7.1.x` upstream images are amd64 only; on
Apple
Silicon `fdb` runs under emulation, which is fine for this purpose.
- A cloud cluster is ~9 GB; give Docker Desktop 12 GB or more.
## Testing
All on 4.1.3 official component images, arm64 (Apple Silicon):
- `resource/smoke-test.sh apache/doris:all-in-one-4.1.3 base` — the
existing
single-container test, unchanged and green.
- `compose/smoke-test.sh multi-node` (1 min 40 s) and
`compose/smoke-test.sh
cloud` (2 min 7 s): bring the topology up under its own project name,
subnet and host ports; assert 3 FE / 3 BE alive; create / insert / read;
three replicas over three BEs, or two compute groups + default vault +
objects in MinIO; stream load through the client; kill the master FE and
write through the new one (elected in 3–9 s); restart a BE while the old
master is down; bring the old master back; tear down.
- `docker compose stop` / `up` on the cloud cluster keeps the data
(instance
creation, vault default and node registration are all idempotent).
- `build.sh -s tarball --tarball-dir <dir>` from outside the repository,
with fe/be/ms extracted from the component images.
**Not verified here:** amd64 (same caveat as #66983 — the BE does not
survive
emulation on an Apple Silicon host).
## Notes
- Publishing: the compose files default to
`apache/doris:all-in-one-4.1.3`,
which on Docker Hub still lacks the meta-service payload and the
role-aware
entrypoint. Once this is merged, the 4.1.3 tags need a rebuild and
re-push
(single-container behaviour is unchanged; the tags gain ~0.2 GB), or the
default image moves to the next release's tag.
- This is a demo / feature-work environment. The `docker()` suites under
`regression-test/suites/cloud_p0` still need `doris-compose`.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
https://claude.ai/code/session_01CvUCjtaEi5tgy89ABkhgMh
Co-authored-by: Claude Opus 5 (1M context) <[email protected]>
---
docker/runtime/all-in-one/4.1/Dockerfile | 70 ++--
.../runtime/all-in-one/4.1/Dockerfile.dockerignore | 9 +-
docker/runtime/all-in-one/4.1/README.md | 138 ++++++-
docker/runtime/all-in-one/4.1/build.sh | 81 +++--
docker/runtime/all-in-one/4.1/compose/cloud.yml | 320 +++++++++++++++++
.../runtime/all-in-one/4.1/compose/multi-node.yml | 168 +++++++++
.../runtime/all-in-one/4.1/compose/smoke-test.sh | 154 ++++++++
.../runtime/all-in-one/4.1/resource/cloud_init.sh | 64 ++++
.../all-in-one/4.1/resource/conf/be_ci.conf | 1 -
.../all-in-one/4.1/resource/conf/fe_ci.conf | 16 +-
.../runtime/all-in-one/4.1/resource/entrypoint.sh | 396 ++++++++++++++-------
.../all-in-one/4.1/resource/health_check.sh | 37 +-
docker/runtime/all-in-one/4.1/resource/lib.sh | 148 ++++++++
13 files changed, 1379 insertions(+), 223 deletions(-)
diff --git a/docker/runtime/all-in-one/4.1/Dockerfile
b/docker/runtime/all-in-one/4.1/Dockerfile
index 31c3c829bf8..173d24e2e21 100644
--- a/docker/runtime/all-in-one/4.1/Dockerfile
+++ b/docker/runtime/all-in-one/4.1/Dockerfile
@@ -18,9 +18,10 @@
# ============================================================================
# Apache Doris all-in-one image for integration testing -- branch-4.1 only.
#
-# A single container running one FE and one BE, sized for use as a test
-# fixture in downstream CI. Build it with ./build.sh; the commands below are
-# what that script ends up running.
+# One image, three ways to run it: a single container with FE and BE (the
+# default entrypoint role), the multi-node compose file, and the cloud
+# (compute-storage separated) compose file. See README.md. Build it with
+# ./build.sh; the commands below are what that script ends up running.
#
# base flavor (default):
# docker buildx build --build-arg DORIS_VERSION=4.1.3 \
@@ -30,8 +31,10 @@
# full flavor (adds hudi / trino / maxcompute):
# ... --build-arg FLAVOR=full -t apache/doris:all-in-one-4.1.3-full .
#
-# The build context is the repository root (so ARTIFACT_SOURCE=local can pick
-# up ./output); Dockerfile.dockerignore narrows it down to a few KB.
+# The build context is the repository root; Dockerfile.dockerignore narrows it
+# down to the resource directory. A local ./output or an extracted release
+# tarball comes in through named contexts (ARTIFACT_SOURCE=dir), which
+# build.sh sets up.
# ============================================================================
# ---- global args: must be declared before the first FROM ----
@@ -39,14 +42,16 @@ ARG DORIS_VERSION=4.1.3
ARG ARTIFACT_SOURCE=image
ARG FE_IMAGE=apache/doris:fe-${DORIS_VERSION}
ARG BE_IMAGE=apache/doris:be-${DORIS_VERSION}
+ARG MS_IMAGE=apache/doris:ms-${DORIS_VERSION}
ARG BASE_IMAGE=ubuntu:22.04
# ============================ artifact sources ==============================
-# Three interchangeable ways to get fe/ and be/ into /artifacts. Everything
+# Two interchangeable ways to get fe/, be/ and ms/ into /artifacts. Everything
# downstream only knows about /artifacts, never about where it came from.
FROM ${FE_IMAGE} AS fe-src
FROM ${BE_IMAGE} AS be-src
+FROM ${MS_IMAGE} AS ms-src
# strip(1) lives here, so this is ubuntu rather than busybox.
FROM ${BASE_IMAGE} AS artifacts-base
@@ -59,18 +64,16 @@ RUN apt-get update -y && \
FROM artifacts-base AS artifacts-image
COPY --from=fe-src /opt/apache-doris/fe /artifacts/fe
COPY --from=be-src /opt/apache-doris/be /artifacts/be
+COPY --from=ms-src /opt/apache-doris/ms /artifacts/ms
-# B. a locally built ./output, for developers testing their own build
-FROM artifacts-base AS artifacts-local
-ARG LOCAL_OUTPUT=output
-COPY ${LOCAL_OUTPUT}/fe /artifacts/fe
-COPY ${LOCAL_OUTPUT}/be /artifacts/be
-
-# C. an extracted release tarball
-FROM artifacts-base AS artifacts-tarball
-ARG TARBALL_DIR
-COPY ${TARBALL_DIR}/fe /artifacts/fe
-COPY ${TARBALL_DIR}/be /artifacts/be
+# B. directories: a locally built ./output or an extracted release tarball.
+# Each part is a named build context (--build-context doris-fe=<dir>/fe ...),
+# which keeps them out of the main context and its dockerignore. A build
+# without a meta-service points doris-ms at an empty directory.
+FROM artifacts-base AS artifacts-dir
+COPY --from=doris-fe / /artifacts/fe
+COPY --from=doris-be / /artifacts/be
+COPY --from=doris-ms / /artifacts/ms
# ========================= strip + flavor pruning ===========================
# Both have to happen HERE, before the final COPY. Doing them in the runtime
@@ -79,28 +82,33 @@ COPY ${TARBALL_DIR}/be /artifacts/be
FROM artifacts-${ARTIFACT_SOURCE} AS artifacts
ARG CTX_PREFIX=docker/runtime/all-in-one/4.1
ARG FLAVOR=base
-ARG STRIP_BE=debug
+ARG STRIP_MODE=debug
COPY ${CTX_PREFIX}/resource/prune.txt /tmp/prune.txt
RUN set -eux; \
case "${FLAVOR}" in base|full) ;; *) echo "bad FLAVOR=${FLAVOR}" >&2; exit
1 ;; esac; \
- # 4.1.3 ships doris_be with debug info: 2213M -> 430M, and --strip-debug
+ # 4.1.3 ships doris_be with debug info: 2213M -> 430M, and the meta-service
+ # payload is 664M of which 456M is an unstripped libfdb_c.so. --strip-debug
# keeps .symtab so crash backtraces still resolve function names.
- case "${STRIP_BE}" in \
- debug) strip --strip-debug /artifacts/be/lib/doris_be ;; \
- full) strip -s /artifacts/be/lib/doris_be ;; \
+ bins="/artifacts/be/lib/doris_be"; \
+ if [ -f /artifacts/ms/lib/doris_cloud ]; then \
+ bins="${bins} /artifacts/ms/lib/doris_cloud $(find /artifacts/ms/lib
-name 'libfdb_c.so')"; \
+ fi; \
+ case "${STRIP_MODE}" in \
+ debug) strip --strip-debug ${bins} ;; \
+ full) strip -s ${bins} ;; \
none) : ;; \
- *) echo "bad STRIP_BE=${STRIP_BE}" >&2; exit 1 ;; \
+ *) echo "bad STRIP_MODE=${STRIP_MODE}" >&2; exit 1 ;; \
esac; \
# "always" rows apply to both flavors, "base" rows only to the base flavor.
awk -v f="${FLAVOR}" '$1=="always" || $1==f {print $2}' /tmp/prune.txt \
| while IFS= read -r p; do echo "prune: ${p}"; rm -rf "/artifacts/${p}";
done; \
rm -f /tmp/prune.txt; \
- rm -rf /artifacts/fe/log /artifacts/be/log; \
- mkdir -p /artifacts/fe/log /artifacts/be/log \
+ rm -rf /artifacts/fe/log /artifacts/be/log /artifacts/ms/log; \
+ mkdir -p /artifacts/fe/log /artifacts/be/log /artifacts/ms \
/artifacts/fe/doris-meta /artifacts/be/storage; \
- echo "=== artifact size (${FLAVOR}) ==="; du -sh /artifacts/fe
/artifacts/be
+ echo "=== artifact size (${FLAVOR}) ==="; du -sh /artifacts/fe
/artifacts/be /artifacts/ms
# ================================ runtime ===================================
@@ -111,7 +119,8 @@ ARG FLAVOR=base
ARG CTX_PREFIX=docker/runtime/all-in-one/4.1
ARG JDK_PKG=openjdk-17-jre-headless
# FE ships -Xmx8192m, far more than a CI runner can spare; BE ships -Xmx2048m
-# for its JNI-side JVM. Both are rewritten in place below.
+# for its JNI-side JVM. Both are rewritten in place below, and again at
+# container start when FE_HEAP / BE_HEAP are set.
ARG FE_HEAP=2048m
ARG BE_HEAP=1024m
@@ -144,12 +153,14 @@ ENV JAVA_HOME=/usr/lib/jvm/java \
COPY --from=artifacts /artifacts/fe ${DORIS_HOME}/fe
COPY --from=artifacts /artifacts/be ${DORIS_HOME}/be
+COPY --from=artifacts /artifacts/ms ${DORIS_HOME}/ms
COPY ${CTX_PREFIX}/resource/ ${CI_HOME}/
RUN set -eux; \
# Integration-test defaults are appended, never edited in: the upstream
# values stay visible above them and the last assignment wins for both the
# shell-sourced ALL_CAPS vars and the property-style lowercase keys.
+ # Topology keys are not in these files; entrypoint.sh adds them per role.
cat "${CI_HOME}/conf/fe_ci.conf" >> "${DORIS_HOME}/fe/conf/fe.conf"; \
cat "${CI_HOME}/conf/be_ci.conf" >> "${DORIS_HOME}/be/conf/be.conf"; \
# The heap lives inside a long JAVA_OPTS_FOR_JDK_17 line that also carries
@@ -162,14 +173,15 @@ RUN set -eux; \
chmod +x "${CI_HOME}"/*.sh
LABEL org.opencontainers.image.title="Apache Doris all-in-one" \
- org.opencontainers.image.description="Single-container Doris (FE+BE) for
integration testing" \
+ org.opencontainers.image.description="Single-container Doris (FE+BE) for
integration testing; also the node image of the multi-node and cloud compose
files" \
org.opencontainers.image.version="${DORIS_VERSION}" \
org.opencontainers.image.source="https://github.com/apache/doris"; \
org.apache.doris.allinone.flavor="${FLAVOR}" \
org.apache.doris.allinone.branch="4.1"
# 8030 FE http | 9030 FE mysql | 8040 BE http (stream load) | 9050 BE heartbeat
-EXPOSE 8030 9030 8040 9050
+# 5000 meta-service (DORIS_ROLE=ms / recycler)
+EXPOSE 8030 9030 8040 9050 5000
WORKDIR ${DORIS_HOME}
diff --git a/docker/runtime/all-in-one/4.1/Dockerfile.dockerignore
b/docker/runtime/all-in-one/4.1/Dockerfile.dockerignore
index 379b506fd84..8fb78e8c783 100644
--- a/docker/runtime/all-in-one/4.1/Dockerfile.dockerignore
+++ b/docker/runtime/all-in-one/4.1/Dockerfile.dockerignore
@@ -18,10 +18,9 @@
# BuildKit reads this in preference to the repository root .dockerignore when
# building with -f docker/runtime/all-in-one/4.1/Dockerfile.
#
-# The context has to be the repository root so that ARTIFACT_SOURCE=local can
-# reach ./output, but nothing else in the tree is needed. Exclude everything,
-# then add back the two paths the build actually reads.
+# The context is the repository root, but the only thing the build reads from
+# it is the resource directory. Local builds and release tarballs arrive as
+# named build contexts (see build.sh), not through this context, so nothing
+# else needs to be let in.
*
!docker/runtime/all-in-one/4.1/resource
-!output/fe
-!output/be
diff --git a/docker/runtime/all-in-one/4.1/README.md
b/docker/runtime/all-in-one/4.1/README.md
index 49d8d1d6888..b18e0fb9192 100644
--- a/docker/runtime/all-in-one/4.1/README.md
+++ b/docker/runtime/all-in-one/4.1/README.md
@@ -20,10 +20,15 @@ under the License.
# Doris all-in-one image (branch-4.1)
One FE and one BE in a single container, sized to be a test fixture in a
-downstream project's CI. It is assembled from the official `apache/doris:fe-*`
-and `apache/doris:be-*` images, so a new Doris release needs no repackaging of
+downstream project's CI. It is assembled from the official `apache/doris:fe-*`,
+`be-*` and `ms-*` images, so a new Doris release needs no repackaging of
anything here.
+The same image also runs one Doris process per container, which is what the
+two compose files under `compose/` do: a three-FE / three-BE cluster, and a
+compute-storage separated (cloud) cluster with its meta-service, FoundationDB
+and MinIO. See [Multi-node and cloud clusters](#multi-node-and-cloud-clusters).
+
This directory targets the **4.1 release line only**. Other lines differ enough
in payload layout to deserve their own directory rather than a version switch.
@@ -31,14 +36,15 @@ in payload layout to deserve their own directory rather
than a version switch.
| tag | covers | size |
|---|---|---|
-| `apache/doris:all-in-one-<version>` | internal tables, Hive, Iceberg
(including system tables), Paimon, JDBC catalogs, external-table writeback,
Java UDF | 2.46 GB |
-| `apache/doris:all-in-one-<version>-full` | the above plus Hudi, Trino
connector, MaxCompute | 2.99 GB |
+| `apache/doris:all-in-one-<version>` | internal tables, Hive, Iceberg
(including system tables), Paimon, JDBC catalogs, external-table writeback,
Java UDF | 2.68 GB |
+| `apache/doris:all-in-one-<version>-full` | the above plus Hudi, Trino
connector, MaxCompute | 3.21 GB |
Pick `-full` only if the tests touch Hudi, the Trino connector or MaxCompute.
Both come up `healthy` in under 20 seconds. Sizes are the uncompressed layer
-sum measured on 4.1.3/arm64, against 4.9 GB for the same payload untouched.
-`docker image inspect --format '{{.Size}}'` reports 1.61 / 2.06 GiB for them;
+sum measured on 4.1.3/arm64, against 5.6 GB for the same payload untouched;
+about 0.2 GB of each is the meta-service, which only the cloud compose file
+uses. `docker image inspect --format '{{.Size}}'` reports 1.73 / 2.18 GiB;
`docker images` can print a much larger figure when the containerd image store
is enabled, because it adds the compressed blobs to the unpacked snapshot.
@@ -54,12 +60,19 @@ A plain run builds for the host architecture only.
Multi-arch needs an explicit
# base tag only, then smoke test it
./build.sh -v 4.1.3 -f base -t
-# from a locally built ./output instead
+# from a locally built ./output instead (build.sh --fe --be --cloud; without
+# --cloud there is no ms/ and the image cannot serve the cloud compose file)
./build.sh -v dev -s local
+
+# from an extracted release tarball, e.g. while the component images for a
+# new release are not on Docker Hub yet
+./build.sh -v 4.1.4 -s tarball --tarball-dir ~/apache-doris-4.1.4-bin-arm64
```
`./build.sh --help` lists the rest. The build context is the repository root,
-narrowed to a few KB by `Dockerfile.dockerignore`; run the script from
anywhere.
+narrowed to a few KB by `Dockerfile.dockerignore`; a local `./output` or a
+tarball directory is passed as BuildKit named contexts instead, so it can live
+anywhere. Run the script from anywhere.
## Multi-architecture
@@ -208,6 +221,7 @@ deliberately omits them and says so.
| | base | -full |
|---|---|---|
| `strip --strip-debug` on `doris_be` | 2213 MB → 430 MB (450 MB on arm64) |
same |
+| `strip --strip-debug` on the meta-service (`doris_cloud` and its two
`libfdb_c.so`) | 664 MB → 183 MB | same |
| `be/lib/meta_tool`, `be/lib/cdc_client`, `fe/arthas` | removed | removed |
| hudi / trino / maxcompute scanners | removed | kept |
@@ -235,17 +249,117 @@ It does **not** read a real external table. To check
that end of things, point
the built image at the fixtures under `docker/thirdparties` and run an Iceberg
or Hive query by hand.
+## Multi-node and cloud clusters
+
+The image's entrypoint takes a `DORIS_ROLE`. The default, `all`, is the
+single container described above; the other roles run one process each and
+are what the compose files are built from:
+
+| `DORIS_ROLE` | runs | needs |
+|---|---|---|
+| `all` | FE + BE on loopback | nothing |
+| `fe` | one FE; bootstraps the cluster when `FE_MASTER` is empty, otherwise
registers with it as `FE_ROLE` (`follower` \| `observer`) and joins |
`FE_MASTER` |
+| `be` | one BE, registered with `FE_MASTER`; in cloud mode into
`COMPUTE_GROUP` | `FE_MASTER` |
+| `ms`, `recycler` | the cloud meta-service / recycler | `FDB_CLUSTER` |
+| `cloud-init` | one-shot: creates the cloud instance on an S3-compatible
store, then exits | `MS_ENDPOINT`, `INSTANCE_ID`, `S3_*` |
+| `client` | waits for `EXPECT_FE` / `EXPECT_BE` live nodes, then idles with
`mysql` and `curl` | `FE_MASTER` |
+
+`DEPLOY_MODE=cloud` turns `fe` and `be` into cloud nodes (`deploy_mode`,
+`meta_service_endpoint`, file cache); the FE takes `INSTANCE_ID` as its
+`cluster_id` and manages nodes by SQL, so no `cloud_unique_id` has to be handed
+out. `FE_MASTER` may list several FEs (`fe-1,fe-2,fe-3`): the first one that
+answers is used, which keeps a node restart from waiting on the one FE that is
+down. An FE that already has metadata rejoins on its own. Every role honours
+`FE_CONFIG_EXTRA` / `BE_CONFIG_EXTRA` / `MS_CONFIG_EXTRA`, `FE_HEAP` /
`BE_HEAP`
+and drops a ready flag that the image `HEALTHCHECK` keys on.
+
+Topology-dependent settings (`priority_networks`, replica count, balancing,
+cloud keys) are written by the entrypoint at container start; the conf files
+baked into the image only carry the size-related defaults.
+
+### `compose/multi-node.yml` — three FEs, three BEs
+
+```shell
+cd docker/runtime/all-in-one/4.1/compose
+docker compose -f multi-node.yml up --wait # ~35 s to healthy
+docker compose -f multi-node.yml exec client mysql -uroot -hfe-1 -P9030
+docker compose -f multi-node.yml kill fe-1 # a new master in a few
seconds
+docker compose -f multi-node.yml start fe-1 # rejoins as a follower
+docker compose -f multi-node.yml down # nothing persists
+```
+
+Tables default to three replicas, one per BE. `fe-3` becomes an observer with
+`FE3_ROLE=observer`. Host ports: `9030`/`8030` for `fe-1`, `9031`/`8031` and
+`9032`/`8032` for the other two, `8040` for `be-1`.
+
+### `compose/cloud.yml` — compute-storage separated
+
+```shell
+docker compose -f cloud.yml up --wait # ~50 s to healthy
+docker compose -f cloud.yml --profile ha up --wait # ... plus two follower
FEs
+docker compose -f cloud.yml exec client mysql -uroot -hfe-1 -P9030
+```
+
+```
+fdb ─ fdb-init ─┬─ ms ─ cloud-init ─ fe-1 ─┬─ be-1, be-2 (compute group cg_a)
+ └─ recycler ├─ be-3 (compute group cg_b)
+minio ─ minio-init ─┘ └─ client
+```
+
+`cloud-init` creates the instance in storage-vault mode with MinIO as the
+vault (path-style, plain HTTP), and `fe-1` marks `built_in_storage_vault` as
+the default, so `CREATE TABLE` works as is. Both are idempotent, so
+`docker compose stop` / `up` keeps the data; `down` wipes everything. The
+MinIO console is on `9001` (`minioadmin` / `minioadmin`), the meta-service
+HTTP API on `15000` (`5000` is taken by AirPlay on macOS).
+
+What this is for: `use @cg_b`, `SHOW COMPUTE GROUPS`, `ALTER SYSTEM ADD
+BACKEND ... ("tag.compute_group_name" = ...)`, storage vaults, warm-up, FE
+failover in cloud mode, watching objects land in the bucket. What it is not:
+a stand-in for the cloud regression pipelines, and the `docker()` suites in
+`regression-test/suites/cloud_p0` still need `doris-compose`.
+
+### Things to know
+
+- **Addresses.** Nodes get fixed IPs on a private subnet (`SUBNET`, default
+ `172.31.80` for cloud and `172.31.81` for multi-node), so a restarted
+ container keeps the identity Doris knows it by. Both files can run at once
+ if one of them is given other host ports.
+- **From the host.** Use the published ports. On Docker Desktop the container
+ addresses are not routable from the host, so a stream load from the host
+ cannot follow FE's redirect to a BE; run it from the `client` service
+ instead, which is inside the network. Linux hosts can reach the nodes
+ directly.
+- **Memory.** Defaults are `FE_HEAP=1024m` and `mem_limit = 25%` per BE. A
+ full cloud cluster is around 9 GB; give Docker Desktop 12 GB or more, and do
+ not expect both topologies to fit side by side on a 16 GB VM.
+- **FoundationDB on arm64.** The upstream `foundationdb/foundationdb:7.1.x`
+ images are amd64 only, so on an Apple Silicon host `fdb` runs under
+ emulation. It is fine for this purpose; `FDB_IMAGE` / `FDB_PLATFORM` switch
+ it out. The meta-service links the 7.1 client, so stay on a 7.1 server.
+- **Smoke test.** `compose/smoke-test.sh <multi-node|cloud> [image:tag]`
+ brings the topology up under its own project name, subnet and host ports,
+ checks replicas or compute groups and the vault, stream loads through the
+ client, kills the master FE, restarts a BE while the old master is down,
+ brings the master back, and tears everything down. Two minutes each.
+
## Layout
```
4.1/
-├── Dockerfile three artifact sources -> strip + prune ->
runtime
+├── Dockerfile artifact sources -> strip + prune -> runtime
├── Dockerfile.dockerignore keeps the repo-root context to a few KB
├── build.sh the only entry point you need
+├── compose/
+│ ├── multi-node.yml 3 FE + 3 BE, storage and compute together
+│ ├── cloud.yml fdb + ms + recycler + minio + 1..3 FE + 3 BE in
2 compute groups
+│ └── smoke-test.sh guards the two compose files
└── resource/
- ├── entrypoint.sh FE -> readiness -> BE -> register -> fail-fast
wait
- ├── health_check.sh backs HEALTHCHECK
- ├── smoke-test.sh guards prune.txt
+ ├── entrypoint.sh one entrypoint, dispatched on DORIS_ROLE
+ ├── lib.sh shared by entrypoint.sh, health_check.sh and
cloud_init.sh
+ ├── cloud_init.sh creates the cloud instance
(DORIS_ROLE=cloud-init)
+ ├── health_check.sh backs HEALTHCHECK, per role
+ ├── smoke-test.sh guards prune.txt and the single-container role
├── prune.txt what each flavor drops, and what must never be
dropped
└── conf/{fe_ci.conf,be_ci.conf} appended to the upstream conf at build
time
```
diff --git a/docker/runtime/all-in-one/4.1/build.sh
b/docker/runtime/all-in-one/4.1/build.sh
index 55f509e1c85..c06eefb6f87 100755
--- a/docker/runtime/all-in-one/4.1/build.sh
+++ b/docker/runtime/all-in-one/4.1/build.sh
@@ -22,13 +22,13 @@ set -euo pipefail
HERE="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
REPO_ROOT="$(cd "${HERE}/../../../.." && pwd)"
-DOCKERFILE="docker/runtime/all-in-one/4.1/Dockerfile"
+DOCKERFILE="${HERE}/Dockerfile"
IMAGE="${IMAGE:-apache/doris}"
VERSION=""
SOURCE="image"
FLAVORS=""
-STRIP_BE="debug"
+STRIP_MODE="debug"
PLATFORM=""
LOCAL_OUTPUT="output"
TARBALL_DIR=""
@@ -51,16 +51,19 @@ Options:
-v, --version <v> Doris version, e.g. 4.1.3. Required.
-f, --flavor <f> base | full | both (default: both)
-s, --source <s> image | local | tarball (default: image)
- image -> apache/doris:fe-<v> and :be-<v>
- local -> ./output/{fe,be} from a local build
+ image -> apache/doris:fe-<v>, :be-<v> and :ms-<v>
+ local -> ./output/{fe,be,ms} from a local build
+ (ms is optional; without it the image
+ cannot serve the cloud compose file)
tarball -> --tarball-dir
- --tarball-dir <d> Extracted release package holding fe/ and be/,
- as a path relative to the repository root.
+ --tarball-dir <d> Extracted release package holding fe/, be/ and ms/.
+ Relative paths are tried against the repository root
+ first, then the current directory.
--local-output <d> Override the ./output path for --source local.
--strip <mode> debug | full | none (default: debug)
debug -> strip --strip-debug, keeps .symtab
full -> strip -s
- none -> ship doris_be as-is (+1.8 GB)
+ none -> ship doris_be and doris_cloud as-is (+2.2
GB)
--platform <p> Target platform(s), e.g. linux/amd64 or
linux/amd64,linux/arm64. Comma-separated values produce
one multi-arch tag (an OCI image index) that resolves
to
@@ -76,7 +79,10 @@ Options:
Examples:
./build.sh -v 4.1.3 # both flavors from the official
images
./build.sh -v 4.1.3 -f base -t # base only, then smoke test it
- ./build.sh -v dev -s local -f full # from a local ./output
+ ./build.sh -v dev -s local -f full # from a local ./output (build.sh
--fe --be --cloud)
+ ./build.sh -v 4.1.4 -s tarball --tarball-dir ~/apache-doris-4.1.4-bin-arm64
+ # from a release tarball, e.g.
before the
+ # component images are on Docker Hub
./build.sh -v 4.1.3 --platform linux/amd64,linux/arm64 --push
# one multi-arch tag for both
@@ -102,7 +108,7 @@ while [[ $# -gt 0 ]]; do
-s|--source) SOURCE="$2"; shift 2 ;;
--tarball-dir) TARBALL_DIR="$2"; shift 2 ;;
--local-output) LOCAL_OUTPUT="$2"; shift 2 ;;
- --strip) STRIP_BE="$2"; shift 2 ;;
+ --strip) STRIP_MODE="$2"; shift 2 ;;
--platform) PLATFORM="$2"; shift 2 ;;
-i|--image) IMAGE="$2"; shift 2 ;;
--no-cache) NO_CACHE=true; shift ;;
@@ -126,16 +132,34 @@ case "${SOURCE}" in
image|local|tarball) ;;
*) echo "error: bad --source '${SOURCE}'" >&2; exit 1 ;;
esac
-case "${STRIP_BE}" in
+case "${STRIP_MODE}" in
debug|full|none) ;;
- *) echo "error: bad --strip '${STRIP_BE}'" >&2; exit 1 ;;
+ *) echo "error: bad --strip '${STRIP_MODE}'" >&2; exit 1 ;;
esac
-if [[ "${SOURCE}" == tarball && -z "${TARBALL_DIR}" ]]; then
- echo "error: --source tarball needs --tarball-dir" >&2; exit 1
+# A local build and a release tarball are the same thing to the Dockerfile: a
+# directory with fe/, be/ and (optionally) ms/, handed over as named build
+# contexts so that the main context can stay narrowed to a few KB.
+ARTIFACT_DIR=""
+if [[ "${SOURCE}" == tarball ]]; then
+ [[ -n "${TARBALL_DIR}" ]] || { echo "error: --source tarball needs
--tarball-dir" >&2; exit 1; }
+ if [[ -d "${REPO_ROOT}/${TARBALL_DIR}" ]]; then
+ ARTIFACT_DIR="$(cd "${REPO_ROOT}/${TARBALL_DIR}" && pwd)"
+ elif [[ -d "${TARBALL_DIR}" ]]; then
+ ARTIFACT_DIR="$(cd "${TARBALL_DIR}" && pwd)"
+ else
+ echo "error: --tarball-dir ${TARBALL_DIR} not found" >&2; exit 1
+ fi
+elif [[ "${SOURCE}" == local ]]; then
+ ARTIFACT_DIR="${REPO_ROOT}/${LOCAL_OUTPUT}"
fi
-if [[ "${SOURCE}" == local && ! -d "${REPO_ROOT}/${LOCAL_OUTPUT}/be" ]]; then
- echo "error: ${REPO_ROOT}/${LOCAL_OUTPUT}/be not found; build Doris first"
>&2; exit 1
+if [[ -n "${ARTIFACT_DIR}" ]]; then
+ for part in fe be; do
+ [[ -d "${ARTIFACT_DIR}/${part}" ]] || { echo "error:
${ARTIFACT_DIR}/${part} not found; build Doris first" >&2; exit 1; }
+ done
+ if [[ ! -d "${ARTIFACT_DIR}/ms" ]]; then
+ echo "note: ${ARTIFACT_DIR}/ms not found (build.sh --cloud); the image
will not serve the cloud compose file"
+ fi
fi
command -v docker >/dev/null || { echo "error: docker not found" >&2; exit 1; }
@@ -159,7 +183,12 @@ if [[ "${MULTI_PLATFORM}" == true && "${RUN_TEST}" == true
]]; then
fi
builder="docker buildx build"
-docker buildx version >/dev/null 2>&1 || builder="docker build"
+if ! docker buildx version >/dev/null 2>&1; then
+ builder="docker build"
+ if [[ -n "${ARTIFACT_DIR}" ]]; then
+ echo "error: --source ${SOURCE} needs docker buildx (named build
contexts)" >&2; exit 1
+ fi
+fi
# Say which platforms are being built. Leaving --platform unset means the host
# architecture only, which is easy to mistake for a multi-arch build.
@@ -173,7 +202,7 @@ fi
echo "repository root : ${REPO_ROOT}"
echo "doris version : ${VERSION}"
echo "artifact source : ${SOURCE}"
-echo "strip mode : ${STRIP_BE}"
+echo "strip mode : ${STRIP_MODE}"
echo "flavors : ${FLAVORS}"
echo "platform(s) : ${platform_note}"
echo "output : $([[ "${PUSH}" == true ]] && echo 'push to registry'
|| echo 'load into local image store')"
@@ -188,12 +217,22 @@ for flavor in ${FLAVORS}; do
args=(
--build-arg "DORIS_VERSION=${VERSION}"
- --build-arg "ARTIFACT_SOURCE=${SOURCE}"
--build-arg "FLAVOR=${flavor}"
- --build-arg "STRIP_BE=${STRIP_BE}"
- --build-arg "LOCAL_OUTPUT=${LOCAL_OUTPUT}"
+ --build-arg "STRIP_MODE=${STRIP_MODE}"
)
- [[ -n "${TARBALL_DIR}" ]] && args+=(--build-arg
"TARBALL_DIR=${TARBALL_DIR}")
+ if [[ -n "${ARTIFACT_DIR}" ]]; then
+ args+=(--build-arg "ARTIFACT_SOURCE=dir"
+ --build-context "doris-fe=${ARTIFACT_DIR}/fe"
+ --build-context "doris-be=${ARTIFACT_DIR}/be")
+ if [[ -d "${ARTIFACT_DIR}/ms" ]]; then
+ args+=(--build-context "doris-ms=${ARTIFACT_DIR}/ms")
+ else
+ empty_ms="$(mktemp -d)"
+ args+=(--build-context "doris-ms=${empty_ms}")
+ fi
+ else
+ args+=(--build-arg "ARTIFACT_SOURCE=image")
+ fi
[[ -n "${PLATFORM}" ]] && args+=(--platform "${PLATFORM}")
[[ "${NO_CACHE}" == true ]] && args+=(--no-cache)
if [[ "${PUSH}" == true ]]; then
diff --git a/docker/runtime/all-in-one/4.1/compose/cloud.yml
b/docker/runtime/all-in-one/4.1/compose/cloud.yml
new file mode 100644
index 00000000000..3d798437540
--- /dev/null
+++ b/docker/runtime/all-in-one/4.1/compose/cloud.yml
@@ -0,0 +1,320 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+# Doris 4.1 in compute-storage separated (cloud) mode, for local feature work
+# and demos. Every Doris process runs from the all-in-one image; FoundationDB
+# and MinIO come from their upstream images.
+#
+# docker compose -f cloud.yml up --wait # 1 FE, 3 BE in 2 compute
groups
+# docker compose -f cloud.yml --profile ha up --wait # ... with 3 FE
+# docker compose -f cloud.yml exec client mysql -uroot -hfe-1 -P9030
+# docker compose -f cloud.yml down # clean slate (nothing
persists)
+#
+# fdb ─ fdb-init ─┬─ ms ─ cloud-init ─ fe-1 ─┬─ be-1, be-2 (compute group
cg_a)
+# └─ recycler ├─ be-3 (compute group
cg_b)
+# minio ─ minio-init ─┘ └─ client (healthy =
cluster ready)
+#
+# Knobs, as environment variables or a .env file next to this file:
+# DORIS_IMAGE apache/doris:all-in-one-4.1.3 any tag built from
../Dockerfile
+# SUBNET 172.31.80 first three octets; change
it to run
+# two clusters side by side
+# FE_PORT 9030 FE_HTTP_PORT 8030 BE_HTTP_PORT 8040 MS_PORT 15000 (5000 is
AirPlay on macOS)
+# MINIO_PORT 9000 MINIO_CONSOLE_PORT 9001 host ports
+# FE_HEAP 1024m BE_MEM_LIMIT 25% per-process memory
+# CG_A cg_a CG_B cg_b compute group names
+# INSTANCE_ID 100001 numeric; also the FEs'
cluster_id
+# S3_BUCKET doris S3_AK / S3_SK minioadmin
+# FDB_IMAGE foundationdb/foundationdb:7.1.23 amd64 only upstream: on an
arm64
+# FDB_PLATFORM linux/amd64 host it runs under
emulation
+#
+# Node addresses are pinned inside the private network, so a restarted
+# container keeps the identity Doris knows it by. From the host, use the
+# published ports; on Docker Desktop the container addresses themselves are
+# not routable, which is what the client service is for.
+
+name: doris-cloud
+
+x-node: &node
+ image: ${DORIS_IMAGE:-apache/doris:all-in-one-4.1.3}
+ stop_grace_period: 60s
+
+x-cloud-env: &cloud-env
+ DEPLOY_MODE: cloud
+ INSTANCE_ID: ${INSTANCE_ID:-100001}
+ MS_ENDPOINT: ms:5000
+ FE_MASTER: fe-1,fe-2,fe-3
+
+x-be: &be
+ <<: *node
+ depends_on:
+ fe-1:
+ condition: service_healthy
+
+services:
+ # ----------------------------------------------------- infrastructure ---
+ fdb:
+ image: ${FDB_IMAGE:-foundationdb/foundationdb:7.1.23}
+ platform: ${FDB_PLATFORM:-linux/amd64}
+ environment:
+ FDB_NETWORKING_MODE: container
+ FDB_PORT: "4500"
+ networks:
+ doris:
+ ipv4_address: ${SUBNET:-172.31.80}.10
+ healthcheck:
+ # "available" only once fdb-init has configured the database.
+ test: ["CMD-SHELL", "fdbcli -C /var/fdb/fdb.cluster --exec 'status
minimal' | grep -q 'is available'"]
+ interval: 5s
+ timeout: 5s
+ retries: 60
+ start_period: 10s
+
+ fdb-init:
+ image: ${FDB_IMAGE:-foundationdb/foundationdb:7.1.23}
+ platform: ${FDB_PLATFORM:-linux/amd64}
+ depends_on:
+ fdb:
+ condition: service_started
+ environment:
+ FDB_CLUSTER_FILE_CONTENTS: docker:docker@${SUBNET:-172.31.80}.10:4500
+ networks:
+ doris:
+ entrypoint: ["bash", "-c"]
+ command:
+ - |
+ echo "$$FDB_CLUSTER_FILE_CONTENTS" > /tmp/fdb.cluster
+ for i in $$(seq 1 60); do
+ out=$$(fdbcli -C /tmp/fdb.cluster --exec 'configure new single ssd'
2>&1)
+ case "$$out" in
+ *"Database created"*|*"already exists"*) echo "$$out"; exit 0 ;;
+ esac
+ sleep 2
+ done
+ echo "$$out"; exit 1
+
+ minio:
+ image: ${MINIO_IMAGE:-minio/minio:RELEASE.2025-01-20T14-49-07Z}
+ command: server /data --console-address :9001
+ environment:
+ MINIO_ROOT_USER: ${S3_AK:-minioadmin}
+ MINIO_ROOT_PASSWORD: ${S3_SK:-minioadmin}
+ ports:
+ - "${MINIO_PORT:-9000}:9000"
+ - "${MINIO_CONSOLE_PORT:-9001}:9001"
+ networks:
+ doris:
+ ipv4_address: ${SUBNET:-172.31.80}.20
+ healthcheck:
+ test: ["CMD", "mc", "ready", "local"]
+ interval: 5s
+ timeout: 5s
+ retries: 30
+
+ minio-init:
+ image: ${MINIO_IMAGE:-minio/minio:RELEASE.2025-01-20T14-49-07Z}
+ depends_on:
+ minio:
+ condition: service_healthy
+ environment:
+ MINIO_ROOT_USER: ${S3_AK:-minioadmin}
+ MINIO_ROOT_PASSWORD: ${S3_SK:-minioadmin}
+ S3_BUCKET: ${S3_BUCKET:-doris}
+ networks:
+ doris:
+ entrypoint: ["sh", "-c"]
+ command:
+ - mc alias set local http://minio:9000 "$$MINIO_ROOT_USER"
"$$MINIO_ROOT_PASSWORD" && mc mb --ignore-existing "local/$$S3_BUCKET"
+
+ # ----------------------------------------------------- doris cloud ------
+ ms:
+ <<: *node
+ environment:
+ DORIS_ROLE: ms
+ DEPLOY_MODE: cloud
+ FDB_CLUSTER: docker:docker@${SUBNET:-172.31.80}.10:4500
+ depends_on:
+ fdb:
+ condition: service_healthy
+ fdb-init:
+ condition: service_completed_successfully
+ ports:
+ - "${MS_PORT:-15000}:5000"
+ networks:
+ doris:
+ ipv4_address: ${SUBNET:-172.31.80}.30
+
+ recycler:
+ <<: *node
+ environment:
+ DORIS_ROLE: recycler
+ DEPLOY_MODE: cloud
+ FDB_CLUSTER: docker:docker@${SUBNET:-172.31.80}.10:4500
+ depends_on:
+ fdb:
+ condition: service_healthy
+ fdb-init:
+ condition: service_completed_successfully
+ networks:
+ doris:
+ ipv4_address: ${SUBNET:-172.31.80}.31
+
+ # One-shot: creates the instance on the meta-service with MinIO as its
+ # storage vault, then exits. A second run finds it and leaves it alone.
+ cloud-init:
+ <<: *node
+ environment:
+ DORIS_ROLE: cloud-init
+ DEPLOY_MODE: cloud
+ MS_ENDPOINT: ms:5000
+ INSTANCE_ID: ${INSTANCE_ID:-100001}
+ S3_ENDPOINT: minio:9000
+ S3_BUCKET: ${S3_BUCKET:-doris}
+ S3_AK: ${S3_AK:-minioadmin}
+ S3_SK: ${S3_SK:-minioadmin}
+ S3_PROVIDER: S3
+ S3_REGION: us-east-1
+ S3_PREFIX: ${S3_PREFIX:-doris}
+ S3_PATH_STYLE: "true"
+ depends_on:
+ ms:
+ condition: service_healthy
+ minio-init:
+ condition: service_completed_successfully
+ healthcheck:
+ disable: true
+ networks:
+ doris:
+
+ # ----------------------------------------------------- frontends --------
+ fe-1:
+ <<: *node
+ environment:
+ <<: *cloud-env
+ DORIS_ROLE: fe
+ FE_MASTER: ""
+ FE_HEAP: ${FE_HEAP:-1024m}
+ depends_on:
+ cloud-init:
+ condition: service_completed_successfully
+ ports:
+ - "${FE_PORT:-9030}:9030"
+ - "${FE_HTTP_PORT:-8030}:8030"
+ networks:
+ doris:
+ ipv4_address: ${SUBNET:-172.31.80}.40
+
+ fe-2:
+ <<: *node
+ profiles: [ha]
+ environment:
+ <<: *cloud-env
+ DORIS_ROLE: fe
+ FE_ROLE: follower
+ FE_HEAP: ${FE_HEAP:-1024m}
+ depends_on:
+ fe-1:
+ condition: service_healthy
+ ports:
+ - "${FE2_PORT:-9031}:9030"
+ - "${FE2_HTTP_PORT:-8031}:8030"
+ networks:
+ doris:
+ ipv4_address: ${SUBNET:-172.31.80}.41
+
+ fe-3:
+ <<: *node
+ profiles: [ha]
+ environment:
+ <<: *cloud-env
+ DORIS_ROLE: fe
+ FE_ROLE: follower
+ FE_HEAP: ${FE_HEAP:-1024m}
+ depends_on:
+ fe-1:
+ condition: service_healthy
+ ports:
+ - "${FE3_PORT:-9032}:9030"
+ - "${FE3_HTTP_PORT:-8032}:8030"
+ networks:
+ doris:
+ ipv4_address: ${SUBNET:-172.31.80}.42
+
+ # ----------------------------------------------------- backends ---------
+ # BEs are compute nodes here: data lives in MinIO, the local disk is only a
+ # file cache. Two compute groups, so `use @cg_b` has somewhere to go.
+ be-1:
+ <<: *be
+ environment:
+ <<: *cloud-env
+ DORIS_ROLE: be
+ COMPUTE_GROUP: ${CG_A:-cg_a}
+ BE_CONFIG_EXTRA: "mem_limit = ${BE_MEM_LIMIT:-25%}"
+ ports:
+ - "${BE_HTTP_PORT:-8040}:8040"
+ networks:
+ doris:
+ ipv4_address: ${SUBNET:-172.31.80}.50
+
+ be-2:
+ <<: *be
+ environment:
+ <<: *cloud-env
+ DORIS_ROLE: be
+ COMPUTE_GROUP: ${CG_A:-cg_a}
+ BE_CONFIG_EXTRA: "mem_limit = ${BE_MEM_LIMIT:-25%}"
+ networks:
+ doris:
+ ipv4_address: ${SUBNET:-172.31.80}.51
+
+ be-3:
+ <<: *be
+ environment:
+ <<: *cloud-env
+ DORIS_ROLE: be
+ COMPUTE_GROUP: ${CG_B:-cg_b}
+ BE_CONFIG_EXTRA: "mem_limit = ${BE_MEM_LIMIT:-25%}"
+ networks:
+ doris:
+ ipv4_address: ${SUBNET:-172.31.80}.52
+
+ # ----------------------------------------------------- client -----------
+ # Turns healthy once every node above is alive, which is what
+ # `docker compose up --wait` blocks on; then idles inside the network with a
+ # mysql client and curl, so stream load redirects to BE addresses resolve.
+ client:
+ <<: *node
+ environment:
+ DORIS_ROLE: client
+ FE_MASTER: fe-1,fe-2,fe-3
+ EXPECT_FE: ${EXPECT_FE:-1}
+ EXPECT_BE: 3
+ depends_on:
+ be-1:
+ condition: service_healthy
+ be-2:
+ condition: service_healthy
+ be-3:
+ condition: service_healthy
+ networks:
+ doris:
+
+networks:
+ doris:
+ driver: bridge
+ ipam:
+ config:
+ - subnet: ${SUBNET:-172.31.80}.0/24
diff --git a/docker/runtime/all-in-one/4.1/compose/multi-node.yml
b/docker/runtime/all-in-one/4.1/compose/multi-node.yml
new file mode 100644
index 00000000000..5f510c15329
--- /dev/null
+++ b/docker/runtime/all-in-one/4.1/compose/multi-node.yml
@@ -0,0 +1,168 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+# Doris 4.1 with storage and compute together on the BEs, three FEs and
+# three BEs, one container per node. For multi-node behaviour -- FE election
+# and failover, three replicas, balancing, decommission -- that the
+# single-container image cannot show.
+#
+# docker compose -f multi-node.yml up --wait
+# docker compose -f multi-node.yml exec client mysql -uroot -hfe-1 -P9030
+# docker compose -f multi-node.yml kill fe-1 # watch the election
+# docker compose -f multi-node.yml down # clean slate (nothing
persists)
+#
+# fe-1 (bootstraps) ─┬─ fe-2, fe-3 (followers, or FE3_ROLE=observer)
+# ├─ be-1, be-2, be-3
+# └─ client (healthy = cluster ready)
+#
+# Knobs, as environment variables or a .env file next to this file:
+# DORIS_IMAGE apache/doris:all-in-one-4.1.3 any tag built from
../Dockerfile
+# SUBNET 172.31.81 first three octets;
differs from
+# cloud.yml so both can run
at once
+# (give one of them other
host ports)
+# FE_PORT 9030 FE2_PORT 9031 FE3_PORT 9032 host ports, mysql protocol
+# FE_HTTP_PORT 8030 FE2_HTTP_PORT 8031 FE3_HTTP_PORT 8032 BE_HTTP_PORT
8040
+# FE_HEAP 1024m BE_MEM_LIMIT 25% per-process memory
+# FE3_ROLE follower or observer
+#
+# Tables default to three replicas here, one per BE, as they would on a real
+# cluster. Node addresses are pinned inside the private network, so a
+# restarted container keeps the identity Doris knows it by. From the host,
+# use the published ports; on Docker Desktop the container addresses
+# themselves are not routable, which is what the client service is for.
+
+name: doris-multi
+
+x-node: &node
+ image: ${DORIS_IMAGE:-apache/doris:all-in-one-4.1.3}
+ stop_grace_period: 60s
+
+x-fe-follower: &fe-follower
+ <<: *node
+ depends_on:
+ fe-1:
+ condition: service_healthy
+
+x-be: &be
+ <<: *node
+ depends_on:
+ fe-1:
+ condition: service_healthy
+
+services:
+ fe-1:
+ <<: *node
+ environment:
+ DORIS_ROLE: fe
+ FE_HEAP: ${FE_HEAP:-1024m}
+ ports:
+ - "${FE_PORT:-9030}:9030"
+ - "${FE_HTTP_PORT:-8030}:8030"
+ networks:
+ doris:
+ ipv4_address: ${SUBNET:-172.31.81}.40
+
+ fe-2:
+ <<: *fe-follower
+ environment:
+ DORIS_ROLE: fe
+ FE_MASTER: fe-1,fe-2,fe-3
+ FE_ROLE: follower
+ FE_HEAP: ${FE_HEAP:-1024m}
+ ports:
+ - "${FE2_PORT:-9031}:9030"
+ - "${FE2_HTTP_PORT:-8031}:8030"
+ networks:
+ doris:
+ ipv4_address: ${SUBNET:-172.31.81}.41
+
+ fe-3:
+ <<: *fe-follower
+ environment:
+ DORIS_ROLE: fe
+ FE_MASTER: fe-1,fe-2,fe-3
+ FE_ROLE: ${FE3_ROLE:-follower}
+ FE_HEAP: ${FE_HEAP:-1024m}
+ ports:
+ - "${FE3_PORT:-9032}:9030"
+ - "${FE3_HTTP_PORT:-8032}:8030"
+ networks:
+ doris:
+ ipv4_address: ${SUBNET:-172.31.81}.42
+
+ be-1:
+ <<: *be
+ environment:
+ DORIS_ROLE: be
+ FE_MASTER: fe-1,fe-2,fe-3
+ BE_CONFIG_EXTRA: "mem_limit = ${BE_MEM_LIMIT:-25%}"
+ ports:
+ - "${BE_HTTP_PORT:-8040}:8040"
+ networks:
+ doris:
+ ipv4_address: ${SUBNET:-172.31.81}.50
+
+ be-2:
+ <<: *be
+ environment:
+ DORIS_ROLE: be
+ FE_MASTER: fe-1,fe-2,fe-3
+ BE_CONFIG_EXTRA: "mem_limit = ${BE_MEM_LIMIT:-25%}"
+ networks:
+ doris:
+ ipv4_address: ${SUBNET:-172.31.81}.51
+
+ be-3:
+ <<: *be
+ environment:
+ DORIS_ROLE: be
+ FE_MASTER: fe-1,fe-2,fe-3
+ BE_CONFIG_EXTRA: "mem_limit = ${BE_MEM_LIMIT:-25%}"
+ networks:
+ doris:
+ ipv4_address: ${SUBNET:-172.31.81}.52
+
+ # Turns healthy once every node above is alive, which is what
+ # `docker compose up --wait` blocks on; then idles inside the network with a
+ # mysql client and curl, so stream load redirects to BE addresses resolve.
+ client:
+ <<: *node
+ environment:
+ DORIS_ROLE: client
+ FE_MASTER: fe-1,fe-2,fe-3
+ EXPECT_FE: 3
+ EXPECT_BE: 3
+ depends_on:
+ fe-2:
+ condition: service_healthy
+ fe-3:
+ condition: service_healthy
+ be-1:
+ condition: service_healthy
+ be-2:
+ condition: service_healthy
+ be-3:
+ condition: service_healthy
+ networks:
+ doris:
+
+networks:
+ doris:
+ driver: bridge
+ ipam:
+ config:
+ - subnet: ${SUBNET:-172.31.81}.0/24
diff --git a/docker/runtime/all-in-one/4.1/compose/smoke-test.sh
b/docker/runtime/all-in-one/4.1/compose/smoke-test.sh
new file mode 100755
index 00000000000..051662a3b48
--- /dev/null
+++ b/docker/runtime/all-in-one/4.1/compose/smoke-test.sh
@@ -0,0 +1,154 @@
+#!/usr/bin/env bash
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+#
+# Smoke test for the compose topologies: brings one up under its own project
+# name, subnet and host ports (so a cluster you are working with stays
+# untouched), exercises what the topology is for, tears it down.
+#
+# ./smoke-test.sh multi-node [image:tag]
+# ./smoke-test.sh cloud [image:tag]
+#
+# Everything runs through the client service, so the host needs only docker.
+
+set -euo pipefail
+
+TOPOLOGY=${1:?usage: smoke-test.sh <multi-node|cloud> [image:tag]}
+export DORIS_IMAGE=${2:-${DORIS_IMAGE:-apache/doris:all-in-one-4.1.3}}
+HERE="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
+WAIT_SECONDS=${WAIT_SECONDS:-600}
+
+case "${TOPOLOGY}" in
+ multi-node) export SUBNET=172.31.91 ;;
+ cloud) export SUBNET=172.31.92 ;;
+ *) echo "unknown topology ${TOPOLOGY}, expected multi-node|cloud" >&2;
exit 1 ;;
+esac
+export COMPOSE_PROJECT_NAME="doris-smoke-${TOPOLOGY}"
+export FE_PORT=29030 FE2_PORT=29031 FE3_PORT=29032 \
+ FE_HTTP_PORT=28030 FE2_HTTP_PORT=28031 FE3_HTTP_PORT=28032 \
+ BE_HTTP_PORT=28040 MS_PORT=25000 MINIO_PORT=29000
MINIO_CONSOLE_PORT=29001
+
+compose() { docker compose -f "${HERE}/${TOPOLOGY}.yml" --profile ha "$@"; }
+cleanup() {
+ local rc=$?
+ if ((rc != 0)); then
+ echo "--- compose state ---" >&2
+ compose ps >&2 || true
+ echo "--- recent logs ---" >&2
+ compose logs --tail 40 --no-log-prefix fe-1 be-1 2>&1 | tail -80 >&2
|| true
+ fi
+ compose down --remove-orphans >/dev/null 2>&1 || true
+ exit "${rc}"
+}
+trap cleanup EXIT
+
+step() { printf '\n== %s\n' "$*"; }
+fail() { echo "FAIL: $*" >&2; exit 1; }
+# q <fe host> <sql>: through the client container, batch output, no header
+q() { compose exec -T client mysql -uroot -h"$1" -P9030 -N --batch
--connect-timeout=3 -e "$2"; }
+# column <fe host> <show statement> <column name> [<filter column> <value>]
+column() {
+ compose exec -T client mysql -uroot -h"$1" -P9030 --batch
--connect-timeout=3 -e "$2" \
+ | awk -F'\t' -v want="$3" -v fc="${4:-}" -v fv="${5:-}" '
+ NR == 1 { for (i = 1; i <= NF; i++) col[$i] = i; next }
+ fc == "" || $col[fc] == fv { print $col[want] }'
+}
+master_fe() { column "$1" 'show frontends' Host IsMaster true; }
+wait_for() { # wait_for <seconds> <description> <command...>
+ local timeout=$1 what=$2; shift 2
+ local deadline=$((SECONDS + timeout))
+ while ((SECONDS < deadline)); do
+ if "$@" >/dev/null 2>&1; then return 0; fi
+ sleep 2
+ done
+ fail "timed out waiting for ${what}"
+}
+healthy() { [[ "$(docker inspect -f '{{.State.Health.Status}}'
"${COMPOSE_PROJECT_NAME}-$1-1" 2>/dev/null)" == healthy ]]; }
+stream_load() { # stream_load <fe host> <db.table> <csv lines>
+ local target=${2/./\/}
+ compose exec -T client bash -c "printf '$3' | curl -sS --location-trusted
-u root: \
+ -H 'column_separator:,' -H 'Expect:100-continue' -T -
http://$1:8030/api/${target}/_stream_load"; \
+ | grep -q '"Status": *"Success"'
+}
+
+step "starting ${TOPOLOGY} from ${DORIS_IMAGE} (project
${COMPOSE_PROJECT_NAME}, subnet ${SUBNET}.0/24)"
+compose up --wait --wait-timeout "${WAIT_SECONDS}" >/dev/null
+echo " up after ${SECONDS}s"
+compose ps --format 'table {{.Service}}\t{{.Status}}' | sed 's/^/ /'
+
+step "three FEs and three BEs alive"
+[[ "$(column fe-1 'show frontends' Alive | grep -c true)" == 3 ]] || fail
"expected 3 live FEs"
+[[ "$(column fe-1 'show backends' Alive | grep -c true)" == 3 ]] || fail
"expected 3 live BEs"
+echo " master is $(master_fe fe-1)"
+
+step "create, insert, read back"
+q fe-1 "create database if not exists smoke"
+q fe-1 "drop table if exists smoke.t"
+q fe-1 "create table smoke.t (k int, v varchar(32)) duplicate key(k)
distributed by hash(k) buckets 4"
+q fe-1 "insert into smoke.t values (1,'a'),(2,'b'),(3,'c')"
+[[ "$(q fe-1 'select count(*) from smoke.t')" == 3 ]] || fail "insert/select
mismatch"
+echo " 3 rows"
+
+if [[ "${TOPOLOGY}" == multi-node ]]; then
+ step "three replicas, one per backend"
+ q fe-1 "show create table smoke.t" | grep -q 'tag.location.default: 3' ||
fail "table is not 3-replica"
+ per_be=$(q fe-1 "show tablets from smoke.t" | awk -F'\t' '{n[$3]++} END
{print length(n)}')
+ [[ "${per_be}" == 3 ]] || fail "replicas spread over ${per_be} backends,
expected 3"
+ echo " 12 replicas over 3 backends"
+else
+ step "two compute groups, default storage vault on MinIO"
+ [[ "$(column fe-1 'show compute groups' BackendNum Name cg_a)" == 2 ]] ||
fail "cg_a should have 2 BEs"
+ [[ "$(column fe-1 'show compute groups' BackendNum Name cg_b)" == 1 ]] ||
fail "cg_b should have 1 BE"
+ [[ "$(column fe-1 'show storage vaults' IsDefault Name
built_in_storage_vault)" == true ]] \
+ || fail "built_in_storage_vault is not the default"
+ [[ "$(q fe-1 'use @cg_b; select count(*) from smoke.t')" == 3 ]] || fail
"read via cg_b failed"
+ echo " cg_a=2 cg_b=1, reads work from cg_b"
+ objects=$(compose exec -T minio sh -c 'mc alias set local
http://127.0.0.1:9000 "$MINIO_ROOT_USER" "$MINIO_ROOT_PASSWORD" >/dev/null &&
mc ls -r local/doris | wc -l')
+ ((objects > 0)) || fail "no objects in MinIO after the insert"
+ echo " ${objects} objects in MinIO"
+fi
+
+step "stream load through the client (redirect to a BE address)"
+stream_load fe-1 smoke.t '4,d\n5,e\n' || fail "stream load did not report
Success"
+[[ "$(q fe-1 'select count(*) from smoke.t')" == 5 ]] || fail "row count after
stream load"
+echo " 5 rows"
+
+step "kill the master FE, expect a new one"
+old_master=$(master_fe fe-1)
+compose kill fe-1 >/dev/null 2>&1
+new_master_elected() {
+ new_master=$(master_fe fe-2 2>/dev/null || true)
+ [[ -n "${new_master}" && "${new_master}" != "${old_master}" ]]
+}
+wait_for 120 "a new master" new_master_elected
+echo " ${old_master} -> ${new_master}"
+q fe-2 "insert into smoke.t values (6,'f')"
+[[ "$(q fe-2 'select count(*) from smoke.t')" == 6 ]] || fail "write through
the new master failed"
+echo " write through fe-2 ok"
+
+step "restart a BE while the old master is down, then bring the old master
back"
+compose restart be-2 >/dev/null 2>&1
+wait_for 180 "be-2 healthy" healthy be-2
+compose start fe-1 >/dev/null 2>&1
+wait_for 180 "fe-1 healthy" healthy fe-1
+[[ "$(column fe-1 'show frontends' Alive | grep -c true)" == 3 ]] || fail
"expected 3 live FEs after the restart"
+[[ "$(column fe-1 'show backends' Alive | grep -c true)" == 3 ]] || fail
"expected 3 live BEs after the restart"
+echo " everything alive again, master is $(master_fe fe-1)"
+
+step "cleanup"
+q fe-1 "drop database smoke force" >/dev/null 2>&1 || q fe-1 "drop database
smoke"
+printf '\nsmoke test passed: %s (%s)\n' "${TOPOLOGY}" "${DORIS_IMAGE}"
diff --git a/docker/runtime/all-in-one/4.1/resource/cloud_init.sh
b/docker/runtime/all-in-one/4.1/resource/cloud_init.sh
new file mode 100644
index 00000000000..57860ce1bb1
--- /dev/null
+++ b/docker/runtime/all-in-one/4.1/resource/cloud_init.sh
@@ -0,0 +1,64 @@
+#!/usr/bin/env bash
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+#
+# One-shot: create the cloud instance on the meta-service, backed by an
+# S3-compatible object store, then exit. Safe to run again -- an instance that
+# already exists is left alone.
+#
+# MS_ENDPOINT host:port of the meta-service
+# INSTANCE_ID numeric; the FEs use the same value as their cluster_id
+# S3_ENDPOINT host:port of the object store (MinIO in the compose file)
+# S3_BUCKET, S3_AK, S3_SK, S3_REGION, S3_PREFIX, S3_PROVIDER (S3 | OSS | COS
| OBS | BOS | GCP | AZURE)
+# S3_PATH_STYLE true for MinIO, false for the public clouds
+# VAULT_MODE true (default) creates a storage-vault instance, false a
legacy obj_info one
+
+set -Eeuo pipefail
+CI_HOME="${CI_HOME:-/opt/doris-ci}"
+# shellcheck source=lib.sh
+source "${CI_HOME}/lib.sh"
+
+: "${MS_ENDPOINT:?}" "${INSTANCE_ID:?}" "${S3_ENDPOINT:?}" "${S3_BUCKET:?}"
"${S3_AK:?}" "${S3_SK:?}"
+S3_REGION="${S3_REGION:-us-east-1}"
+S3_PREFIX="${S3_PREFIX:-doris}"
+S3_PROVIDER="${S3_PROVIDER:-S3}"
+S3_PATH_STYLE="${S3_PATH_STYLE:-true}"
+VAULT_MODE="${VAULT_MODE:-true}"
+MS_TOKEN="${MS_TOKEN:-greedisgood9999}"
+INSTANCE_NAME="${INSTANCE_NAME:-doris_${INSTANCE_ID}}"
+
+ms_url() { echo "http://${MS_ENDPOINT}/MetaService/http/$1?token=${MS_TOKEN}";;
}
+ms_up() { curl -fsS --max-time 4 "http://${MS_ENDPOINT}/health"; >/dev/null
2>&1; }
+
+wait_until "${START_TIMEOUT}" "meta-service at ${MS_ENDPOINT}" ms_up
+
+obj_info=$(printf
'{"ak":"%s","sk":"%s","bucket":"%s","endpoint":"%s","external_endpoint":"%s","prefix":"%s","region":"%s","provider":"%s","use_path_style":%s}'
\
+ "${S3_AK}" "${S3_SK}" "${S3_BUCKET}" "${S3_ENDPOINT}" "${S3_ENDPOINT}"
"${S3_PREFIX}" "${S3_REGION}" "${S3_PROVIDER}" "${S3_PATH_STYLE}")
+if [[ "${VAULT_MODE}" == true ]]; then
+ body=$(printf
'{"instance_id":"%s","name":"%s","user_id":"doris","vault":{"obj_info":%s}}'
"${INSTANCE_ID}" "${INSTANCE_NAME}" "${obj_info}")
+else
+ body=$(printf
'{"instance_id":"%s","name":"%s","user_id":"doris","obj_info":%s}'
"${INSTANCE_ID}" "${INSTANCE_NAME}" "${obj_info}")
+fi
+
+info "creating instance ${INSTANCE_ID} on ${S3_PROVIDER}
${S3_ENDPOINT}/${S3_BUCKET}/${S3_PREFIX} (vault=${VAULT_MODE},
path_style=${S3_PATH_STYLE})"
+out=$(curl -sS --max-time 30 "$(ms_url create_instance)" -d "${body}") || die
"create_instance request failed"
+code=$(sed -nE 's/.*"code"[[:space:]]*:[[:space:]]*"([A-Z_]+)".*/\1/p'
<<<"$(tr -d '\n' <<<"${out}")")
+case "${code}" in
+ OK) info "instance ${INSTANCE_ID} created" ;;
+ ALREADY_EXISTED) info "instance ${INSTANCE_ID} already exists, leaving it
alone" ;;
+ *) die "create_instance failed: ${out}" ;;
+esac
diff --git a/docker/runtime/all-in-one/4.1/resource/conf/be_ci.conf
b/docker/runtime/all-in-one/4.1/resource/conf/be_ci.conf
index 8e26b7cb095..b3cc7efe38b 100644
--- a/docker/runtime/all-in-one/4.1/resource/conf/be_ci.conf
+++ b/docker/runtime/all-in-one/4.1/resource/conf/be_ci.conf
@@ -21,7 +21,6 @@
# The JNI-side heap is not set here; the Dockerfile rewrites -Xmx/-Xms inside
# the upstream JAVA_OPTS_FOR_JDK_17 line.
-priority_networks = 127.0.0.1/32
# Upstream default is 90%, measured against the cgroup limit when there is one
# and against host memory when there is not. An unconstrained container on a
diff --git a/docker/runtime/all-in-one/4.1/resource/conf/fe_ci.conf
b/docker/runtime/all-in-one/4.1/resource/conf/fe_ci.conf
index 374fe60d120..c07c0d9a1a7 100644
--- a/docker/runtime/all-in-one/4.1/resource/conf/fe_ci.conf
+++ b/docker/runtime/all-in-one/4.1/resource/conf/fe_ci.conf
@@ -24,18 +24,10 @@
# The heap is not set here; the Dockerfile rewrites the -Xmx/-Xms tokens inside
# the upstream JAVA_OPTS_FOR_JDK_17 line so the --add-opens flags stay intact.
-# One BE means one replica. Saves every downstream CREATE TABLE from having to
-# spell out replication_num=1. Config.java describes this as test-environment
-# only, which is exactly what this image is.
-force_olap_table_replication_num = 1
-
-# Everything lives on the container loopback. Pinning IP mode as well keeps a
-# restart under a different container hostname from invalidating the metadata.
-priority_networks = 127.0.0.1/32
-enable_fqdn_mode = false
-
-# Nothing to balance across a single backend.
-disable_balance = true
+# Keys that depend on the topology -- network identity, replica count,
+# balancing -- are not here: entrypoint.sh writes them per DORIS_ROLE at
+# container start, so that the same image serves the single-container image
+# and the multi-node and cloud compose files.
# Keep the log directory bounded over a long CI run.
sys_log_roll_num = 2
diff --git a/docker/runtime/all-in-one/4.1/resource/entrypoint.sh
b/docker/runtime/all-in-one/4.1/resource/entrypoint.sh
index 1ce68ee0efa..4fe54c8a5be 100755
--- a/docker/runtime/all-in-one/4.1/resource/entrypoint.sh
+++ b/docker/runtime/all-in-one/4.1/resource/entrypoint.sh
@@ -16,93 +16,168 @@
# specific language governing permissions and limitations
# under the License.
#
-# Brings up one FE and one BE inside a single container and keeps them there.
+# Entrypoint for every role of the all-in-one image. DORIS_ROLE selects what
+# the container runs; each role is plain Doris started with --console, so
+# `docker logs` carries the process output and the container exits non-zero
+# when the process does. `docker stop` shuts down gracefully and exits 0.
#
-# Fail-fast by design: if either process exits, so does the container, with a
-# non-zero status. A test fixture that quietly restarts a dead FE turns a
-# two-second failure into a job timeout.
+# all one FE and one BE on loopback -- the single-container image
(default)
+# fe an FE: bootstraps the cluster when FE_MASTER is unset, joins
it otherwise
+# be a BE that registers with FE_MASTER
+# ms the cloud meta-service } DEPLOY_MODE=cloud only; FDB_CLUSTER
+# recycler the cloud recycler } names the FoundationDB to use
+# cloud-init one-shot: create the cloud instance on the object store, then
exit 0
+# client waits until EXPECT_FE / EXPECT_BE nodes are alive, then idles
+#
+# Every role reads FE_CONFIG_EXTRA / BE_CONFIG_EXTRA / MS_CONFIG_EXTRA
(appended
+# to the respective conf), FE_HEAP / BE_HEAP (JVM -Xmx, e.g. 1024m) and
+# START_TIMEOUT. The compose files under ../compose show the rest in use.
set -Eeuo pipefail
-
-# Job control, so each child lands in its own process group. This is what makes
-# shutdown work: start_fe.sh / start_be.sh run the real java / doris_be process
-# in the foreground and do not forward signals, and neither writes a usable pid
-# file under --console, so stop_fe.sh / stop_be.sh cannot help us. Signalling
-# the whole group reaches the actual process.
set -m
-DORIS_HOME="${DORIS_HOME:-/opt/apache-doris}"
-FE_HOME="${DORIS_HOME}/fe"
-BE_HOME="${DORIS_HOME}/be"
-READY_FLAG="${DORIS_HOME}/.ready"
-
-HOST=127.0.0.1
-FE_HTTP_PORT="${FE_HTTP_PORT:-8030}"
-FE_QUERY_PORT="${FE_QUERY_PORT:-9030}"
-BE_HEARTBEAT_PORT="${BE_HEARTBEAT_PORT:-9050}"
-START_TIMEOUT="${START_TIMEOUT:-300}"
-STOP_TIMEOUT="${STOP_TIMEOUT:-30}"
+CI_HOME="${CI_HOME:-/opt/doris-ci}"
+# shellcheck source=lib.sh
+source "${CI_HOME}/lib.sh"
+MY_IP="$(my_ip)"
+PRIORITY_NETWORKS="${PRIORITY_NETWORKS:-${MY_IP}/32}"
+MASTER_IP=
FE_PID=
BE_PID=
+MS_PID=
-log() { printf '%s [%-5s] [entrypoint] %s\n' "$(date -Iseconds)" "$1"
"${*:2}"; }
-info() { log INFO "$@"; }
-warn() { log WARN "$@" >&2; }
-die() { log ERROR "$@" >&2; exit 1; }
-
-sql() {
- mysql -uroot -h"${HOST}" -P"${FE_QUERY_PORT}" -N --batch
--connect-timeout=2 -e "$1" 2>/dev/null
+require_env() {
+ local v
+ for v in "$@"; do
+ [[ -n "${!v:-}" ]] || die "${v} must be set for
DORIS_ROLE=${DORIS_ROLE} DEPLOY_MODE=${DEPLOY_MODE}"
+ done
}
-fe_health() {
- # Public endpoint (HealthAction): 503 until FE is ready, otherwise a body
- # carrying online_backend_num. curl -f turns the 503 into a non-zero exit.
- curl -fsS --max-time 4 "http://${HOST}:${FE_HTTP_PORT}/api/health";
2>/dev/null
-}
+# ------------------------------------------------------------ conf blocks ---
+# What each role appends to the shipped conf. Topology keys live here rather
+# than in the image so that one image serves every topology.
-# ---------------------------------------------------------------- config ----
-# The image already carries the integration-test defaults; this is the runtime
-# escape hatch for downstream projects that need one knob changed.
-apply_env_overrides() {
- if [[ -n "${FE_CONFIG_EXTRA:-}" ]]; then
- info "appending FE_CONFIG_EXTRA to fe.conf"
- printf '\n# --- FE_CONFIG_EXTRA ---\n%s\n' "${FE_CONFIG_EXTRA}"
>>"${FE_HOME}/conf/fe.conf"
+fe_conf_block() {
+ printf 'priority_networks = %s\nenable_fqdn_mode = false\n'
"${PRIORITY_NETWORKS}"
+ if [[ "${DORIS_ROLE}" == all ]]; then
+ # One BE means one replica: saves every downstream CREATE TABLE from
+ # spelling out replication_num=1 (Config.java calls this a
+ # test-environment knob, which is what this image is), and there is
+ # nothing to balance across a single backend.
+ printf 'force_olap_table_replication_num = 1\ndisable_balance = true\n'
fi
- if [[ -n "${BE_CONFIG_EXTRA:-}" ]]; then
- info "appending BE_CONFIG_EXTRA to be.conf"
- printf '\n# --- BE_CONFIG_EXTRA ---\n%s\n' "${BE_CONFIG_EXTRA}"
>>"${BE_HOME}/conf/be.conf"
+ if [[ "${DEPLOY_MODE}" == cloud ]]; then
+ # SQL-managed cloud node: FE derives its cloud_unique_id from
+ # cluster_id, which therefore has to be the numeric instance id that
+ # cloud-init created on the meta-service.
+ printf 'deploy_mode = cloud\ncluster_id = %s\nmeta_service_endpoint =
%s\n' \
+ "${INSTANCE_ID}" "${MS_ENDPOINT}"
fi
+ [[ -z "${FE_CONFIG_EXTRA:-}" ]] || printf '# --- FE_CONFIG_EXTRA
---\n%s\n' "${FE_CONFIG_EXTRA}"
}
-# ------------------------------------------------------------------- FE -----
-start_fe() {
- info "starting FE"
- "${FE_HOME}/bin/start_fe.sh" --console &
- FE_PID=$!
+be_conf_block() {
+ printf 'priority_networks = %s\n' "${PRIORITY_NETWORKS}"
+ if [[ "${DEPLOY_MODE}" == cloud ]]; then
+ printf 'deploy_mode = cloud\nmeta_service_endpoint =
%s\nenable_file_cache = true\n' "${MS_ENDPOINT}"
+ printf 'file_cache_path =
[{"path":"%s/storage/file_cache","total_size":%s,"query_limit":%s}]\n' \
+ "${BE_HOME}" "${BE_FILE_CACHE_BYTES:-2147483648}"
"${BE_FILE_CACHE_QUERY_LIMIT_BYTES:-1073741824}"
+ printf 'tmp_file_dirs =
[{"path":"%s/storage/tmp","max_cache_bytes":104857600,"max_upload_bytes":104857600}]\n'
\
+ "${BE_HOME}"
+ fi
+ [[ -z "${BE_CONFIG_EXTRA:-}" ]] || printf '# --- BE_CONFIG_EXTRA
---\n%s\n' "${BE_CONFIG_EXTRA}"
+}
+
+ms_conf_block() {
+ printf 'fdb_cluster = %s\nbrpc_listen_port = %s\n' "${FDB_CLUSTER}"
"${MS_PORT}"
+ [[ -z "${MS_CONFIG_EXTRA:-}" ]] || printf '# --- MS_CONFIG_EXTRA
---\n%s\n' "${MS_CONFIG_EXTRA}"
}
-wait_fe_ready() {
- local deadline=$((SECONDS + START_TIMEOUT))
+# ---------------------------------------------------------------- probes ---
+
+fe_up() { fe_health "$1" >/dev/null; }
+
+# Pick the FE to talk to: the first entry of FE_MASTER (comma-separated
+# hosts) that resolves and answers /api/health. Any live FE will do --
+# ALTER SYSTEM is forwarded to whichever FE is master -- so listing every FE
+# keeps a BE restart from waiting on the one FE that happens to be down.
+# Doris identifies nodes by IP here (enable_fqdn_mode is off), so
+# registration, --helper and the SQL probes all use the address.
+resolve_master() {
+ require_env FE_MASTER
+ local deadline=$((SECONDS + START_TIMEOUT)) host ip
while ((SECONDS < deadline)); do
- kill -0 "${FE_PID}" 2>/dev/null \
- || die "FE exited during startup, see ${FE_HOME}/log/fe.log"
- # Two gates: the HTTP endpoint reports FE readiness, and a metadata
- # query proves the MySQL port is actually serving. It has to be a
- # metadata query -- `select 1` goes through Nereids, which picks a
- # backend as its scan node and fails with "No backend available" until
- # one is registered. The BE is not started yet at this point, so using
- # it here would deadlock the two waits against each other.
- if fe_health >/dev/null && sql 'show frontends' | grep -q "${HOST}";
then
- info "FE is ready after ${SECONDS}s"
- return 0
- fi
+ for host in ${FE_MASTER//,/ }; do
+ ip="$(getent hosts "${host}" | awk '{print $1; exit}')" || true
+ [[ -n "${ip}" ]] || continue
+ if fe_up "${ip}"; then
+ MASTER_IP="${ip}"
+ info "using FE ${host} (${ip})"
+ return 0
+ fi
+ done
sleep 1
done
- die "FE did not become ready within ${START_TIMEOUT}s, see
${FE_HOME}/log/fe.log"
+ die "no FE in FE_MASTER=${FE_MASTER} answered within ${START_TIMEOUT}s"
+}
+
+# The local FE is up: the HTTP endpoint reports ready, and a metadata query
+# proves the MySQL port serves. It has to be a metadata query -- `select 1`
+# goes through Nereids, which picks a backend as its scan node and fails with
+# "No backend available" until one is registered.
+fe_alive_here() {
+ kill -0 "${FE_PID}" 2>/dev/null || die "FE exited during startup, see
${FE_HOME}/log/fe.log"
+ fe_up 127.0.0.1 && [[ "$(node_alive 127.0.0.1 frontends "${MY_IP}"
"${FE_EDIT_LOG_PORT}")" == true ]]
+}
+
+# The local BE is up as far as the master FE is concerned, which is the only
+# opinion that matters to a client.
+be_alive_here() {
+ kill -0 "${BE_PID}" 2>/dev/null || die "BE exited during startup, see
${BE_HOME}/log/be.INFO and ${BE_HOME}/log/be.out"
+ [[ "$(node_alive "${MASTER_IP}" backends "${MY_IP}"
"${BE_HEARTBEAT_PORT}")" == true ]]
+}
+
+tcp_open() { (exec 3<>"/dev/tcp/$1/$2") 2>/dev/null; }
+
+ms_alive_here() {
+ kill -0 "${MS_PID}" 2>/dev/null || die "${DORIS_ROLE} exited during
startup, see ${MS_HOME}/log/"
+ curl -fsS --max-time 4 "http://127.0.0.1:${MS_PORT}/health"; >/dev/null 2>&1
+}
+
+# A cloud instance created in storage-vault mode carries a
built_in_storage_vault
+# that nothing marks as default, and CREATE TABLE refuses to pick one on its
own.
+ensure_default_vault() {
+ local vaults
+ vaults="$(sql 127.0.0.1 'show storage vaults')" || return 0
+ if awk -F'\t' '$NF == "true" { found = 1 } END { exit !found }'
<<<"${vaults}"; then
+ return 0
+ fi
+ if grep -q '^built_in_storage_vault' <<<"${vaults}"; then
+ info "setting built_in_storage_vault as the default storage vault"
+ sql 127.0.0.1 'set built_in_storage_vault as default storage vault' \
+ || warn "could not set the default storage vault; CREATE TABLE
will need an explicit one"
+ fi
+}
+
+# park <name> <pid>: block until the process exits, then fail the container.
+# `wait` also returns when a trap fires; the trap exits on its own, so getting
+# past it means the process is really gone.
+park() {
+ local rc=0
+ wait "$2" || rc=$?
+ rm -f "${READY_FLAG}"
+ die "$1 exited (rc=${rc})"
+}
+
+# ------------------------------------------------------------------ roles ---
+
+start_fe() {
+ info "starting FE${*:+ $*}"
+ "${FE_HOME}/bin/start_fe.sh" "$@" --console &
+ FE_PID=$!
}
-# ------------------------------------------------------------------- BE -----
start_be() {
info "starting BE"
"${BE_HOME}/bin/start_be.sh" --console &
@@ -110,85 +185,158 @@ start_be() {
}
register_be() {
- # Idempotent: a container restarted on a mounted doris-meta already has the
- # backend in its metadata.
- if sql 'show backends' | grep -qE
"[[:space:]]${HOST}[[:space:]]+${BE_HEARTBEAT_PORT}[[:space:]]"; then
- info "backend ${HOST}:${BE_HEARTBEAT_PORT} already registered"
+ local stmt="alter system add backend '${MY_IP}:${BE_HEARTBEAT_PORT}'"
+ if [[ "${DEPLOY_MODE}" == cloud ]]; then
+ stmt+=" properties ('tag.compute_group_name' =
'${COMPUTE_GROUP:-default_compute_group}')"
+ fi
+ # Idempotent: a container restarted on a mounted doris-meta already has it.
+ if [[ -n "$(node_alive "${MASTER_IP}" backends "${MY_IP}"
"${BE_HEARTBEAT_PORT}")" ]]; then
+ info "backend ${MY_IP}:${BE_HEARTBEAT_PORT} already registered"
else
- info "registering backend ${HOST}:${BE_HEARTBEAT_PORT}"
- sql "alter system add backend '${HOST}:${BE_HEARTBEAT_PORT}'" \
- || die "ALTER SYSTEM ADD BACKEND failed"
+ info "registering backend
${MY_IP}:${BE_HEARTBEAT_PORT}${COMPUTE_GROUP:+ in compute group
${COMPUTE_GROUP}}"
+ sql "${MASTER_IP}" "${stmt}" || die "ALTER SYSTEM ADD BACKEND failed"
fi
}
-wait_be_alive() {
- local deadline=$((SECONDS + START_TIMEOUT))
- while ((SECONDS < deadline)); do
- kill -0 "${BE_PID}" 2>/dev/null \
- || die "BE exited during startup, see ${BE_HOME}/log/be.INFO and
${BE_HOME}/log/be.out"
- # FE reports how many backends it considers alive, so one request
- # answers both "is the BE up" and "did FE notice".
- if fe_health | grep -qE
'"online_backend_num"[[:space:]]*:[[:space:]]*[1-9]'; then
- info "backend is alive after ${SECONDS}s"
- return 0
+run_all() {
+ [[ "${DEPLOY_MODE}" == local ]] || die "DORIS_ROLE=all only supports
DEPLOY_MODE=local"
+ MASTER_IP=127.0.0.1
+ render_conf "${FE_HOME}/conf/fe.conf" "$(fe_conf_block)"
+ render_conf "${BE_HOME}/conf/be.conf" "$(be_conf_block)"
+ [[ -z "${FE_HEAP:-}" ]] || set_heap "${FE_HOME}/conf/fe.conf" "${FE_HEAP}"
+ [[ -z "${BE_HEAP:-}" ]] || set_heap "${BE_HOME}/conf/be.conf" "${BE_HEAP}"
+ start_fe
+ wait_until "${START_TIMEOUT}" "FE to come up" fe_alive_here
+ start_be
+ register_be
+ wait_until "${START_TIMEOUT}" "the backend to come alive" be_alive_here
+ touch "${READY_FLAG}"
+ info "cluster is ready -- mysql -uroot -h127.0.0.1 -P${FE_QUERY_PORT}"
+ local rc=0
+ wait -n "${FE_PID}" "${BE_PID}" || rc=$?
+ rm -f "${READY_FLAG}"
+ if ! kill -0 "${FE_PID}" 2>/dev/null; then
+ die "FE exited (rc=${rc}), see ${FE_HOME}/log/fe.log"
+ fi
+ die "BE exited (rc=${rc}), see ${BE_HOME}/log/be.INFO and
${BE_HOME}/log/be.out"
+}
+
+run_fe() {
+ [[ "${DEPLOY_MODE}" != cloud ]] || require_env INSTANCE_ID MS_ENDPOINT
+ render_conf "${FE_HOME}/conf/fe.conf" "$(fe_conf_block)"
+ [[ -z "${FE_HEAP:-}" ]] || set_heap "${FE_HOME}/conf/fe.conf" "${FE_HEAP}"
+ if [[ -z "${FE_MASTER}" ]]; then
+ start_fe
+ wait_until "${START_TIMEOUT}" "FE to come up" fe_alive_here
+ [[ "${DEPLOY_MODE}" != cloud ]] || ensure_default_vault
+ else
+ local role="${FE_ROLE:-follower}"
+ case "${role}" in follower|observer) ;; *) die "bad FE_ROLE=${role},
expected follower|observer" ;; esac
+ if [[ -d "${FE_HOME}/doris-meta/bdb" ]]; then
+ # Already a member: the metadata knows the peers, so rejoin
+ # without waiting on any other FE, which may well be down.
+ info "metadata present, rejoining the cluster"
+ start_fe
+ else
+ resolve_master
+ # A node has to be in the master's metadata before it may join
+ # with --helper.
+ if [[ -z "$(node_alive "${MASTER_IP}" frontends "${MY_IP}"
"${FE_EDIT_LOG_PORT}")" ]]; then
+ info "registering as ${role} ${MY_IP}:${FE_EDIT_LOG_PORT}"
+ sql "${MASTER_IP}" "alter system add ${role}
'${MY_IP}:${FE_EDIT_LOG_PORT}'" \
+ || die "ALTER SYSTEM ADD ${role^^} failed"
+ fi
+ start_fe --helper "${MASTER_IP}:${FE_EDIT_LOG_PORT}"
fi
- sleep 1
- done
- die "backend did not come alive within ${START_TIMEOUT}s"
-}
-
-# -------------------------------------------------------------- shutdown ----
-stop_one() {
- local name=$1 pid=$2
- [[ -n "${pid}" ]] || return 0
- kill -0 "${pid}" 2>/dev/null || return 0
- info "stopping ${name}"
- # Negative pid signals the whole process group, which is where the real
- # java / doris_be process lives.
- kill -TERM -"${pid}" 2>/dev/null || kill -TERM "${pid}" 2>/dev/null || true
- local deadline=$((SECONDS + STOP_TIMEOUT))
- while ((SECONDS < deadline)); do
- kill -0 "${pid}" 2>/dev/null || { info "${name} stopped"; return 0; }
- sleep 1
- done
- warn "${name} did not stop within ${STOP_TIMEOUT}s, killing"
- kill -KILL -"${pid}" 2>/dev/null || kill -KILL "${pid}" 2>/dev/null || true
+ wait_until "${START_TIMEOUT}" "FE to join the cluster" fe_alive_here
+ fi
+ touch "${READY_FLAG}"
+ info "FE is ready -- mysql -uroot -h${MY_IP} -P${FE_QUERY_PORT}"
+ park FE "${FE_PID}"
+}
+
+run_be() {
+ [[ "${DEPLOY_MODE}" != cloud ]] || require_env MS_ENDPOINT
+ mkdir -p "${BE_HOME}/storage/file_cache" "${BE_HOME}/storage/tmp"
+ render_conf "${BE_HOME}/conf/be.conf" "$(be_conf_block)"
+ [[ -z "${BE_HEAP:-}" ]] || set_heap "${BE_HOME}/conf/be.conf" "${BE_HEAP}"
+ resolve_master
+ register_be
+ start_be
+ wait_until "${START_TIMEOUT}" "the backend to come alive" be_alive_here
+ touch "${READY_FLAG}"
+ info "BE is ready"
+ park BE "${BE_PID}"
+}
+
+run_ms() {
+ require_env FDB_CLUSTER
+ [[ -x "${MS_HOME}/bin/start.sh" ]] || die "this image carries no
meta-service payload (${MS_HOME})"
+ local flag=--meta-service
+ [[ "${DORIS_ROLE}" == recycler ]] && flag=--recycler
+ render_conf "${MS_HOME}/conf/doris_cloud.conf" "$(ms_conf_block)"
+ mkdir -p "${MS_HOME}/log"
+ # doris_cloud gives up quickly when FoundationDB is not there yet; the
+ # compose files order it after fdb, this covers a hand-run container.
+ local fdb_addr="${FDB_CLUSTER##*@}"
+ wait_until "${START_TIMEOUT}" "FoundationDB at ${fdb_addr}" tcp_open
"${fdb_addr%:*}" "${fdb_addr##*:}"
+ info "starting ${DORIS_ROLE}"
+ (cd "${MS_HOME}" && bash bin/start.sh ${flag} --console) &
+ MS_PID=$!
+ wait_until "${START_TIMEOUT}" "${DORIS_ROLE} to come up" ms_alive_here
+ touch "${READY_FLAG}"
+ info "${DORIS_ROLE} is ready on ${MY_IP}:${MS_PORT}"
+ park "${DORIS_ROLE}" "${MS_PID}"
}
+run_cloud_init() {
+ exec "${CI_HOME}/cloud_init.sh"
+}
+
+# Sit in the network as a ready-made client: `docker compose up --wait`
+# returns once this container is healthy, i.e. once every expected node is
+# alive, and `docker compose exec client mysql ...` reaches the cluster.
+run_client() {
+ resolve_master
+ local want_fe="${EXPECT_FE:-1}" want_be="${EXPECT_BE:-1}"
+ count_alive() {
+ [[ "$(alive_count "${MASTER_IP}" frontends)" -ge "${want_fe}" ]] \
+ && [[ "$(alive_count "${MASTER_IP}" backends)" -ge "${want_be}" ]]
+ }
+ wait_until "${START_TIMEOUT}" "${want_fe} FE and ${want_be} BE to be
alive" count_alive
+ touch "${READY_FLAG}"
+ info "cluster is ready: $(alive_count "${MASTER_IP}" frontends) FE,
$(alive_count "${MASTER_IP}" backends) BE"
+ info "connect with: mysql -uroot -h${FE_MASTER%%,*} -P${FE_QUERY_PORT}"
+ sleep infinity &
+ wait $!
+}
+
+# --------------------------------------------------------------- lifecycle ---
+
shutdown() {
trap - SIGTERM SIGINT
rm -f "${READY_FLAG}"
# BE first, so it stops reporting to an FE that is about to go away.
stop_one BE "${BE_PID}"
stop_one FE "${FE_PID}"
+ stop_one "${DORIS_ROLE}" "${MS_PID}"
exit 0
}
-# ------------------------------------------------------------------ main ----
main() {
trap shutdown SIGTERM SIGINT
rm -f "${READY_FLAG}"
-
- apply_env_overrides
- start_fe
- wait_fe_ready
- start_be
- register_be
- wait_be_alive
-
- touch "${READY_FLAG}"
- info "cluster is ready -- mysql -uroot -h127.0.0.1 -P${FE_QUERY_PORT}"
-
- # Park here until something dies. `wait -n` also returns when a trap fires,
- # so the explicit re-check below distinguishes the two cases.
- local rc=0
- wait -n "${FE_PID}" "${BE_PID}" || rc=$?
- rm -f "${READY_FLAG}"
-
- if ! kill -0 "${FE_PID}" 2>/dev/null; then
- die "FE exited (rc=${rc}), see ${FE_HOME}/log/fe.log"
- fi
- die "BE exited (rc=${rc}), see ${BE_HOME}/log/be.INFO and
${BE_HOME}/log/be.out"
+ case "${DEPLOY_MODE}" in local|cloud) ;; *) die "bad
DEPLOY_MODE=${DEPLOY_MODE}, expected local|cloud" ;; esac
+ info "role=${DORIS_ROLE} mode=${DEPLOY_MODE} ip=${MY_IP}"
+ case "${DORIS_ROLE}" in
+ all) run_all ;;
+ fe) run_fe ;;
+ be) run_be ;;
+ ms|recycler) run_ms ;;
+ cloud-init) run_cloud_init ;;
+ client) run_client ;;
+ *) die "unknown DORIS_ROLE=${DORIS_ROLE}" ;;
+ esac
}
main "$@"
diff --git a/docker/runtime/all-in-one/4.1/resource/health_check.sh
b/docker/runtime/all-in-one/4.1/resource/health_check.sh
index 16eaeebc18f..86d881761a8 100755
--- a/docker/runtime/all-in-one/4.1/resource/health_check.sh
+++ b/docker/runtime/all-in-one/4.1/resource/health_check.sh
@@ -16,26 +16,25 @@
# specific language governing permissions and limitations
# under the License.
#
-# Backs the image HEALTHCHECK. Downstream CI waits on the resulting docker
-# health status instead of sleeping.
+# Backs the image HEALTHCHECK. The entrypoint drops a ready flag once its
+# role has come up; from then on the role's own endpoint has to keep
+# answering, so a dead process turns the container unhealthy.
set -uo pipefail
+CI_HOME="${CI_HOME:-/opt/doris-ci}"
+# shellcheck source=lib.sh
+source "${CI_HOME}/lib.sh"
-DORIS_HOME="${DORIS_HOME:-/opt/apache-doris}"
-HOST=127.0.0.1
-FE_HTTP_PORT="${FE_HTTP_PORT:-8030}"
-BE_HTTP_PORT="${BE_HTTP_PORT:-8040}"
-
-# Bootstrap not finished yet: FE may answer while the backend is still being
-# registered, and a test that connects then sees a cluster with no capacity.
-[[ -f "${DORIS_HOME}/.ready" ]] || exit 1
-
-# FE readiness and backend liveness in one request: HealthAction returns 503
-# until FE is ready, and online_backend_num once it is.
-curl -fsS --max-time 4 "http://${HOST}:${FE_HTTP_PORT}/api/health"; 2>/dev/null
\
- | grep -qE '"online_backend_num"[[:space:]]*:[[:space:]]*[1-9]' || exit 1
-
-# The BE http port also serves stream load, so check it directly.
-curl -fsS --max-time 4 "http://${HOST}:${BE_HTTP_PORT}/api/health"; >/dev/null
2>&1 || exit 1
-
+[[ -f "${READY_FLAG}" ]] || exit 1
+case "${DORIS_ROLE}" in
+ all)
+ fe_health 127.0.0.1 | grep -qE
'"online_backend_num"[[:space:]]*:[[:space:]]*[1-9]' || exit 1
+ curl -fsS --max-time 4 "http://127.0.0.1:${BE_HTTP_PORT}/api/health";
>/dev/null 2>&1 || exit 1
+ ;;
+ fe) fe_health 127.0.0.1 >/dev/null || exit 1 ;;
+ be) curl -fsS --max-time 4
"http://127.0.0.1:${BE_HTTP_PORT}/api/health"; >/dev/null 2>&1 || exit 1 ;;
+ ms|recycler) curl -fsS --max-time 4 "http://127.0.0.1:${MS_PORT}/health";
>/dev/null 2>&1 || exit 1 ;;
+ client) ;;
+ *) exit 1 ;;
+esac
exit 0
diff --git a/docker/runtime/all-in-one/4.1/resource/lib.sh
b/docker/runtime/all-in-one/4.1/resource/lib.sh
new file mode 100644
index 00000000000..b9a1fba803f
--- /dev/null
+++ b/docker/runtime/all-in-one/4.1/resource/lib.sh
@@ -0,0 +1,148 @@
+#!/usr/bin/env bash
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+#
+# Shared helpers for entrypoint.sh and health_check.sh. Sourced, not run.
+
+DORIS_HOME="${DORIS_HOME:-/opt/apache-doris}"
+FE_HOME="${DORIS_HOME}/fe"
+BE_HOME="${DORIS_HOME}/be"
+MS_HOME="${DORIS_HOME}/ms"
+READY_FLAG="${DORIS_HOME}/.ready"
+
+# What this container is. See the header of entrypoint.sh for the list.
+DORIS_ROLE="${DORIS_ROLE:-all}"
+# local = storage and compute on the BEs; cloud = BEs are compute nodes over a
+# meta-service and an object store.
+DEPLOY_MODE="${DEPLOY_MODE:-local}"
+
+# Ports are the Doris defaults. They only need changing when several nodes
+# share one network namespace, which none of the shipped topologies do.
+FE_HTTP_PORT="${FE_HTTP_PORT:-8030}"
+FE_QUERY_PORT="${FE_QUERY_PORT:-9030}"
+FE_EDIT_LOG_PORT="${FE_EDIT_LOG_PORT:-9010}"
+BE_HTTP_PORT="${BE_HTTP_PORT:-8040}"
+BE_HEARTBEAT_PORT="${BE_HEARTBEAT_PORT:-9050}"
+MS_PORT="${MS_PORT:-5000}"
+
+# The master FE other nodes register with. Empty on an FE means "I am the
+# first FE and bootstrap the cluster". A hostname resolved by docker DNS works
+# as well as an IP.
+FE_MASTER="${FE_MASTER:-}"
+
+START_TIMEOUT="${START_TIMEOUT:-300}"
+STOP_TIMEOUT="${STOP_TIMEOUT:-30}"
+
+log() { printf '%s [%-5s] [%s] %s\n' "$(date -Iseconds)" "$1" "${DORIS_ROLE}"
"${*:2}"; }
+info() { log INFO "$@"; }
+warn() { log WARN "$@" >&2; }
+die() { log ERROR "$@" >&2; exit 1; }
+
+# The address this node is known by. Single-container mode binds everything
+# to loopback; every other role uses the container's first IP, which the
+# compose files pin so that a restarted container keeps its identity.
+my_ip() {
+ if [[ "${DORIS_ROLE}" == all ]]; then
+ echo 127.0.0.1
+ else
+ hostname -i | awk '{print $1}'
+ fi
+}
+
+# sql <host> <statement>: root, no password, batch output, no header.
+sql() {
+ mysql -uroot -h"$1" -P"${FE_QUERY_PORT}" -N --batch --connect-timeout=2 -e
"$2" 2>/dev/null
+}
+
+# The node probes go through SHOW FRONTENDS / SHOW BACKENDS, which FE answers
+# on its own. The frontends() / backends() table functions look handier but
+# are queries, and a query needs a live BE to scan -- which is the very thing
+# being waited for. Columns are picked by header name, not position.
+
+# node_alive <fe host> <frontends|backends> <node ip> <port>: prints the
+# node's Alive column (true/false), nothing when the node is not listed.
+node_alive() {
+ local fe=$1 what=$2 ip=$3 port=$4 portcol
+ case "${what}" in
+ frontends) portcol=EditLogPort ;;
+ backends) portcol=HeartbeatPort ;;
+ esac
+ mysql -uroot -h"${fe}" -P"${FE_QUERY_PORT}" --batch --connect-timeout=2 -e
"show ${what}" 2>/dev/null \
+ | awk -F'\t' -v ip="${ip}" -v port="${port}" -v pc="${portcol}" '
+ NR == 1 { for (i = 1; i <= NF; i++) col[$i] = i; next }
+ $col["Host"] == ip && $col[pc] == port { print $col["Alive"]; exit
}'
+}
+
+# alive_count <fe host> <frontends|backends>: how many nodes report Alive.
+alive_count() {
+ mysql -uroot -h"$1" -P"${FE_QUERY_PORT}" --batch --connect-timeout=2 -e
"show $2" 2>/dev/null \
+ | awk -F'\t' 'NR == 1 { for (i = 1; i <= NF; i++) col[$i] = i; next }
+ $col["Alive"] == "true" { n++ } END { print n + 0 }'
+}
+
+# Public FE endpoint (HealthAction): 503 until FE is ready, otherwise a body
+# carrying online_backend_num. curl -f turns the 503 into a non-zero exit.
+fe_health() {
+ curl -fsS --max-time 4 "http://$1:${FE_HTTP_PORT}/api/health"; 2>/dev/null
+}
+
+# wait_until <seconds> <description> <command...>: poll the command once a
+# second until it succeeds; die when the deadline passes.
+wait_until() {
+ local timeout=$1 what=$2; shift 2
+ local deadline=$((SECONDS + timeout))
+ while ((SECONDS < deadline)); do
+ if "$@"; then return 0; fi
+ sleep 1
+ done
+ die "timed out after ${timeout}s waiting for ${what}"
+}
+
+# render_conf <conf file> <block>: rewrite the file as the shipped copy plus
+# the block, so that a restarted container (same layer, entrypoint run again)
+# does not keep appending. The shipped copy is kept next to it on first run.
+# Doris takes the last assignment of a key, which is what lets the block win
+# over the upstream defaults above it.
+render_conf() {
+ local conf=$1 block=$2
+ [[ -f "${conf}.orig" ]] || cp "${conf}" "${conf}.orig"
+ { cat "${conf}.orig"; printf '\n# --- generated by %s at container start
---\n%s\n' "$(basename "$0")" "${block}"; } >"${conf}"
+}
+
+# set_heap <conf file> <size>: the JVM heap lives inside a long JAVA_OPTS line
+# that also carries every --add-opens FE needs on JDK 17. Rewriting only the
+# -Xmx/-Xms tokens keeps the rest of that line as shipped.
+set_heap() {
+ sed -i -E "s/-Xmx[0-9]+[kKmMgG]/-Xmx$2/g; s/-Xms[0-9]+[kKmMgG]/-Xms$2/g"
"$1"
+}
+
+# Kill a background job by pid, gracefully first. Negative pid signals the
+# whole process group, which is where the real java / doris_be process lives.
+stop_one() {
+ local name=$1 pid=$2
+ [[ -n "${pid}" ]] || return 0
+ kill -0 "${pid}" 2>/dev/null || return 0
+ info "stopping ${name}"
+ kill -TERM -"${pid}" 2>/dev/null || kill -TERM "${pid}" 2>/dev/null || true
+ local deadline=$((SECONDS + STOP_TIMEOUT))
+ while ((SECONDS < deadline)); do
+ kill -0 "${pid}" 2>/dev/null || { info "${name} stopped"; return 0; }
+ sleep 1
+ done
+ warn "${name} did not stop within ${STOP_TIMEOUT}s, killing"
+ kill -KILL -"${pid}" 2>/dev/null || kill -KILL "${pid}" 2>/dev/null || true
+}
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
