This is an automated email from the ASF dual-hosted git repository. klesh pushed a commit to branch feat/single-binary-image in repository https://gitbox.apache.org/repos/asf/devlake.git
commit 9ad70bf49ee3a0edc1e265145100933404011a65 Author: Klesh Wong <[email protected]> AuthorDate: Sat Oct 31 12:24:53 2026 +0800 build(image): ship one binary and keep the .so files out of the image `make build` is now `build-server`: the framework, the API and every plugin in the generated registry link into a single bin/lake, so the runtime stage has no plugin directory to copy. - backend/Dockerfile: copy only /app/bin/lake instead of the whole bin/, and create an empty /app/bin/plugins as the sideload mount point. It is created as root, before the chgrp that makes /app group-writable, because the stage otherwise runs as the devlake user. - Drop `COPY --from=build /go/bin /usr/bin`: mockery and swag are build tools and nothing at runtime uses them. - Dockerfile.local: one build step instead of build-plugin plus build-server, and copy only the binary. - build-server.sh: add -trimpath and -s -w. A DEVLAKE_DEBUG build keeps the symbols so delve still works. - Makefile: build, dev and godev no longer build plugins; build-plugin stays for sideload and core-only builds. - env.example: PLUGIN_DIR now documents the sideload directory. GO_PLUGINS keeps working: it reaches the registry generator, so it now selects which plugins are compiled in rather than which .so files are built. plugins-registry clears GOOS/GOARCH: the Dockerfile exports them to cross-compile the server, and `go run` would otherwise build the generator for the target platform and fail with "exec format error" on an amd64 build host. --- Makefile | 2 +- backend/Dockerfile | 10 +++++++--- backend/Dockerfile.local | 16 ++++++++-------- backend/Makefile | 17 +++++++++++++---- backend/scripts/build-server.sh | 6 +++++- env.example | 5 ++++- 6 files changed, 38 insertions(+), 18 deletions(-) diff --git a/Makefile b/Makefile index 99cd0d95e..cd74395f9 100644 --- a/Makefile +++ b/Makefile @@ -77,7 +77,7 @@ plugins-registry: build-server: make build-server -C backend -build: build-plugin build-server +build: build-server all: build diff --git a/backend/Dockerfile b/backend/Dockerfile index b85a5ec8d..2c31a3d53 100644 --- a/backend/Dockerfile +++ b/backend/Dockerfile @@ -168,18 +168,22 @@ ARG DEBUG= ENV LD_LIBRARY_PATH=/app/libs RUN mkdir -p /app/libs COPY --from=build /usr/local/deps/target/lib/*.so* /app/libs -COPY --from=build /go/bin /usr/bin RUN ldconfig -vn /app/libs # apps -COPY --from=build /app/bin /app/bin +# The single binary already contains every plugin; the .so files that +# `make build-plugin` produces stay in the builder stage and are never copied. +COPY --from=build /app/bin/lake /app/bin/lake COPY --from=build /app/resources /app/resources ENV PATH="/app/bin:${PATH}" ENV DEBUG="$DEBUG" USER root -RUN chgrp -R 0 /app && \ +# Empty mount point for sideloaded plugins: dropping a .so here registers it on +# top of the compiled-in set at startup; see PLUGIN_DIR. +RUN mkdir -p /app/bin/plugins && \ + chgrp -R 0 /app && \ chmod -R g=u /app USER devlake diff --git a/backend/Dockerfile.local b/backend/Dockerfile.local index 8651900d5..e2046db3b 100644 --- a/backend/Dockerfile.local +++ b/backend/Dockerfile.local @@ -72,15 +72,12 @@ RUN --mount=type=cache,target=/root/.cache/go-build \ --mount=type=cache,target=/go/pkg/mod \ make swag -# Build plugins (parallel, with persistent Go build cache) +# Build the server. `build` is `build-server`: the framework, the API and every +# plugin in the generated registry link into this one binary. No .so files are +# produced for the release image. RUN --mount=type=cache,target=/root/.cache/go-build \ --mount=type=cache,target=/go/pkg/mod \ - make build-plugin - -# Build server -RUN --mount=type=cache,target=/root/.cache/go-build \ - --mount=type=cache,target=/go/pkg/mod \ - VERSION=${TAG}@${SHA} make build-server + VERSION=${TAG}@${SHA} make build FROM debian:bookworm-slim @@ -102,7 +99,10 @@ RUN ldconfig WORKDIR /app -COPY --from=builder /app/bin ./bin +COPY --from=builder /app/bin/lake ./bin/lake + +# Empty mount point for sideloaded plugins +RUN mkdir -p bin/plugins # Create empty python plugins dir to avoid startup error RUN mkdir -p python/plugins diff --git a/backend/Makefile b/backend/Makefile index a09f198a2..e8bb9128e 100644 --- a/backend/Makefile +++ b/backend/Makefile @@ -57,8 +57,13 @@ swag: mock # Regenerate plugins/all, the package that links the Go plugins into the server # binary. Run it after adding, removing or renaming a plugin; CI checks that the # checked-in file matches the tree. +# +# GOOS/GOARCH are cleared because `backend/Dockerfile` exports them to +# cross-compile the server: `go run` would otherwise build the generator for the +# target platform and fail with "exec format error" on the build host. Same +# reason as MOCKERY above. plugins-registry: - go run ./tools/pluginsgen + GOOS= GOARCH= CGO_ENABLED=0 go run ./tools/pluginsgen build-plugin: scripts/build-plugins.sh @@ -72,15 +77,19 @@ build-server: plugins-registry swag build-python: #don't mix this with the other build commands scripts/build-python.sh -build: build-plugin build-server +# `build` produces the only artifact the release ships: one binary with the +# framework, the API and every plugin in the generated registry compiled in. +# Use build-plugin when you want shared objects instead, either to sideload +# them into an existing binary or to ship a core-only server. +build: build-server run: go run server/main.go -dev: build-plugin build-python run +dev: build-python run godev: - DISABLED_REMOTE_PLUGINS=true make build-plugin run + DISABLED_REMOTE_PLUGINS=true make run debug: build-plugin-debug dlv debug server/main.go diff --git a/backend/scripts/build-server.sh b/backend/scripts/build-server.sh index b390185c3..479e1fcfa 100755 --- a/backend/scripts/build-server.sh +++ b/backend/scripts/build-server.sh @@ -22,9 +22,13 @@ echo build-server.sh VERSION: $VERSION ROOT_DIR=$(dirname $(dirname "$0")) VERSION=${VERSION:-$(git describe --tags --always --dirty || true)} EXTRA="" +# Strip the symbol table and DWARF from release builds: they roughly halve the +# binary and nothing needs them at runtime. A debug build keeps them for delve. +LDFLAGS="-s -w" if [ -n "$DEVLAKE_DEBUG" ]; then EXTRA="-gcflags='all=-N -l'" + LDFLAGS="" fi -go build $EXTRA -ldflags "-X 'github.com/apache/devlake/core/version.Version=$VERSION'" -o $ROOT_DIR/bin/lake $ROOT_DIR/server/ \ No newline at end of file +go build $EXTRA -trimpath -ldflags "$LDFLAGS -X 'github.com/apache/devlake/core/version.Version=$VERSION'" -o $ROOT_DIR/bin/lake $ROOT_DIR/server/ \ No newline at end of file diff --git a/env.example b/env.example index effca30db..68be8d6d2 100755 --- a/env.example +++ b/env.example @@ -46,7 +46,10 @@ # certificates) on the dashboard container to file paths mounted into it. # GRAFANA_DATABASE_SSL_MODE=disable -# Lake plugin dir, absolute path or relative path +# Lake plugin dir, absolute path or relative path. +# The release binary already contains every official plugin. This directory is +# scanned at startup for sideloaded .so plugins, which are registered on top of +# the compiled-in set. PLUGIN_DIR=bin/plugins REMOTE_PLUGIN_DIR=python/plugins
