numinnex commented on code in PR #3658:
URL: https://github.com/apache/iggy/pull/3658#discussion_r3691119923
##########
core/connectors/runtime/Dockerfile:
##########
@@ -82,28 +92,40 @@ RUN
--mount=type=cache,target=/usr/local/cargo/registry,id=cargo-registry-${TARG
"linux/arm64:glibc") RUST_TARGET="aarch64-unknown-linux-gnu" ;; \
*) echo "Unsupported platform/libc combination: $TARGETPLATFORM/$LIBC" &&
exit 1 ;; \
esac && \
- if [ "$PROFILE" = "debug" ]; then \
- cargo zigbuild --locked --target ${RUST_TARGET} --bin iggy-connectors && \
- cp /app/target/${RUST_TARGET}/debug/iggy-connectors /app/iggy-connectors; \
- else \
- cargo zigbuild --locked --target ${RUST_TARGET} --bin iggy-connectors
--release && \
- cp /app/target/${RUST_TARGET}/release/iggy-connectors
/app/iggy-connectors; \
- fi
+ PLUGIN_FLAGS="$(sed 's/^/-p /' connector-plugins.txt | tr '\n' ' ')" && \
+ if [ "$PROFILE" = "debug" ]; then PROFILE_DIR=debug; PROFILE_FLAG=""; else
PROFILE_DIR=release; PROFILE_FLAG="--release"; fi && \
+ cargo zigbuild --locked --target ${RUST_TARGET} -p iggy-connectors
$PLUGIN_FLAGS $PROFILE_FLAG && \
+ cp /app/target/${RUST_TARGET}/${PROFILE_DIR}/iggy-connectors
/app/iggy-connectors && \
+ mkdir -p /app/plugins && \
+ while IFS= read -r name; do \
+ [ -z "$name" ] && continue; \
+ so_file="/app/target/${RUST_TARGET}/${PROFILE_DIR}/lib${name}.so"; \
+ if [ ! -f "$so_file" ]; then echo "expected plugin artifact missing:
$so_file" >&2 && exit 1; fi; \
+ cp "$so_file" /app/plugins/; \
+ done < connector-plugins.txt
Review Comment:
The plugin compilation lives in the shared `builder` stage, so the slim
flavor pays for it too.
`runtime` (slim) copies `/app/iggy-connectors` and `/app/LICENSE-binary` out
of `builder`. Both of those are produced by RUN layers that sit at or after
this one, so `--target runtime` still executes this `cargo zigbuild -p
iggy-connectors $PLUGIN_FLAGS` (17 cdylibs) and the license RUN below that also
generates `LICENSE-binary-fat`. BuildKit cannot skip a layer that an earlier
needed artifact depends on.
Net effect: the `-slim` build goes from a runtime-binary-only compile to
roughly the full fat build time (~20 min per the PR description), plus a
cargo-about pass over 18 dependency closures, and then discards all of it. With
the matrix at flavors x arches that is two extra full plugin compiles per
publish run.
Suggestion: keep `builder` producing only the runtime binary and
`LICENSE-binary`, then add a stage that only the fat image consumes:
```dockerfile
FROM builder AS plugin-builder
# cargo zigbuild ... $PLUGIN_FLAGS, collect /app/plugins, generate
LICENSE-binary-fat
FROM runtime AS runtime-fat
COPY --from=plugin-builder /app/plugins/ /usr/local/lib/
COPY --from=plugin-builder /app/LICENSE-binary-fat
/usr/share/doc/iggy-connect/LICENSE-binary
```
`--target runtime` then stops at `builder` and the slim image build is
unchanged from today.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
