[
https://issues.apache.org/jira/browse/CAMEL-24646?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18121028#comment-18121028
]
Torsten Mielke commented on CAMEL-24646:
----------------------------------------
[~apupier] may I ask you to review the current analysis? I am still not really
keen on the proposed solution, but have not found a better fix yet...
Any opinion on your side about the fix?
Analysis so far...
*Problem*
The Camel JBang IT tests fail with a {{ContainerFetchException}} when running
via Maven ({{{}mvn verify -Djbang-it-test{}}}) because the Docker image build
cannot resolve SNAPSHOT artifacts.
The Dockerfile executes {{jbang app install -Dcamel.jbang.version=<SNAPSHOT>}}
during docker build, which triggers JBang's dependency resolution for
{{camel-jbang-core}} and its transitives. JBang resolves these from Maven
Central and Apache Snapshots, but unpublished SNAPSHOT versions (common on
maintenance branches like camel-4.22.x, and during local development on main)
are not available in those remote repositories. The artifacts exist in the
developer's local Maven repository ({{{}~/.m2/repository{}}}), but that
repository is only bind-mounted at *container runtime* via
{{withFileSystemBind()}} in {{CliBuiltContainer.java}} — it is not accessible
during the docker build phase. Since all 34 IT test classes share a singleton
container, a failed Docker build causes every single IT test to fail.
The partial fix in commit {{6b303da9ee78}} (correcting the branch reference
from main to camel-4.22.x) resolves IDE-based runs where the version defaults
to "default" (using the released version), but Maven-based runs still fail
because {{cli.jbang.version}} resolves to the SNAPSHOT version.
*Suggested fix*
The proposed fix keeps jbang app install at Docker build time (preserving the
current architecture and avoiding a ~30-60 second runtime penalty per container
startup) by making SNAPSHOT artifacts available during the build. During the
pre-integration-test phase, the existing {{maven-antrun-plugin}} execution in
{{camel-jbang-it/pom.xml }}would be extended to copy a version-filtered subset
of the local Maven repo (*{{{}{*}/${project.version}/{*}*{}}} under
{{{}org/apache/camel/{}}}) into {{{}target/docker-mvn-repo/{}}}. This staged
directory is then added to the Docker build context via Testcontainers'
{{withFileFromPath()}} in {{{}CliBuiltContainer.java{}}}, and a {{COPY m2repo/
/home/jbang/.m2/repository/org/apache/camel/}} instruction in the Dockerfile
pre-populates the container's Maven repo *before* {{jbang app install}} runs.
The version filter keeps the copy targeted to the current build version only
(typically 150-300 MB for a targeted build), rather than copying the entire
local Maven repo. This approach requires changes to only 3 files:
{{{}camel-jbang-it/pom.xml, CliBuiltContainer.java{}}}, and the
{{{}Dockerfile{}}}.
*Discarded alternatives*
We considered three alternative approaches before settling on the antrun-based
solution.
* First, deferring {{jbang app install }}from the {{Dockerfile}} to the
container's {{entrypoint.sh}} — at container runtime the local Maven repo is
already bind-mounted, so JBang would find the SNAPSHOT artifacts. We discarded
this because it would add ~30-60 seconds of dependency resolution overhead to
every container startup, whereas the current build-time approach pays that cost
once and caches it in the Docker image layer.
* Second, running a temporary HTTP server during the Docker build to serve
Maven artifacts from the host — this was discarded due to excessive complexity
(lifecycle management, port allocation, configuring JBang to use the ad-hoc
repo URL).
* Third, using {{maven-dependency-plugin:copy-dependencies }}to copy artifacts
in a Maven-native way — this was investigated but found to be blocked by the 13
exclusions declared on the {{camel-jbang-core}} dependency in the IT POM (lines
108-173). These exclusions strip critical transitive dependencies like
{{{}camel-main, camel-kamelet-main{}}}, and {{camel-cli-connector}} from the
resolved dependency tree, and copy-dependencies operates on that pruned tree,
so it would produce an incomplete set of artifacts. The {{maven-antrun-plugin}}
filesystem copy bypasses Maven's dependency resolution entirely, avoiding this
problem.
> Camel JBang 4.22.x IT tests cannot build the Docker image
> ---------------------------------------------------------
>
> Key: CAMEL-24646
> URL: https://issues.apache.org/jira/browse/CAMEL-24646
> Project: Camel
> Issue Type: Test
> Components: camel-jbang
> Affects Versions: 4.22.0
> Reporter: Aurélien Pupier
> Assignee: Torsten Mielke
> Priority: Major
>
> half of the tests are failing with this kind of error:
> {noformat}
> org.testcontainers.containers.ContainerFetchException: Can't get Docker
> image: RemoteDockerImage(imageName=<resolving>,
> imagePullPolicy=DefaultPullPolicy(),
> imageNameSubstitutor=org.testcontainers.utility.ImageNameSubstitutor$LogWrappedImageNameSubstitutor@235d29d6)
> at
> org.testcontainers.containers.GenericContainer.getDockerImageName(GenericContainer.java:1308)
> at
> org.testcontainers.containers.GenericContainer.doStart(GenericContainer.java:346)
> at
> org.testcontainers.containers.GenericContainer.start(GenericContainer.java:317)
> at
> org.apache.camel.test.infra.cli.services.CliLocalContainerService.initialize(CliLocalContainerService.java:79)
> at
> org.apache.camel.test.infra.common.services.TestServiceUtil.tryInitialize(TestServiceUtil.java:54)
> at
> org.apache.camel.test.infra.common.services.TestService.beforeAll(TestService.java:28)
> at java.base/java.util.ArrayList.forEach(ArrayList.java:1604)
> Caused by: com.github.dockerjava.api.exception.DockerClientException: Could
> not build image: The command '/bin/sh -c source ~/.bashrc && if [[
> "$CAMEL_JBANG_VERSION" == "default" ]] ; then jbang app install
> camel@$CAMEL_REPO/$CAMEL_REF ; else jbang app install
> -Dcamel.jbang.version=$CAMEL_JBANG_VERSION camel@$CAMEL_REPO/$CAMEL_REF ;
> fi' returned a non-zero code: 1
> at
> com.github.dockerjava.api.command.BuildImageResultCallback.getImageId(BuildImageResultCallback.java:71)
> at
> com.github.dockerjava.api.command.BuildImageResultCallback.awaitImageId(BuildImageResultCallback.java:50)
> at
> org.testcontainers.images.builder.ImageFromDockerfile.resolve(ImageFromDockerfile.java:165)
> at
> org.testcontainers.images.builder.ImageFromDockerfile.resolve(ImageFromDockerfile.java:43)
> at
> org.testcontainers.utility.LazyFuture.getResolvedValue(LazyFuture.java:20)
> at org.testcontainers.utility.LazyFuture.get(LazyFuture.java:41)
> at
> org.testcontainers.shaded.com.google.common.util.concurrent.Futures$1.get(Futures.java:538)
> at
> org.testcontainers.images.RemoteDockerImage.getImageName(RemoteDockerImage.java:172)
> at
> org.testcontainers.images.RemoteDockerImage.resolve(RemoteDockerImage.java:76)
> at
> org.testcontainers.images.RemoteDockerImage.resolve(RemoteDockerImage.java:35)
> at
> org.testcontainers.utility.LazyFuture.getResolvedValue(LazyFuture.java:20)
> at org.testcontainers.utility.LazyFuture.get(LazyFuture.java:41)
> at
> org.testcontainers.containers.GenericContainer.getDockerImageName(GenericContainer.java:1306)
> ... 6 more
> {noformat}
> it doesn't find jbang-core 4.22.1-SNAPSHOT
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
