This is an automated email from the ASF dual-hosted git repository.
apupier pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/camel.git
The following commit(s) were added to refs/heads/main by this push:
new 80bc80e966ea CAMEL-27029: camel-launcher - speed up the tests of the
module
80bc80e966ea is described below
commit 80bc80e966ea611347cf3917b1261bfd852c080a
Author: Aurélien Pupier <[email protected]>
AuthorDate: Mon Nov 16 12:20:17 2026 +0100
CAMEL-27029: camel-launcher - speed up the tests of the module
camel-launcher is a leaf module on the critical path of the build: it
waits for camel-jbang-core and all the pre-installed plugins. Its tests
take about 30s of the 43s of a full `mvn install` on 22 cores (and about
45s of 54s on 4 cores). Almost all of it is spent spawning processes, not
running launcher code. Two test-only changes remove most of it:
WebsiteManifestGeneratorTest
Every test launched the generator as a source file (`java
WebsiteManifestGenerator.java`), which recompiles it in memory each time:
about 0.45s wall and 1.8s CPU per launch, 51 launches in the class. The
generator is now compiled once in a @BeforeAll with the JDK compiler and
launched with -cp and -XX:TieredStopAtLevel=1 (about 0.03s). The
compile uses -proc:none: the generator only uses JDK classes, so no
annotation processor of the test class path (log4j-core registers one)
must run. One test keeps the source-file launch, as it is the way
camel-package.sh runs the generator. The class goes from 23.7s to 3.4s when
run alone.
WebsiteInstallTest.PosixShell
The 86 tests ran one after the other, each running install.sh against its
own local HTTPS server. They already share no state: every test has its
own @TempDir, HOME and server on an ephemeral port, the TLS certificate
is created once behind a lock, and nothing sets system properties. The
nested class is now @Execution(ExecutionMode.CONCURRENT), on top of the
class-level parallelism the module already enables. It goes from 17s to
3s on 22 cores and from 15.8s to 5.1s on 4 cores, 4 runs per core count
without a failure. The Windows PowerShell nested class is left as is: its
@AfterEach restores the user PATH in the registry, which is shared state.
Measured on the module with `mvn install -o` (22 and 4 cores selected
with taskset, no `camel` on the PATH, 418 tests, 47 skipped on Linux):
cores before after
22 38.3-40.7s ~22s
4 53.7-54.1s ~26s
Not changed on purpose: SelfUpdateIntegrationTest sets the user.home and
camel.launcher.jar system properties, and PackagePlanTest shares fixed
files under target/ that its @AfterEach deletes, so neither can run its
tests concurrently without a rework. They are the remaining 9s and 6s.
Co-authored-by: IBM Bob 2.2.1
Signed-off-by: Aurélien Pupier <[email protected]>
---
.../dsl/jbang/launcher/WebsiteInstallTest.java | 3 ++
.../launcher/WebsiteManifestGeneratorTest.java | 33 ++++++++++++++++++++--
2 files changed, 34 insertions(+), 2 deletions(-)
diff --git
a/dsl/camel-jbang/camel-launcher/src/test/java/org/apache/camel/dsl/jbang/launcher/WebsiteInstallTest.java
b/dsl/camel-jbang/camel-launcher/src/test/java/org/apache/camel/dsl/jbang/launcher/WebsiteInstallTest.java
index ec12d8a4afd7..c028e1f6ed3c 100644
---
a/dsl/camel-jbang/camel-launcher/src/test/java/org/apache/camel/dsl/jbang/launcher/WebsiteInstallTest.java
+++
b/dsl/camel-jbang/camel-launcher/src/test/java/org/apache/camel/dsl/jbang/launcher/WebsiteInstallTest.java
@@ -49,6 +49,8 @@ import org.junit.jupiter.api.condition.DisabledOnOs;
import org.junit.jupiter.api.condition.EnabledOnOs;
import org.junit.jupiter.api.condition.OS;
import org.junit.jupiter.api.io.TempDir;
+import org.junit.jupiter.api.parallel.Execution;
+import org.junit.jupiter.api.parallel.ExecutionMode;
import org.junit.jupiter.params.ParameterizedTest;
import org.junit.jupiter.params.provider.Arguments;
import org.junit.jupiter.params.provider.MethodSource;
@@ -223,6 +225,7 @@ class WebsiteInstallTest {
@Nested
@DisabledOnOs(OS.WINDOWS)
+ @Execution(ExecutionMode.CONCURRENT)
class PosixShell {
private static final Path SCRIPT =
Paths.get("src/install/install.sh").toAbsolutePath();
diff --git
a/dsl/camel-jbang/camel-launcher/src/test/java/org/apache/camel/dsl/jbang/launcher/WebsiteManifestGeneratorTest.java
b/dsl/camel-jbang/camel-launcher/src/test/java/org/apache/camel/dsl/jbang/launcher/WebsiteManifestGeneratorTest.java
index b4d6dc43b3e0..41d489ffe8dc 100644
---
a/dsl/camel-jbang/camel-launcher/src/test/java/org/apache/camel/dsl/jbang/launcher/WebsiteManifestGeneratorTest.java
+++
b/dsl/camel-jbang/camel-launcher/src/test/java/org/apache/camel/dsl/jbang/launcher/WebsiteManifestGeneratorTest.java
@@ -30,6 +30,10 @@ import java.util.HexFormat;
import java.util.List;
import java.util.concurrent.TimeUnit;
+import javax.tools.JavaCompiler;
+import javax.tools.ToolProvider;
+
+import org.junit.jupiter.api.BeforeAll;
import org.junit.jupiter.api.Test;
import org.junit.jupiter.api.io.TempDir;
@@ -48,6 +52,11 @@ class WebsiteManifestGeneratorTest {
private static final Path GENERATOR =
Paths.get("src/jreleaser/java/WebsiteManifestGenerator.java");
+ // The generator compiled once for the whole class: launching it as a
source file (`java X.java`) recompiles it in
+ // memory on every call, which costs ~0.5s wall and ~2s CPU per test,
against ~0.03s for the compiled class.
+ @TempDir
+ static Path generatorClasses;
+
// Must stay byte-identical to WebsiteManifestGenerator.LICENSE_HEADER:
the generator now prepends
// this ASF header to every manifest, and the assertions below compare the
output byte-for-byte.
// Package-private so PackagePlanTest (which runs the generator via
camel-package.sh) can reuse it.
@@ -75,11 +84,31 @@ class WebsiteManifestGeneratorTest {
String stderr;
}
+ @BeforeAll
+ static void compileGenerator() {
+ JavaCompiler compiler = ToolProvider.getSystemJavaCompiler();
+ // -proc:none: the generator only uses JDK classes, so do not run the
annotation processors javac would
+ // otherwise discover on the test class path
+ assertEquals(0,
+ compiler.run(null, null, null, "-proc:none", "-d",
generatorClasses.toString(), GENERATOR.toString()),
+ "the generator did not compile");
+ }
+
private Result run(String... args) throws Exception {
+ return launch(List.of("-XX:TieredStopAtLevel=1", "-cp",
generatorClasses.toString(), "WebsiteManifestGenerator"),
+ args);
+ }
+
+ // The way camel-package.sh runs the generator: keep at least one test on
this path.
+ private Result runFromSource(String... args) throws Exception {
+ return launch(List.of(GENERATOR.toString()), args);
+ }
+
+ private Result launch(List<String> javaArgs, String... args) throws
Exception {
String javaBin = Paths.get(System.getProperty("java.home"), "bin",
"java").toString();
List<String> cmd = new ArrayList<>();
cmd.add(javaBin);
- cmd.add(GENERATOR.toString());
+ cmd.addAll(javaArgs);
Collections.addAll(cmd, args);
Process p = new ProcessBuilder(cmd).start();
String out = new String(p.getInputStream().readAllBytes(),
StandardCharsets.UTF_8);
@@ -114,7 +143,7 @@ class WebsiteManifestGeneratorTest {
Path zip = writeFixture(work, "camel-launcher-4.22.0-bin.zip",
"zip-content-A");
Path output = work.resolve("website");
- Result r = run("--version", "4.22.0", "--tar", tar.toString(),
"--zip", zip.toString(), "--output",
+ Result r = runFromSource("--version", "4.22.0", "--tar",
tar.toString(), "--zip", zip.toString(), "--output",
output.toString(), "--latest", "true");
assertEquals(0, r.exit, r.stderr);