This is an automated email from the ASF dual-hosted git repository.
davsclaus pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/camel.git
The following commit(s) were added to refs/heads/main by this push:
new f485348ae0b1 CAMEL-26901: camel-tooling-maven - a Maven repository
that does not answer fails after 60 seconds, and one that cannot be connected
to is not asked again (#29781)
f485348ae0b1 is described below
commit f485348ae0b14475ab18e50764dbc53d9a2c4ce0
Author: Claus Ibsen <[email protected]>
AuthorDate: Sat Nov 14 09:42:59 2026 +0100
CAMEL-26901: camel-tooling-maven - a Maven repository that does not answer
fails after 60 seconds, and one that cannot be connected to is not asked again
(#29781)
- The request timeout is 60 seconds (Maven Resolver's default is 30
minutes), so a repository that stops sending
no longer hangs camel run for that long. The
aether.connector.requestTimeout system property still wins.
- A repository that cannot be connected to (unknown host, connection
refused, connect timeout of HttpClient 4,
HttpClient 5 or the JDK HTTP client) is logged once with a WARN and left
out of the next resolutions, instead of
costing the connect timeout for every POM and JAR. A resolution failure
lists it as "(not reachable)".
camel run with two missing dependencies and an unreachable repository: 23s
instead of 44s.
Signed-off-by: Claus Ibsen <[email protected]>
Co-Authored-By: Claude Opus 5.5 <[email protected]>
---
.../ROOT/pages/camel-4x-upgrade-guide-4_24.adoc | 8 ++
.../modules/ROOT/pages/camel-jbang-running.adoc | 11 +++
.../camel/tooling/maven/MavenDownloaderImpl.java | 110 ++++++++++++++++++++-
.../tooling/maven/MavenDownloaderImplTest.java | 105 ++++++++++++++++++++
4 files changed, 232 insertions(+), 2 deletions(-)
diff --git
a/docs/user-manual/modules/ROOT/pages/camel-4x-upgrade-guide-4_24.adoc
b/docs/user-manual/modules/ROOT/pages/camel-4x-upgrade-guide-4_24.adoc
index f2f9f06296cf..87f0af8119a7 100644
--- a/docs/user-manual/modules/ROOT/pages/camel-4x-upgrade-guide-4_24.adoc
+++ b/docs/user-manual/modules/ROOT/pages/camel-4x-upgrade-guide-4_24.adoc
@@ -2666,6 +2666,14 @@ All commands find running integrations the same way:
* The `--timeout` of `camel cmd send`, `camel cmd receive`, `camel eval
expression`, `camel transform message` and the
`camel semantic` commands also takes a duration such as `30s` (a number is
still millis).
+=== camel-jbang - a Maven repository that does not answer fails the download
after 60 seconds
+
+The Camel CLI (`camel run`, `camel export`, the CLI plugins, and other tools
that download with `camel-tooling-maven`) now
+waits 60 seconds for data from a Maven repository, instead of the 30 minutes
of Maven Resolver, so a repository that
+stops sending no longer hangs the download for that long. Set the
`aether.connector.requestTimeout` system property
+(in millis) for a longer timeout. A repository that cannot be connected to is
now logged with a WARN and not used
+for the rest of the run.
+
=== camel-pqc - KeyLifecycleManager behaviour aligned across the
implementations
Four differences between the `KeyLifecycleManager` implementations are
resolved. All four are corrections, but each
diff --git a/docs/user-manual/modules/ROOT/pages/camel-jbang-running.adoc
b/docs/user-manual/modules/ROOT/pages/camel-jbang-running.adoc
index 9f8d664c80e3..c299857ed441 100644
--- a/docs/user-manual/modules/ROOT/pages/camel-jbang-running.adoc
+++ b/docs/user-manual/modules/ROOT/pages/camel-jbang-running.adoc
@@ -359,6 +359,17 @@ To disable automatic downloading:
camel run foo.java --download=false
----
+A repository that cannot be connected to (the host does not resolve, refuses
the connection, or does not answer
+within the connect timeout of 10 seconds) is logged once with a WARN and not
used for the rest of the run, so each
+download does not wait for it again. A repository that stops sending data
fails the download after 60 seconds.
+Set the `aether.connector.connectTimeout` and
`aether.connector.requestTimeout` system properties (in millis) to
+change the timeouts:
+
+[source,bash]
+----
+export JAVA_TOOL_OPTIONS="-Daether.connector.requestTimeout=180000"
+----
+
[[advanced-class-to-dependency]]
=== Advanced: how a class becomes a dependency
diff --git
a/tooling/camel-tooling-maven/src/main/java/org/apache/camel/tooling/maven/MavenDownloaderImpl.java
b/tooling/camel-tooling-maven/src/main/java/org/apache/camel/tooling/maven/MavenDownloaderImpl.java
index 10cc259193c8..13e0141c04ee 100644
---
a/tooling/camel-tooling-maven/src/main/java/org/apache/camel/tooling/maven/MavenDownloaderImpl.java
+++
b/tooling/camel-tooling-maven/src/main/java/org/apache/camel/tooling/maven/MavenDownloaderImpl.java
@@ -19,9 +19,12 @@ package org.apache.camel.tooling.maven;
import java.io.BufferedInputStream;
import java.io.File;
import java.io.InputStream;
+import java.net.ConnectException;
import java.net.MalformedURLException;
+import java.net.SocketTimeoutException;
import java.net.URI;
import java.net.URL;
+import java.net.UnknownHostException;
import java.nio.file.Files;
import java.util.ArrayList;
import java.util.Arrays;
@@ -34,6 +37,7 @@ import java.util.List;
import java.util.Map;
import java.util.Objects;
import java.util.Set;
+import java.util.concurrent.ConcurrentHashMap;
import java.util.concurrent.atomic.AtomicInteger;
import java.util.function.Predicate;
import java.util.stream.Collectors;
@@ -84,9 +88,12 @@ import
org.eclipse.aether.resolution.DependencyResolutionException;
import org.eclipse.aether.resolution.DependencyResult;
import org.eclipse.aether.resolution.MetadataRequest;
import org.eclipse.aether.resolution.MetadataResult;
+import org.eclipse.aether.transfer.AbstractTransferListener;
import org.eclipse.aether.transfer.MetadataNotFoundException;
import org.eclipse.aether.transfer.RepositoryOfflineException;
+import org.eclipse.aether.transfer.TransferEvent;
import org.eclipse.aether.util.graph.selector.AndDependencySelector;
+import org.eclipse.aether.util.listener.ChainedTransferListener;
import org.eclipse.aether.util.repository.SimpleArtifactDescriptorPolicy;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
@@ -116,6 +123,7 @@ public class MavenDownloaderImpl extends ServiceSupport
implements MavenDownload
// see org.eclipse.aether.internal.impl.DefaultUpdateCheckManager
private static final String UPDATE_CHECK_SESSION_STATE =
"aether.updateCheckManager.sessionState";
private static final DependencyFilter ACCEPT_ALL = (node, parents) -> true;
+ private static final int DEFAULT_REQUEST_TIMEOUT = 60000;
// Configuration
private boolean mavenCentralEnabled = true;
@@ -127,6 +135,8 @@ public class MavenDownloaderImpl extends ServiceSupport
implements MavenDownload
private boolean fresh;
private boolean offline;
private boolean preferLocal = true;
+ // the resolver's default is 30 minutes, so a repository that stops
sending would hang the download for that long
+ private int requestTimeout = DEFAULT_REQUEST_TIMEOUT;
private RemoteArtifactDownloadListener remoteArtifactDownloadListener;
private RepositoryResolver repositoryResolver;
@@ -151,6 +161,8 @@ public class MavenDownloaderImpl extends ServiceSupport
implements MavenDownload
private RepositoryPolicy defaultPolicy;
private boolean apacheSnapshotsIncluded;
private AtomicInteger customRepositoryCounter = new AtomicInteger(1);
+ // the repositories that could not be connected to (by URL): they are not
used for the rest of the run
+ private Set<String> unreachableRepositories =
ConcurrentHashMap.newKeySet();
/**
* Default constructor for standalone mode. MIMA will be used to create
RepositorySystem and
@@ -361,7 +373,7 @@ public class MavenDownloaderImpl extends ServiceSupport
implements MavenDownload
// Get RepositorySystem and RepositorySystemSession from MIMA
repositorySystem = mimaContext.repositorySystem();
- repositorySystemSession =
withTrackingKeyCompat(mimaContext.repositorySystemSession());
+ repositorySystemSession =
withTimeoutsAndReachability(withTrackingKeyCompat(mimaContext.repositorySystemSession()));
// Configure repositories (same as embedded mode)
List<RemoteRepository> originalRepositories = new ArrayList<>();
@@ -817,7 +829,12 @@ public class MavenDownloaderImpl extends ServiceSupport
implements MavenDownload
} catch (DependencyResolutionException | ArtifactResolutionException
e) {
MavenResolutionException mre = new
MavenResolutionException(e.getMessage(), e);
- repositories.forEach(r -> mre.getRepositories().add(r.getUrl()));
+ Set<String> unreachable = new
LinkedHashSet<>(unreachableRepositories);
+ for (RemoteRepository r : repositories) {
+ boolean reachable =
!unreachable.remove(repositoryUrl(r.getUrl()));
+ mre.getRepositories().add(reachable ? r.getUrl() : r.getUrl()
+ " (not reachable)");
+ }
+ unreachable.forEach(url -> mre.getRepositories().add(url + " (not
reachable)"));
throw mre;
}
}
@@ -840,9 +857,97 @@ public class MavenDownloaderImpl extends ServiceSupport
implements MavenDownload
session, extraRepos);
repositories.addAll(resolvedExtraRepos);
}
+ if (!unreachableRepositories.isEmpty()) {
+ // asking them again would only wait for the connect timeout again
+ repositories.removeIf(r ->
unreachableRepositories.contains(repositoryUrl(r.getUrl())));
+ }
return repositories;
}
+ /**
+ * Sets the request timeout of Camel (unless configured, such as with the
aether.connector.requestTimeout system
+ * property), and stops using a repository that cannot be connected to.
+ */
+ private RepositorySystemSession
withTimeoutsAndReachability(RepositorySystemSession session) {
+ DefaultRepositorySystemSession answer = new
DefaultRepositorySystemSession(session);
+ if
(session.getConfigProperties().get(ConfigurationProperties.REQUEST_TIMEOUT) ==
null) {
+ answer.setConfigProperty(ConfigurationProperties.REQUEST_TIMEOUT,
requestTimeout);
+ }
+
answer.setTransferListener(ChainedTransferListener.newInstance(session.getTransferListener(),
+ new UnreachableRepositoryListener(unreachableRepositories)));
+ return answer;
+ }
+
+ /**
+ * Notes the repositories that cannot be connected to: a host that does
not resolve, refuses the connection, or does
+ * not answer within the connect timeout. A repository that is slow to
send an artifact is still used.
+ */
+ static final class UnreachableRepositoryListener extends
AbstractTransferListener {
+
+ private final Set<String> unreachable;
+
+ UnreachableRepositoryListener(Set<String> unreachable) {
+ this.unreachable = unreachable;
+ }
+
+ @Override
+ public void transferFailed(TransferEvent event) {
+ Throwable cause = connectFailure(event.getException());
+ if (cause != null && event.getResource() != null &&
event.getResource().getRepositoryUrl() != null) {
+ String url =
repositoryUrl(event.getResource().getRepositoryUrl());
+ if (unreachable.add(url)) {
+ LOG.warn("Cannot connect to Maven repository: {} due to:
{}. The repository is not used anymore.",
+ url, cause.getMessage() != null ?
cause.getMessage() : cause.getClass().getName());
+ }
+ }
+ }
+
+ static Throwable connectFailure(Throwable e) {
+ // guard against a cause that points back to an exception in the
chain
+ int depth = 0;
+ for (Throwable t = e; t != null && depth < 20; t = t.getCause(),
depth++) {
+ if (t instanceof ConnectException || t instanceof
UnknownHostException) {
+ return t;
+ }
+ // the connect timeout of the JDK socket
+ if (t instanceof SocketTimeoutException && "Connect timed
out".equalsIgnoreCase(t.getMessage())) {
+ return t;
+ }
+ // the connect timeout of an HTTP client, by name to need none
of them on the classpath:
+ // org.apache.http.conn.ConnectTimeoutException (HttpClient 4,
the transport of Maven Resolver 1.x),
+ // org.apache.hc.client5.http.ConnectTimeoutException
(HttpClient 5), whose message is not
+ // "Connect timed out", and
java.net.http.HttpConnectTimeoutException (the JDK HTTP client)
+ if
(t.getClass().getSimpleName().endsWith("ConnectTimeoutException")) {
+ return t;
+ }
+ }
+ return null;
+ }
+ }
+
+ private static String repositoryUrl(String url) {
+ return url.endsWith("/") ? url.substring(0, url.length() - 1) : url;
+ }
+
+ /**
+ * The request timeout in millis: how long to wait for data from a Maven
repository before failing (default 60
+ * seconds). The aether.connector.requestTimeout system property overrides
it.
+ */
+ public void setRequestTimeout(int requestTimeout) {
+ this.requestTimeout = requestTimeout;
+ }
+
+ public int getRequestTimeout() {
+ return requestTimeout;
+ }
+
+ /**
+ * The repositories that could not be connected to (by URL), which are not
used anymore.
+ */
+ Set<String> getUnreachableRepositories() {
+ return unreachableRepositories;
+ }
+
@Override
public MavenArtifactDescriptor resolveArtifactDescriptor(String gav,
Set<String> extraRepositories)
throws MavenResolutionException {
@@ -1127,6 +1232,7 @@ public class MavenDownloaderImpl extends ServiceSupport
implements MavenDownload
copy.fresh = fresh;
copy.apacheSnapshotsIncluded = apacheSnapshotsIncluded;
copy.customRepositoryCounter = customRepositoryCounter;
+ copy.unreachableRepositories = unreachableRepositories;
copy.repositoryResolver = repositoryResolver;
copy.offline = offline;
diff --git
a/tooling/camel-tooling-maven/src/test/java/org/apache/camel/tooling/maven/MavenDownloaderImplTest.java
b/tooling/camel-tooling-maven/src/test/java/org/apache/camel/tooling/maven/MavenDownloaderImplTest.java
index 512aef05d888..11e6e486c946 100644
---
a/tooling/camel-tooling-maven/src/test/java/org/apache/camel/tooling/maven/MavenDownloaderImplTest.java
+++
b/tooling/camel-tooling-maven/src/test/java/org/apache/camel/tooling/maven/MavenDownloaderImplTest.java
@@ -17,6 +17,13 @@
package org.apache.camel.tooling.maven;
import java.io.File;
+import java.io.IOException;
+import java.net.ConnectException;
+import java.net.ServerSocket;
+import java.net.Socket;
+import java.net.SocketTimeoutException;
+import java.net.UnknownHostException;
+import java.net.http.HttpConnectTimeoutException;
import java.nio.charset.StandardCharsets;
import java.nio.file.Files;
import java.security.MessageDigest;
@@ -24,11 +31,13 @@ import java.util.Base64;
import java.util.List;
import java.util.Set;
import java.util.concurrent.Callable;
+import java.util.concurrent.CopyOnWriteArrayList;
import java.util.concurrent.TimeUnit;
import org.apache.camel.util.FileUtil;
import org.apache.commons.codec.binary.Hex;
import org.apache.http.Header;
+import org.apache.http.conn.ConnectTimeoutException;
import org.apache.http.entity.StringEntity;
import org.apache.http.impl.bootstrap.HttpServer;
import org.apache.http.impl.bootstrap.ServerBootstrap;
@@ -47,6 +56,7 @@ import static org.junit.jupiter.api.Assertions.assertEquals;
import static org.junit.jupiter.api.Assertions.assertFalse;
import static org.junit.jupiter.api.Assertions.assertNotNull;
import static org.junit.jupiter.api.Assertions.assertNull;
+import static org.junit.jupiter.api.Assertions.assertThrows;
import static org.junit.jupiter.api.Assertions.assertTrue;
import static org.junit.jupiter.api.Assertions.fail;
@@ -1146,4 +1156,99 @@ class MavenDownloaderImplTest {
""".formatted(artifactId, body);
Files.writeString(new File(dir, artifactId + "-1.0.0.pom").toPath(),
pom);
}
+
+ @Test
+ void testUnreachableRepositoryIsNotUsedAnymore() throws Exception {
+ File repo = createDescriptorFixtureRepository();
+ File localRepo = new File(tempDir, "unreachable-local-repo");
+ // nothing listens on this port, so the connection is refused
+ String dead = "http://localhost:"; +
AvailablePortFinder.getNextAvailable(20000, 30000) + "/maven";
+
+ try (MavenDownloaderImpl downloader = new MavenDownloaderImpl()) {
+ downloader.setMavenCentralEnabled(false);
+ downloader.setMavenApacheSnapshotEnabled(false);
+ downloader.setMavenSettingsLocation("false");
+ downloader.setRepos(dead + "," + repo.toURI());
+ downloader.build();
+ MavenDownloader customDownloader =
downloader.customize(localRepo.getAbsolutePath(), 5000, 5000);
+
+ // an artifact no repository has: both are asked
+ assertThrows(MavenResolutionException.class,
+ () ->
customDownloader.resolveArtifacts(List.of("org.example.fixture:missing:1.0.0"),
null, false,
+ false));
+ assertEquals(Set.of(dead),
downloader.getUnreachableRepositories());
+
+ // the other repository is still used
+ List<MavenArtifact> artifacts = customDownloader.resolveArtifacts(
+ List.of("org.example.fixture:fixture-bom:pom:1.0.0"),
null, false, false);
+ assertEquals(1, artifacts.size());
+
+ // the repository is not asked again, and the failure says so
+ MavenResolutionException e =
assertThrows(MavenResolutionException.class,
+ () ->
customDownloader.resolveArtifacts(List.of("org.example.fixture:missing2:1.0.0"),
null, false,
+ false));
+ assertTrue(e.getRepositories().contains(dead + " (not
reachable)"), e.getRepositories().toString());
+ assertFalse(e.getRepositories().contains(dead),
e.getRepositories().toString());
+ }
+ }
+
+ @Test
+ void testRepositoryThatDoesNotAnswerFailsOnTheRequestTimeout() throws
Exception {
+ File localRepo = new File(tempDir, "stalled-local-repo");
+ // accepts the connection, and never answers
+ try (ServerSocket server = new ServerSocket(0);
+ MavenDownloaderImpl downloader = new MavenDownloaderImpl()) {
+ List<Socket> accepted = new CopyOnWriteArrayList<>();
+ Thread acceptor = new Thread(() -> {
+ try {
+ while (!server.isClosed()) {
+ accepted.add(server.accept());
+ }
+ } catch (Exception e) {
+ // closed
+ }
+ });
+ acceptor.setDaemon(true);
+ acceptor.start();
+ String stalled = "http://localhost:"; + server.getLocalPort() +
"/maven";
+
+ downloader.setMavenCentralEnabled(false);
+ downloader.setMavenApacheSnapshotEnabled(false);
+ downloader.setMavenSettingsLocation("false");
+ downloader.setRepos(stalled);
+ downloader.build();
+ assertEquals(60000, downloader.getRequestTimeout());
+ MavenDownloader customDownloader =
downloader.customize(localRepo.getAbsolutePath(), 5000, 1000);
+
+ long start = System.nanoTime();
+ assertThrows(MavenResolutionException.class,
+ () ->
customDownloader.resolveArtifacts(List.of("org.example.fixture:stalled:1.0.0"),
null, false,
+ false));
+ assertTrue(TimeUnit.NANOSECONDS.toSeconds(System.nanoTime() -
start) < 30);
+ // a slow repository is still used
+ assertTrue(downloader.getUnreachableRepositories().isEmpty());
+ for (Socket socket : accepted) {
+ socket.close();
+ }
+ }
+ }
+
+ @Test
+ void testConnectFailures() {
+
assertNotNull(MavenDownloaderImpl.UnreachableRepositoryListener.connectFailure(
+ new RuntimeException(new ConnectException("Connection
refused"))));
+
assertNotNull(MavenDownloaderImpl.UnreachableRepositoryListener.connectFailure(new
UnknownHostException("nope")));
+
assertNotNull(MavenDownloaderImpl.UnreachableRepositoryListener.connectFailure(
+ new SocketTimeoutException("Connect timed out")));
+ // Apache HttpClient 4 (the HTTP transport of Maven Resolver) has its
own exception for a connect timeout
+
assertNotNull(MavenDownloaderImpl.UnreachableRepositoryListener.connectFailure(
+ new RuntimeException(new ConnectTimeoutException("Connect to
localhost:1234 timed out"))));
+
assertNotNull(MavenDownloaderImpl.UnreachableRepositoryListener.connectFailure(
+ new HttpConnectTimeoutException("HTTP connect timed out")));
+ // a read timeout is a slow repository, not an unreachable one
+
assertNull(MavenDownloaderImpl.UnreachableRepositoryListener.connectFailure(
+ new SocketTimeoutException("Read timed out")));
+
assertNull(MavenDownloaderImpl.UnreachableRepositoryListener.connectFailure(new
IOException("Not found")));
+
assertNull(MavenDownloaderImpl.UnreachableRepositoryListener.connectFailure(null));
+ }
}
