This is an automated email from the ASF dual-hosted git repository. davsclaus pushed a commit to branch fix/CAMEL-24678 in repository https://gitbox.apache.org/repos/asf/camel.git
commit 19e14fa77b37f30d4048531886ac0aaa3b2f3f65 Author: Claus Ibsen <[email protected]> AuthorDate: Thu Sep 10 15:47:00 2026 +0200 CAMEL-24678: camel-jbang - infra ps lists every running instance infra ps built its rows from the test-infra catalog metadata, keyed by alias, so there was exactly one row per alias no matter how many instances were running. The PID for that row came from findPid(alias), which returned the first infra-<alias>-<pid>.json that File.list() happened to yield, and getServiceData() then rendered that same arbitrary instance. A second instance of a service was invisible, and the PID and SERVICE_DATA columns could describe a different process than the user expected. Split row construction from the table rendering: listServices() now delegates to an overridable buildRows(), the default being the catalog view that infra list keeps, while infra ps builds one row per running instance from the pid files. Rows are ordered by alias and then numerically by pid, so the output is stable rather than filesystem dependent, and a running service that has no catalog metadata is now listed instead of being silently dropped. A service is stopped by deleting its pid file, and infra run removes the pid and log files on shutdown, so a pid file with no live process behind it means the process was killed without running its shutdown hook. Such instances are no longer reported as running and their leftover files are pruned. The liveness check sits behind an overridable aliveCheck predicate so tests can use synthetic pids. Also fold in the same fix for infra log, which followed only the first log file matching an alias: it now tails every running instance and prefixes the lines with <alias>-<pid> when a service has more than one, and adds pid to the --json output so two instances of the same alias are distinguishable to a machine reader. Reading the pid files once also removes the directory listing that findPid() performed for each of the 54 catalog aliases on every invocation. infra get, infra stop and infra restart are unchanged: they already enumerate every instance through findPids(), and stop/restart must keep seeing stale pid files because deleting them is how they clean up. Co-Authored-By: Claude Opus 5 (1M context) <[email protected]> Signed-off-by: Claus Ibsen <[email protected]> --- .../ROOT/pages/camel-4x-upgrade-guide-4_23.adoc | 24 +++ .../ROOT/pages/camel-jbang-dev-services.adoc | 29 ++- .../core/commands/infra/InfraBaseCommand.java | 222 +++++++++++++++------ .../dsl/jbang/core/commands/infra/InfraLog.java | 71 +++---- .../dsl/jbang/core/commands/infra/InfraPs.java | 62 +++--- .../camel/dsl/jbang/core/model/InfraBaseDTO.java | 19 ++ .../commands/infra/InfraCommandTestSupport.java | 19 +- .../jbang/core/commands/infra/InfraListTest.java | 65 ++++++ .../jbang/core/commands/infra/InfraLogTest.java | 78 ++++++++ .../dsl/jbang/core/commands/infra/InfraPsTest.java | 149 +++++++++++++- .../camel/dsl/jbang/it/InfrastructureITCase.java | 5 +- 11 files changed, 598 insertions(+), 145 deletions(-) diff --git a/docs/user-manual/modules/ROOT/pages/camel-4x-upgrade-guide-4_23.adoc b/docs/user-manual/modules/ROOT/pages/camel-4x-upgrade-guide-4_23.adoc index 2a07916c1d7f..c187a6c91a1a 100644 --- a/docs/user-manual/modules/ROOT/pages/camel-4x-upgrade-guide-4_23.adoc +++ b/docs/user-manual/modules/ROOT/pages/camel-4x-upgrade-guide-4_23.adoc @@ -1848,3 +1848,27 @@ as a header cannot take effect. A route that set either header was silently igno Configure the `target` and `credentials` endpoint options instead, or route to a different endpoint with `toD` when the destination varies per message. The constants remain in place for backwards compatibility and are now marked deprecated in the component metadata. + +=== camel-jbang - camel infra ps lists every running instance + +`camel infra ps` now prints one row per running service instance instead of one row per service alias. +Starting the same service twice previously collapsed into a single row, and the PID and `SERVICE_DATA` +columns of that row were taken from whichever `infra-<service>-<pid>.json` file the filesystem happened +to return first, so they could describe a different process than expected. Rows are now ordered by +alias and then numerically by pid, so the output is stable across invocations. + +`camel infra ps` also no longer reports a service whose process is gone. A service is stopped by +deleting its pid file, and `camel infra run` removes both the pid file and the log file on shutdown, so +a leftover pid file means the process was killed without running its shutdown hook. Such files are now +pruned, which means `camel infra ps` and `camel infra list` may delete stale `infra-<service>-<pid>.json` +and `infra-<service>-<pid>.log` files from the Camel directory. + +`camel infra ps --json` gained a `pid` field for each running instance. It is omitted by +`camel infra list --json`, which describes the catalog rather than running processes. + +`camel infra log` now tails the log of every running instance of a service, rather than the first log +file matching the alias. When a service has more than one instance, its log lines are prefixed with +`<alias>-<pid>` instead of `<alias>` so the two can be told apart. + +`camel infra list` is unchanged: it keeps one row per alias, has no PID column, and lists services that +are not running. diff --git a/docs/user-manual/modules/ROOT/pages/camel-jbang-dev-services.adoc b/docs/user-manual/modules/ROOT/pages/camel-jbang-dev-services.adoc index 7c26f560eb46..ab20b48ddde2 100644 --- a/docs/user-manual/modules/ROOT/pages/camel-jbang-dev-services.adoc +++ b/docs/user-manual/modules/ROOT/pages/camel-jbang-dev-services.adoc @@ -93,10 +93,32 @@ camel infra restart kafka --background [source,bash] ---- $ camel infra ps - ALIAS IMPLEMENTATION DESCRIPTION - arangodb ArangoDB is a multi-model database for high-performance applications. + PID ALIAS IMPLEMENTATION UI DESCRIPTION SERVICE_DATA + 45211 arangodb ArangoDB is a multi-model database for high-pe... {"getServiceAddress":"http:/... ---- +There is one row per running instance, so starting the same service twice lists both processes, each +with its own PID and connection details: + +[source,bash] +---- +$ camel infra ps + PID ALIAS IMPLEMENTATION UI DESCRIPTION SERVICE_DATA + 45244 ftp FTP is a standard communication protocol ... {"port":21,"host":"localhost"... + 45262 ftp FTP is a standard communication protocol ... {"port":2121,"host":"localhos... +---- + +Pass a service name to list only its instances, and `--json` to get the full connection details of each +instance as machine-readable JSON: + +[source,bash] +---- +camel infra ps ftp +camel infra ps ftp --json +---- + +Services whose process is no longer running are not listed, and their leftover files are removed. + == Getting service details Retrieve connection details for a running service: @@ -127,6 +149,9 @@ Or from a specific service: camel infra log openldap ---- +Every running instance of the service is tailed. When more than one instance is running, the log lines +are prefixed with `<alias>-<pid>` instead of `<alias>` so they can be told apart. + == See Also The xref:camel-jbang-devtools.adoc#_sending_to_infrastructure_services[Sending messages to infrastructure services] feature in the Development Tools page can send messages directly to services started with `camel infra run`. diff --git a/dsl/camel-jbang/camel-jbang-core/src/main/java/org/apache/camel/dsl/jbang/core/commands/infra/InfraBaseCommand.java b/dsl/camel-jbang/camel-jbang-core/src/main/java/org/apache/camel/dsl/jbang/core/commands/infra/InfraBaseCommand.java index 8c6b0e7e9a54..14061161db48 100644 --- a/dsl/camel-jbang/camel-jbang-core/src/main/java/org/apache/camel/dsl/jbang/core/commands/infra/InfraBaseCommand.java +++ b/dsl/camel-jbang/camel-jbang-core/src/main/java/org/apache/camel/dsl/jbang/core/commands/infra/InfraBaseCommand.java @@ -30,9 +30,9 @@ import java.util.HashSet; import java.util.LinkedHashMap; import java.util.List; import java.util.Map; -import java.util.Objects; import java.util.Set; import java.util.function.Consumer; +import java.util.function.LongPredicate; import java.util.stream.Collectors; import java.util.stream.Stream; @@ -50,6 +50,7 @@ import org.apache.camel.dsl.jbang.core.commands.CamelCommand; import org.apache.camel.dsl.jbang.core.commands.CamelJBangMain; import org.apache.camel.dsl.jbang.core.common.CamelTableColumns; import org.apache.camel.dsl.jbang.core.common.CommandLineHelper; +import org.apache.camel.dsl.jbang.core.common.PathUtils; import org.apache.camel.dsl.jbang.core.common.TerminalWidthHelper; import org.apache.camel.dsl.jbang.core.model.InfraBaseDTO; import org.apache.camel.support.PatternHelper; @@ -65,6 +66,12 @@ public abstract class InfraBaseCommand extends CamelCommand { description = "Output in JSON Format") boolean jsonOutput; + /** + * Whether the process behind a pid file is still running. Overridable so tests can use synthetic pids in their pid + * files without the command pruning them as stale. + */ + protected LongPredicate aliveCheck = pid -> ProcessHandle.of(pid).map(ProcessHandle::isAlive).orElse(false); + protected InfraBaseCommand(CamelJBangMain main) { super(main); @@ -77,7 +84,35 @@ public abstract class InfraBaseCommand extends CamelCommand { protected static Map<Long, Path> findPids(String name) throws Exception { Map<Long, Path> pids = new HashMap<>(); - // we need to know the pids of the running camel integrations + final String pattern = toPidPattern(name); + + try (Stream<Path> files = Files.list(CommandLineHelper.getCamelDir())) { + List<Path> pidFiles = files.filter(p -> { + var n = p.getFileName().toString(); + return n.startsWith("infra-") && n.endsWith(".json"); + }) + .toList(); + for (Path pidFile : pidFiles) { + String fn = pidFile.getFileName().toString(); + String sn = serviceNameFromPidFile(fn); + String pid = pidFromPidFile(fn); + if (pid.equals(pattern) || PatternHelper.matchPattern(sn, pattern)) { + pids.put(Long.valueOf(pid), pidFile); + } + } + } catch (NoSuchFileException e) { + // camel directory does not exist yet + } + + return pids; + } + + /** + * Normalizes a user supplied service name into the pattern used to match against the service names parsed out of + * the pid files. A plain name is widened to a prefix match, a trailing {@code !} pins it to that name only, and a + * numeric argument is left as-is so it can be matched against the pid instead. + */ + private static String toPidPattern(String name) { if (!name.matches("\\d+")) { if (name.endsWith("!")) { // exclusive this name only @@ -87,10 +122,27 @@ public abstract class InfraBaseCommand extends CamelCommand { name = name + "*"; } } + return name; + } + + /** + * Finds the infra service instances that are actually running, one per {@code infra-<service>-<pid>.json} pid file, + * ordered by service name and then numerically by pid so repeated invocations are stable. + * <p> + * A service is stopped by deleting its pid file, and {@code infra run} deletes both the pid file and the log file + * on shutdown. A pid file whose process is gone therefore means the process was killed hard and never ran its + * shutdown hook, so the leftovers are pruned here instead of being reported as a running service. + * + * @param name the service name or pid to match, or {@code null} to return every running instance. + * @return the running instances, never {@code null}. + */ + protected List<RunningService> findRunningServices(String name) { + List<RunningService> answer = new ArrayList<>(); - final String pattern = name; + final String pattern = name != null ? toPidPattern(name) : null; + Path camelDir = CommandLineHelper.getCamelDir(); - try (Stream<Path> files = Files.list(CommandLineHelper.getCamelDir())) { + try (Stream<Path> files = Files.list(camelDir)) { List<Path> pidFiles = files.filter(p -> { var n = p.getFileName().toString(); return n.startsWith("infra-") && n.endsWith(".json"); @@ -100,15 +152,37 @@ public abstract class InfraBaseCommand extends CamelCommand { String fn = pidFile.getFileName().toString(); String sn = serviceNameFromPidFile(fn); String pid = pidFromPidFile(fn); - if (pid.equals(pattern) || PatternHelper.matchPattern(sn, pattern)) { - pids.put(Long.valueOf(pid), pidFile); + long pidNumber; + try { + pidNumber = Long.parseLong(pid); + } catch (NumberFormatException e) { + // not a file we recognise, so neither report nor prune it + continue; + } + if (!aliveCheck.test(pidNumber)) { + pruneStaleFiles(camelDir, sn, pid); + continue; + } + if (pattern == null || pid.equals(pattern) || PatternHelper.matchPattern(sn, pattern)) { + answer.add(new RunningService(sn, pid, pidNumber, pidFile)); } } - } catch (NoSuchFileException e) { - // camel directory does not exist yet + } catch (IOException e) { + // camel directory does not exist yet, or cannot be read } - return pids; + answer.sort(Comparator.comparing(RunningService::alias).thenComparingLong(RunningService::pidNumber)); + + return answer; + } + + /** + * Removes the pid and log files left behind by an instance whose process is gone, so it stops being reported as + * running. + */ + private void pruneStaleFiles(Path camelDir, String service, String pid) { + PathUtils.deleteFile(camelDir.resolve(getJsonFileName(service, pid))); + PathUtils.deleteFile(camelDir.resolve(getLogFileName(service, pid))); } /** @@ -152,14 +226,14 @@ public abstract class InfraBaseCommand extends CamelCommand { return metadata; } - public int listServices(Consumer<List<Row>> serviceConsumer) throws IOException { + /** + * Folds the test-infra catalog metadata into the set of known service aliases, merging the implementations and UI + * support of every metadata entry that shares an alias. + */ + protected Map<String, InfraServiceAlias> aliasMetadata() throws IOException { Map<String, InfraServiceAlias> services = new LinkedHashMap<>(); - List<TestInfraService> metadata = getMetadata(); - - List<InfraList.Row> rows = new ArrayList<>(metadata.size()); - - for (TestInfraService service : metadata) { + for (TestInfraService service : getMetadata()) { for (String alias : service.alias()) { if (!services.containsKey(alias)) { services.put(alias, new InfraServiceAlias(service.description())); @@ -173,32 +247,84 @@ public abstract class InfraBaseCommand extends CamelCommand { } } - int width = 0; + return services; + } + + /** + * Builds the table rows. + * <p> + * This default is the catalog view used by {@code infra list}: one row per known alias, describing what can be run + * rather than what is running. When an alias does happen to be running, the lowest pid of its instances supplies + * the SERVICE_DATA column. {@code infra ps} overrides this to emit one row per running <em>instance</em> instead, + * so two instances of the same service each get their own PID and SERVICE_DATA. + */ + protected List<Row> buildRows(Map<String, InfraServiceAlias> services) { + Map<String, List<RunningService>> running = findRunningServices(null).stream() + .collect(Collectors.groupingBy(RunningService::alias)); + + List<Row> rows = new ArrayList<>(services.size()); for (Map.Entry<String, InfraServiceAlias> entry : services.entrySet()) { - width = Math.max(width, entry.getKey().length()); - String pid = findPid(entry.getKey()); - rows.add(new InfraList.Row( - pid, + List<RunningService> instances = running.get(entry.getKey()); + RunningService instance = instances == null || instances.isEmpty() ? null : instances.get(0); + rows.add(new Row( + instance != null ? instance.pid() : null, entry.getKey(), - entry.getValue().getAliasImplementation() - .stream() - .sorted() - .collect(Collectors.joining(", ")), + implementationsOf(entry.getValue()), entry.getValue().getDescription(), - getServiceData(entry.getKey(), pid), + instance != null ? readServiceData(instance.pidFile()) : null, entry.getValue().isUiSupported())); } - rows.sort(Comparator.comparing(InfraList.Row::alias)); + return rows; + } + + /** + * Renders the implementations of an alias as a stable, comma separated list. + */ + protected static String implementationsOf(InfraServiceAlias alias) { + if (alias == null) { + return null; + } + return alias.getAliasImplementation() + .stream() + .sorted() + .collect(Collectors.joining(", ")); + } + + /** + * Reads the connection details an instance wrote to its pid file. Returns {@code null} when the file has since + * gone, which happens when the service is stopped while the table is being built. + */ + protected static String readServiceData(Path pidFile) { + try { + return Files.readString(pidFile); + } catch (IOException e) { + return null; + } + } + + public int listServices(Consumer<List<Row>> serviceConsumer) throws IOException { + Map<String, InfraServiceAlias> services = aliasMetadata(); + + List<Row> rows = buildRows(services); + + // sorting by alias only, which is a stable sort, so the pid ordering that buildRows established within an + // alias is preserved + rows.sort(Comparator.comparing(Row::alias)); // Do something with the full services list (for example, filter) serviceConsumer.accept(rows); + int width = 0; + for (String alias : services.keySet()) { + width = Math.max(width, alias.length()); + } + if (jsonOutput) { printer().println( Jsoner.serialize( rows.stream().map(row -> new InfraBaseDTO( - row.alias, row.aliasImplementation, row.description, + row.pid, row.alias, row.aliasImplementation, row.description, parseServiceData(row.serviceData()), row.uiSupported())) .map(InfraBaseDTO::toMap) .collect(Collectors.toList()))); @@ -248,34 +374,6 @@ public abstract class InfraBaseCommand extends CamelCommand { } } - private String getServiceData(String key, String pid) { - Path jsonFilePath = CommandLineHelper.getCamelDir().resolve(getJsonFileName(key, pid)); - if (jsonFilePath.toFile().exists()) { - try { - return Files.readString(jsonFilePath); - } catch (IOException e) { - // ignore - } - } - - return null; - } - - private String findPid(String key) { - Path p = CommandLineHelper.getCamelDir(); - try { - Files.createDirectories(p); - for (String s : Objects.requireNonNull(p.toFile().list())) { - if (s.startsWith("infra-" + key + "-") && s.endsWith(".json")) { - return pidFromPidFile(s); - } - } - } catch (Exception e) { - // ignore - } - return null; - } - public String getLogFileName(String service, String pid) { return String.format("infra-%s-%s.log", service, pid); } @@ -301,7 +399,19 @@ public abstract class InfraBaseCommand extends CamelCommand { boolean uiSupported) { } - private static class InfraServiceAlias { + /** + * A single running infra service instance, as discovered from its {@code infra-<service>-<pid>.json} pid file. + * There can be more than one instance per service alias, each with its own pid and connection details. + * + * @param alias the service alias, such as {@code ftp}. + * @param pid the pid as written in the file name, used for display. + * @param pidNumber the same pid parsed as a number, used for ordering. + * @param pidFile the pid file holding the connection details of this instance. + */ + protected record RunningService(String alias, String pid, long pidNumber, Path pidFile) { + } + + protected static class InfraServiceAlias { private final String description; private final Set<String> aliasImplementation = new HashSet<>(); private boolean uiSupported; diff --git a/dsl/camel-jbang/camel-jbang-core/src/main/java/org/apache/camel/dsl/jbang/core/commands/infra/InfraLog.java b/dsl/camel-jbang/camel-jbang-core/src/main/java/org/apache/camel/dsl/jbang/core/commands/infra/InfraLog.java index 97a78992d32f..671deed7c079 100644 --- a/dsl/camel-jbang/camel-jbang-core/src/main/java/org/apache/camel/dsl/jbang/core/commands/infra/InfraLog.java +++ b/dsl/camel-jbang/camel-jbang-core/src/main/java/org/apache/camel/dsl/jbang/core/commands/infra/InfraLog.java @@ -22,10 +22,11 @@ import java.nio.file.Files; import java.nio.file.Path; import java.util.ArrayList; import java.util.List; +import java.util.Map; import java.util.concurrent.ExecutorService; import java.util.concurrent.Executors; import java.util.concurrent.Future; -import java.util.stream.Stream; +import java.util.stream.Collectors; import org.apache.camel.dsl.jbang.core.commands.CamelJBangMain; import org.apache.camel.dsl.jbang.core.common.CommandLineHelper; @@ -38,11 +39,12 @@ import picocli.CommandLine; showDefaultValues = true, footer = { "%nExamples:", + " camel infra log", " camel infra log kafka" }) public class InfraLog extends InfraBaseCommand { @CommandLine.Parameters(description = "Service name", arity = "0..2") - private List<String> serviceName; + List<String> serviceName; @CommandLine.Option(names = { "--lines" }, defaultValue = "50", description = "The number of lines from the end of the log to use as starting offset") @@ -56,51 +58,40 @@ public class InfraLog extends InfraBaseCommand { executorService = Executors.newFixedThreadPool(10); } + /** + * Tails the log of every running instance, rather than the first log file that happens to match the alias. Two + * instances of the same service each have their own log, so both are followed and their lines are prefixed with the + * pid to tell them apart. + */ @Override public Integer doCall() throws Exception { + String name = serviceName == null || serviceName.isEmpty() ? null : serviceName.get(0); + + List<RunningService> instances = findRunningServices(name); + + // only carry the pid in the prefix when there is more than one instance of that alias to disambiguate + Map<String, Long> instancesPerAlias = instances.stream() + .collect(Collectors.groupingBy(RunningService::alias, Collectors.counting())); + List<Future<?>> futures = new ArrayList<>(); - if (serviceName == null || serviceName.isEmpty()) { - // Log everything - try (Stream<Path> files = Files.list(CommandLineHelper.getCamelDir())) { - List<Path> logFiles = files.filter(p -> { - String name = p.getFileName().toString(); - return name.startsWith("infra-") && name.endsWith(".log"); - }) - .toList(); - - for (Path logFile : logFiles) { - String alias = serviceNameFromPidFile(logFile.getFileName().toString()); - createTailer(logFile.toFile(), alias, futures); - } - } catch (IOException e) { - // ignore + for (RunningService instance : instances) { + Path logFile = CommandLineHelper.getCamelDir().resolve(getLogFileName(instance.alias(), instance.pid())); + if (!Files.isRegularFile(logFile)) { + // the service has not written any log yet + continue; } + String prefix = instancesPerAlias.get(instance.alias()) > 1 + ? instance.alias() + "-" + instance.pid() : instance.alias(); + createTailer(logFile.toFile(), prefix, futures); + } - if (futures.isEmpty()) { + if (futures.isEmpty()) { + if (name != null) { + printer().printErr("Log not found for service " + name); + } else { printer().println("There are no running services"); - return -1; } - } else { - String alias = serviceName.get(0); - - Path logFile = null; - try (Stream<Path> files = Files.list(CommandLineHelper.getCamelDir())) { - logFile = files.filter(p -> { - String name = p.getFileName().toString(); - return name.startsWith("infra-" + alias + "-") && name.endsWith(".log"); - }) - .findFirst() - .orElse(null); - } catch (IOException e) { - // ignore - } - - if (logFile == null) { - printer().printErr("Log not found for service " + alias); - return -1; - } - - createTailer(logFile.toFile(), alias, futures); + return -1; } for (Future<?> future : futures) { diff --git a/dsl/camel-jbang/camel-jbang-core/src/main/java/org/apache/camel/dsl/jbang/core/commands/infra/InfraPs.java b/dsl/camel-jbang/camel-jbang-core/src/main/java/org/apache/camel/dsl/jbang/core/commands/infra/InfraPs.java index f761be085a8b..f32b0f96e930 100644 --- a/dsl/camel-jbang/camel-jbang-core/src/main/java/org/apache/camel/dsl/jbang/core/commands/infra/InfraPs.java +++ b/dsl/camel-jbang/camel-jbang-core/src/main/java/org/apache/camel/dsl/jbang/core/commands/infra/InfraPs.java @@ -16,27 +16,23 @@ */ package org.apache.camel.dsl.jbang.core.commands.infra; -import java.io.IOException; -import java.nio.file.Files; -import java.nio.file.Path; -import java.util.HashSet; +import java.util.ArrayList; import java.util.List; -import java.util.Set; -import java.util.stream.Stream; +import java.util.Map; import org.apache.camel.dsl.jbang.core.commands.CamelJBangMain; -import org.apache.camel.dsl.jbang.core.common.CommandLineHelper; import picocli.CommandLine; @CommandLine.Command(name = "ps", description = "Displays running services", sortOptions = false, showDefaultValues = true, footer = { "%nExamples:", - " camel infra ps" }) + " camel infra ps", + " camel infra ps kafka" }) public class InfraPs extends InfraBaseCommand { @CommandLine.Parameters(description = "Service name", arity = "0..1") - private List<String> serviceName; + List<String> serviceName; public InfraPs(CamelJBangMain main) { super(main); @@ -47,34 +43,36 @@ public class InfraPs extends InfraBaseCommand { return true; } + /** + * Emits one row per running instance rather than one row per alias, so starting the same service twice shows both + * processes, each with its own PID and SERVICE_DATA. The alias metadata only fills in the descriptive columns; the + * rows themselves come from the pid files, so a service that is running but absent from the catalog is still + * listed. + */ @Override - public Integer doCall() throws Exception { - // retrieve running services to filter output - Set<String> runningAliases = new HashSet<>(); - try (Stream<Path> files = Files.list(CommandLineHelper.getCamelDir())) { - List<Path> pidFiles = files.filter(p -> { - if (serviceName == null) { - return p.getFileName().toString().startsWith("infra-"); - } else { - return p.getFileName().toString().startsWith("infra-" + serviceName.get(0)); - } - }) - .toList(); + protected List<Row> buildRows(Map<String, InfraServiceAlias> services) { + String name = serviceName == null || serviceName.isEmpty() ? null : serviceName.get(0); + + List<RunningService> instances = findRunningServices(name); - for (Path pidFile : pidFiles) { - String runningServiceName = serviceNameFromPidFile(pidFile.getFileName().toString()); - runningAliases.add(runningServiceName); - } - } catch (IOException e) { - // ignore + List<Row> rows = new ArrayList<>(instances.size()); + for (RunningService instance : instances) { + InfraServiceAlias alias = services.get(instance.alias()); + rows.add(new Row( + instance.pid(), + instance.alias(), + implementationsOf(alias), + alias != null ? alias.getDescription() : null, + readServiceData(instance.pidFile()), + alias != null && alias.isUiSupported())); } + return rows; + } + + @Override + public Integer doCall() throws Exception { return listServices(rows -> { - if (runningAliases.isEmpty()) { - rows.clear(); - } else { - rows.removeIf(row -> !runningAliases.contains(row.alias())); - } }); } } diff --git a/dsl/camel-jbang/camel-jbang-core/src/main/java/org/apache/camel/dsl/jbang/core/model/InfraBaseDTO.java b/dsl/camel-jbang/camel-jbang-core/src/main/java/org/apache/camel/dsl/jbang/core/model/InfraBaseDTO.java index e92a0bac7f80..0981a304145d 100644 --- a/dsl/camel-jbang/camel-jbang-core/src/main/java/org/apache/camel/dsl/jbang/core/model/InfraBaseDTO.java +++ b/dsl/camel-jbang/camel-jbang-core/src/main/java/org/apache/camel/dsl/jbang/core/model/InfraBaseDTO.java @@ -22,6 +22,7 @@ import org.apache.camel.util.json.JsonObject; public class InfraBaseDTO { + private String pid; private String alias; private String aliasImplementation; private String description; @@ -33,6 +34,12 @@ public class InfraBaseDTO { public InfraBaseDTO(String alias, String aliasImplementation, String description, Object serviceData, boolean uiSupported) { + this(null, alias, aliasImplementation, description, serviceData, uiSupported); + } + + public InfraBaseDTO(String pid, String alias, String aliasImplementation, String description, Object serviceData, + boolean uiSupported) { + this.pid = pid; this.alias = alias; this.aliasImplementation = aliasImplementation; this.description = description; @@ -40,6 +47,14 @@ public class InfraBaseDTO { this.uiSupported = uiSupported; } + public String getPid() { + return pid; + } + + public void setPid(String pid) { + this.pid = pid; + } + public String getAlias() { return alias; } @@ -82,6 +97,10 @@ public class InfraBaseDTO { public Map<String, Object> toMap() { JsonObject jo = new JsonObject(); + // only running services have a pid, so it is omitted for the catalog listing + if (pid != null) { + jo.put("pid", pid); + } jo.put("alias", alias); if (aliasImplementation != null) { jo.put("aliasImplementation", aliasImplementation); diff --git a/dsl/camel-jbang/camel-jbang-core/src/test/java/org/apache/camel/dsl/jbang/core/commands/infra/InfraCommandTestSupport.java b/dsl/camel-jbang/camel-jbang-core/src/test/java/org/apache/camel/dsl/jbang/core/commands/infra/InfraCommandTestSupport.java index 8e587c3c4bd2..38e9ab36b995 100644 --- a/dsl/camel-jbang/camel-jbang-core/src/test/java/org/apache/camel/dsl/jbang/core/commands/infra/InfraCommandTestSupport.java +++ b/dsl/camel-jbang/camel-jbang-core/src/test/java/org/apache/camel/dsl/jbang/core/commands/infra/InfraCommandTestSupport.java @@ -62,7 +62,22 @@ abstract class InfraCommandTestSupport extends CamelCommandBaseTestSupport { * Writes a pid file named {@code infra-<service>-<pid>.json} into the Camel directory with the given JSON content. */ protected static void writePidFile(String service, long pid, String content) throws IOException { - Path pidFile = CommandLineHelper.getCamelDir().resolve("infra-" + service + "-" + pid + ".json"); - Files.writeString(pidFile, content); + Files.writeString(pidFile(service, pid), content); + } + + /** + * Writes a log file named {@code infra-<service>-<pid>.log} into the Camel directory, as {@code infra run} does + * alongside the pid file. + */ + protected static void writeLogFile(String service, long pid, String content) throws IOException { + Files.writeString(logFile(service, pid), content); + } + + protected static Path pidFile(String service, long pid) { + return CommandLineHelper.getCamelDir().resolve("infra-" + service + "-" + pid + ".json"); + } + + protected static Path logFile(String service, long pid) { + return CommandLineHelper.getCamelDir().resolve("infra-" + service + "-" + pid + ".log"); } } diff --git a/dsl/camel-jbang/camel-jbang-core/src/test/java/org/apache/camel/dsl/jbang/core/commands/infra/InfraListTest.java b/dsl/camel-jbang/camel-jbang-core/src/test/java/org/apache/camel/dsl/jbang/core/commands/infra/InfraListTest.java new file mode 100644 index 000000000000..3ad5e311e5f4 --- /dev/null +++ b/dsl/camel-jbang/camel-jbang-core/src/test/java/org/apache/camel/dsl/jbang/core/commands/infra/InfraListTest.java @@ -0,0 +1,65 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.camel.dsl.jbang.core.commands.infra; + +import org.apache.camel.dsl.jbang.core.commands.CamelJBangMain; +import org.junit.jupiter.api.Test; + +import static org.junit.jupiter.api.Assertions.assertEquals; +import static org.junit.jupiter.api.Assertions.assertFalse; +import static org.junit.jupiter.api.Assertions.assertTrue; + +/** + * Unit tests for {@link InfraList}. + * + * {@code infra list} describes the catalog of services that <em>can</em> be run, so unlike {@code infra ps} it keeps + * one row per alias, has no PID column, and lists services that are not running. These tests pin that shape, because + * both commands share the table rendering in {@link InfraBaseCommand}. + */ +class InfraListTest extends InfraCommandTestSupport { + + @Test + void shouldKeepOneRowPerAliasWhenSeveralInstancesAreRunning() throws Exception { + // CAMEL-24678: infra ps became one row per instance, but infra list must stay one row per alias + writePidFile("ftp", 1234, "{\"port\":21}"); + writePidFile("ftp", 5678, "{\"port\":2121}"); + + InfraList command = new InfraList(new CamelJBangMain().withPrinter(printer)); + command.aliveCheck = pid -> true; + int exit = command.doCall(); + + assertEquals(0, exit); + String out = printer.getOutput(); + assertEquals(1, out.lines().filter(l -> l.stripLeading().startsWith("ftp ")).count(), + "the ftp alias must appear exactly once, was: " + out); + assertFalse(out.contains("PID"), "infra list must not render the PID column, was: " + out); + // the catalog view lists what can be run, not only what is running + assertTrue(out.contains("minio"), "a service that is not running should still be listed, was: " + out); + } + + @Test + void shouldListTheCatalogWhenNothingIsRunning() throws Exception { + InfraList command = new InfraList(new CamelJBangMain().withPrinter(printer)); + int exit = command.doCall(); + + assertEquals(0, exit); + String out = printer.getOutput(); + assertTrue(out.contains("ALIAS"), "the table header should be printed, was: " + out); + assertTrue(out.contains("ftp"), "every known service should be listed, was: " + out); + assertTrue(out.contains("minio"), "every known service should be listed, was: " + out); + } +} diff --git a/dsl/camel-jbang/camel-jbang-core/src/test/java/org/apache/camel/dsl/jbang/core/commands/infra/InfraLogTest.java b/dsl/camel-jbang/camel-jbang-core/src/test/java/org/apache/camel/dsl/jbang/core/commands/infra/InfraLogTest.java new file mode 100644 index 000000000000..7995540555be --- /dev/null +++ b/dsl/camel-jbang/camel-jbang-core/src/test/java/org/apache/camel/dsl/jbang/core/commands/infra/InfraLogTest.java @@ -0,0 +1,78 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.camel.dsl.jbang.core.commands.infra; + +import java.nio.file.Files; +import java.util.List; + +import org.apache.camel.dsl.jbang.core.commands.CamelJBangMain; +import org.junit.jupiter.api.Test; + +import static org.junit.jupiter.api.Assertions.assertEquals; +import static org.junit.jupiter.api.Assertions.assertFalse; +import static org.junit.jupiter.api.Assertions.assertTrue; + +/** + * Unit tests for {@link InfraLog}. + * + * {@code infra log} discovers what to tail from the pid files, so it only follows services that are actually running, + * and it follows <em>every</em> instance of a service rather than the first log file that happens to match the alias. + * Only the paths that return without tailing are covered here, because tailing itself blocks until the services stop. + */ +class InfraLogTest extends InfraCommandTestSupport { + + @Test + void shouldReportNoRunningServices() throws Exception { + InfraLog command = new InfraLog(new CamelJBangMain().withPrinter(printer)); + int exit = command.doCall(); + + assertEquals(-1, exit); + assertTrue(printer.getOutput().contains("There are no running services"), + "was: " + printer.getOutput()); + } + + @Test + void shouldReportMissingLogForNamedService() throws Exception { + writePidFile("ftp", 1234, "{}"); + + InfraLog command = new InfraLog(new CamelJBangMain().withPrinter(printer)); + command.aliveCheck = pid -> true; + command.serviceName = List.of("kafka"); + int exit = command.doCall(); + + assertEquals(-1, exit); + assertTrue(printer.getOutput().contains("Log not found for service kafka"), + "was: " + printer.getOutput()); + } + + @Test + void shouldNotTailAServiceWhoseProcessIsGone() throws Exception { + // the log of a service that was killed hard is pruned along with its pid file, rather than tailed forever + writePidFile("ftp", 1234, "{}"); + writeLogFile("ftp", 1234, "starting\n"); + + InfraLog command = new InfraLog(new CamelJBangMain().withPrinter(printer)); + command.aliveCheck = pid -> false; + int exit = command.doCall(); + + assertEquals(-1, exit); + assertTrue(printer.getOutput().contains("There are no running services"), + "was: " + printer.getOutput()); + assertFalse(Files.exists(pidFile("ftp", 1234)), "the dead instance's pid file should be pruned"); + assertFalse(Files.exists(logFile("ftp", 1234)), "the dead instance's log file should be pruned"); + } +} diff --git a/dsl/camel-jbang/camel-jbang-core/src/test/java/org/apache/camel/dsl/jbang/core/commands/infra/InfraPsTest.java b/dsl/camel-jbang/camel-jbang-core/src/test/java/org/apache/camel/dsl/jbang/core/commands/infra/InfraPsTest.java index 3ae0a669890c..4c197ba980d5 100644 --- a/dsl/camel-jbang/camel-jbang-core/src/test/java/org/apache/camel/dsl/jbang/core/commands/infra/InfraPsTest.java +++ b/dsl/camel-jbang/camel-jbang-core/src/test/java/org/apache/camel/dsl/jbang/core/commands/infra/InfraPsTest.java @@ -16,6 +16,9 @@ */ package org.apache.camel.dsl.jbang.core.commands.infra; +import java.nio.file.Files; +import java.util.List; + import org.apache.camel.dsl.jbang.core.commands.CamelJBangMain; import org.junit.jupiter.api.Test; @@ -26,20 +29,27 @@ import static org.junit.jupiter.api.Assertions.assertTrue; /** * Unit tests for {@link InfraPs}. * - * Unlike {@code infra list}, which prints every known service, {@code infra ps} only shows the services that have a - * running pid file ({@code infra-<service>-<pid>.json}) in the Camel directory, and includes the PID column. The home - * directory is redirected to an isolated folder by {@link InfraCommandTestSupport} so the tests do not depend on - * locally running services. + * Unlike {@code infra list}, which prints every known service as a single row per alias, {@code infra ps} prints one + * row per <em>running instance</em> ({@code infra-<service>-<pid>.json}) and includes the PID column, so the same + * service started twice shows up twice. The home directory is redirected to an isolated folder by + * {@link InfraCommandTestSupport} so the tests do not depend on locally running services, and {@code aliveCheck} is + * stubbed so the synthetic pids in the pid files are not pruned as stale. */ class InfraPsTest extends InfraCommandTestSupport { + private InfraPs command() { + InfraPs command = new InfraPs(new CamelJBangMain().withPrinter(printer)); + // the pid files carry synthetic pids, so treat every pid as a live process + command.aliveCheck = pid -> true; + return command; + } + @Test void shouldOnlyListRunningServicesWithPid() throws Exception { // kafka is running, minio is not writePidFile("kafka", 1234, "{}"); - InfraPs command = new InfraPs(new CamelJBangMain().withPrinter(printer)); - int exit = command.doCall(); + int exit = command().doCall(); assertEquals(0, exit); String out = printer.getOutput(); @@ -54,8 +64,7 @@ class InfraPsTest extends InfraCommandTestSupport { // second hyphen-delimited segment ("hive"), otherwise the catalog row would be filtered out writePidFile("hive-mq", 1234, "{}"); - InfraPs command = new InfraPs(new CamelJBangMain().withPrinter(printer)); - int exit = command.doCall(); + int exit = command().doCall(); assertEquals(0, exit); String out = printer.getOutput(); @@ -66,12 +75,132 @@ class InfraPsTest extends InfraCommandTestSupport { @Test void shouldListNoServicesWhenNoneRunning() throws Exception { // no pid files written: the service table must be cleared - InfraPs command = new InfraPs(new CamelJBangMain().withPrinter(printer)); - int exit = command.doCall(); + int exit = command().doCall(); assertEquals(0, exit); String out = printer.getOutput(); assertFalse(out.contains("kafka"), "no service rows should be printed when nothing is running, was: " + out); assertFalse(out.contains("minio"), "no service rows should be printed when nothing is running, was: " + out); } + + @Test + void shouldListEveryInstanceOfTheSameService() throws Exception { + // CAMEL-24678: the same service started twice used to collapse into a single row with an arbitrary pid + writePidFile("ftp", 1234, "{\"port\":21}"); + writePidFile("ftp", 5678, "{\"port\":2121}"); + + int exit = command().doCall(); + + assertEquals(0, exit); + String out = printer.getOutput(); + assertTrue(out.contains("1234"), "the first ftp instance should be listed, was: " + out); + assertTrue(out.contains("5678"), "the second ftp instance should be listed, was: " + out); + // both instances render their own connection details, not the details of whichever file was read first + assertTrue(out.contains("21"), "the first instance should show its own service data, was: " + out); + assertTrue(out.contains("2121"), "the second instance should show its own service data, was: " + out); + assertEquals(2, out.lines().filter(l -> l.contains("ftp")).count(), + "there should be exactly one row per running instance, was: " + out); + } + + @Test + void shouldOrderInstancesByAliasThenPid() throws Exception { + // the pid used to come from File.list() order, which is filesystem dependent; the output must be stable + writePidFile("ftp", 5678, "{}"); + writePidFile("ftp", 1234, "{}"); + writePidFile("kafka", 4321, "{}"); + + int exit = command().doCall(); + + assertEquals(0, exit); + String out = printer.getOutput(); + assertTrue(out.indexOf("1234") < out.indexOf("5678"), + "instances of the same alias should be ordered numerically by pid, was: " + out); + assertTrue(out.indexOf("5678") < out.indexOf("4321"), + "rows should be grouped by alias, was: " + out); + } + + @Test + void shouldFilterByServiceName() throws Exception { + writePidFile("ftp", 1234, "{}"); + writePidFile("kafka", 5678, "{}"); + + InfraPs command = command(); + command.serviceName = List.of("ftp"); + int exit = command.doCall(); + + assertEquals(0, exit); + String out = printer.getOutput(); + assertTrue(out.contains("1234"), "the matching ftp instance should be listed, was: " + out); + assertFalse(out.contains("5678"), "the unmatched kafka instance must not be listed, was: " + out); + } + + @Test + void shouldDropAndPruneInstancesWhoseProcessIsGone() throws Exception { + // a service is stopped by deleting its pid file, and infra run removes the pid and log files on shutdown, so + // a pid file with no process behind it means the process was killed hard: report it as gone and clean up + writePidFile("ftp", 1234, "{}"); + writeLogFile("ftp", 1234, "starting\n"); + writePidFile("ftp", 5678, "{}"); + writeLogFile("ftp", 5678, "starting\n"); + + InfraPs command = command(); + command.aliveCheck = pid -> pid == 1234; + int exit = command.doCall(); + + assertEquals(0, exit); + String out = printer.getOutput(); + assertTrue(out.contains("1234"), "the live instance should still be listed, was: " + out); + assertFalse(out.contains("5678"), "the dead instance must not be listed, was: " + out); + + assertTrue(Files.exists(pidFile("ftp", 1234)), "the live instance's pid file must be kept"); + assertTrue(Files.exists(logFile("ftp", 1234)), "the live instance's log file must be kept"); + assertFalse(Files.exists(pidFile("ftp", 5678)), "the dead instance's pid file should be pruned"); + assertFalse(Files.exists(logFile("ftp", 5678)), "the dead instance's log file should be pruned"); + } + + @Test + void shouldTreatTheCurrentProcessAsAliveByDefault() throws Exception { + // exercises the real ProcessHandle based aliveCheck rather than a stub: this JVM is certainly running + long pid = ProcessHandle.current().pid(); + writePidFile("ftp", pid, "{}"); + + InfraPs command = new InfraPs(new CamelJBangMain().withPrinter(printer)); + int exit = command.doCall(); + + assertEquals(0, exit); + String out = printer.getOutput(); + assertTrue(out.contains(String.valueOf(pid)), "the running instance should be listed, was: " + out); + assertTrue(Files.exists(pidFile("ftp", pid)), "a live instance's pid file must not be pruned"); + } + + @Test + void shouldListRunningServiceMissingFromTheCatalog() throws Exception { + // the rows come from the pid files, so an alias with no catalog metadata is still reported as running + writePidFile("not-in-catalog", 1234, "{}"); + + int exit = command().doCall(); + + assertEquals(0, exit); + String out = printer.getOutput(); + assertTrue(out.contains("not-in-catalog"), "an unknown running alias should still be listed, was: " + out); + assertTrue(out.contains("1234"), "an unknown running alias should show its pid, was: " + out); + } + + @Test + void shouldIncludePidInJsonOutput() throws Exception { + writePidFile("ftp", 1234, "{\"port\":21}"); + writePidFile("ftp", 5678, "{\"port\":2121}"); + + InfraPs command = command(); + command.jsonOutput = true; + int exit = command.doCall(); + + assertEquals(0, exit); + String out = printer.getOutput(); + // without the pid, two instances of the same alias would be indistinguishable to a machine reader + assertTrue(out.contains("\"pid\":\"1234\""), "json output should carry the pid, was: " + out); + assertTrue(out.contains("\"pid\":\"5678\""), "json output should carry the pid, was: " + out); + assertTrue(out.contains("\"port\":21"), "json output should carry the per-instance service data, was: " + out); + assertTrue(out.contains("\"port\":2121"), "json output should carry the per-instance service data, was: " + out); + } } diff --git a/dsl/camel-jbang/camel-jbang-it/src/test/java/org/apache/camel/dsl/jbang/it/InfrastructureITCase.java b/dsl/camel-jbang/camel-jbang-it/src/test/java/org/apache/camel/dsl/jbang/it/InfrastructureITCase.java index 4a63451e48f4..d7beba63fda1 100644 --- a/dsl/camel-jbang/camel-jbang-it/src/test/java/org/apache/camel/dsl/jbang/it/InfrastructureITCase.java +++ b/dsl/camel-jbang/camel-jbang-it/src/test/java/org/apache/camel/dsl/jbang/it/InfrastructureITCase.java @@ -37,9 +37,8 @@ public class InfrastructureITCase extends JBangTestSupport { private static final String IMPLEMENTATION = "amqp"; /** - * Stops the services started by the tests, even when a test failed before reaching its own stop command. A leaked - * service cascades into the following tests, because {@code infra ps} lists a single row per alias and would keep - * reporting the stale PID. + * Stops the services started by the tests, even when a test failed before reaching its own stop command, so a + * leaked container does not keep holding its port and resources for the following tests. */ @AfterEach public void stopInfraServices() {
