This is an automated email from the ASF dual-hosted git repository.
Croway pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/camel.git
The following commit(s) were added to refs/heads/main by this push:
new c9bbd1d4290d CAMEL-25555: camel-jbang - camel validate normalize
--output <dir> writes one file per input file, so a top-level onException keeps
its scope (#27818)
c9bbd1d4290d is described below
commit c9bbd1d4290d3935baad90e6277a84aa301f1b0b
Author: Federico Mariani <[email protected]>
AuthorDate: Wed Oct 14 14:04:17 2026 +0200
CAMEL-25555: camel-jbang - camel validate normalize --output <dir> writes
one file per input file, so a top-level onException keeps its scope (#27818)
* CAMEL-25555: camel-jbang - camel validate normalize --output <dir> writes
one file per input file, so a top-level onException keeps its scope
Co-Authored-By: Claude Opus 5.5 <[email protected]>
* CAMEL-25555: unique output names, the warning on stderr, no in-place
rewrite of an input file, upgrade note in 4.24
Co-Authored-By: Claude Opus 5.5 <[email protected]>
---------
Co-authored-by: Claude Opus 5.5 <[email protected]>
---
.../ROOT/pages/camel-4x-upgrade-guide-4_24.adoc | 18 ++
.../modules/ROOT/pages/camel-jbang-validation.adoc | 42 ++++
.../camel-jbang-validate-normalize.adoc | 2 +-
.../camel-jbang-plugin-commands-metadata.json | 2 +-
.../validate/NormalizeDumpRoutesStrategy.java | 95 +++++++
.../commands/validate/YamlNormalizeCommand.java | 273 +++++++++++++++++++--
.../validate/YamlNormalizePerFileTest.java | 257 +++++++++++++++++++
7 files changed, 668 insertions(+), 21 deletions(-)
diff --git
a/docs/user-manual/modules/ROOT/pages/camel-4x-upgrade-guide-4_24.adoc
b/docs/user-manual/modules/ROOT/pages/camel-4x-upgrade-guide-4_24.adoc
index afd11a244abb..baddc8dc32d6 100644
--- a/docs/user-manual/modules/ROOT/pages/camel-4x-upgrade-guide-4_24.adoc
+++ b/docs/user-manual/modules/ROOT/pages/camel-4x-upgrade-guide-4_24.adoc
@@ -29,6 +29,24 @@ that a route sends to and no route consumes, so they report
the same endpoints o
So `camel validate` can report an endpoint it did not report before.
+=== camel-jbang - camel validate normalize writes one file per input file to
an output directory
+
+`camel validate normalize <files> --output=<dir>` now writes one normalized
file per input file into the
+directory, with the name of the input file, instead of merging all the routes
into `normalized.yaml`. A
+top-level `onException` of a file then keeps applying to the routes of that
file only, as it does in the
+source. This applies to a single input file too: `camel validate normalize
orders.yaml --output=out/` now
+writes `out/orders.yaml` instead of `out/normalized.yaml`. `normalized.yaml`
is only written for what was
+not loaded from one of the input files. Input files with the same name in
different directories are
+written as `<parent directory>-<name>`. Scripts that read
`<dir>/normalized.yaml` must read the per-file
+outputs instead.
+
+The command now refuses to write over one of its input files (for example
`--output=.` in the directory of
+the routes) and exits with an error.
+
+With `--output=<file>` or without `--output`, the routes are still merged, and
the command now warns, on
+the standard error, when the top-level `onException` of one file applies to
the routes of the other files
+in the merged output.
+
=== camel-jpa, camel-hazelcast - an exchange marked rollback only is not
consumed
When a route marks the exchange rollback only (for example with
`markRollbackOnly()` or `markRollbackOnlyLast()`)
diff --git a/docs/user-manual/modules/ROOT/pages/camel-jbang-validation.adoc
b/docs/user-manual/modules/ROOT/pages/camel-jbang-validation.adoc
index 450969d5bb6c..92459f1110d7 100644
--- a/docs/user-manual/modules/ROOT/pages/camel-jbang-validation.adoc
+++ b/docs/user-manual/modules/ROOT/pages/camel-jbang-validation.adoc
@@ -115,6 +115,48 @@ Without `--output` the result is printed. The routes are
loaded (not run) to wri
placeholders are kept as they are. Use `camel validate yaml --canonical` to
report the shorthands instead
of rewriting them.
+=== Normalizing several files
+
+When `--output` is a directory (an existing one, or a name ending with `/`),
each input file is normalized
+into a file of its own in that directory, with the name of the input file:
+
+[source,bash]
+----
+camel validate normalize a.yaml b.yaml --output=out/
+----
+
+writes `out/a.yaml` and `out/b.yaml`. A top-level `onException` applies to the
routes of its own file only,
+and so it does in the normalized files. A Kamelet is written back to a file of
its own name too, with only
+its template normalized.
+
+The files of the output directory are named as follows:
+
+* an input file keeps its name;
+* input files with the same name in different directories get the name of
their parent directory in front:
+`orders/routes.yaml` and `billing/routes.yaml` are written as
`orders-routes.yaml` and `billing-routes.yaml`;
+* when these names are the same too (`x/orders/routes.yaml` and
`y/orders/routes.yaml`), the input files are
+numbered in the order they are given, before the extension: `routes-1.yaml`,
`routes-2.yaml`
+(`action-1.kamelet.yaml` for `action.kamelet.yaml`);
+* a generated name is never the name of another input file or of another
output file: a number already in use is
+skipped (with an input file named `routes-1.yaml`, the two `routes.yaml`
become `routes-2.yaml` and `routes-3.yaml`).
+
+Beans, data formats, rests, route configurations and route templates go to the
file they were loaded from.
+What was not loaded from one of the input files goes to `normalized.yaml` in
the output directory
+(`normalized-1.yaml` if an input file is named `normalized.yaml`).
+
+The command refuses to write over one of its input files, for example with
`--output=.` in the directory of the
+routes: the normalized files go to another directory.
+
+With `--output` set to a file, or without `--output`, the routes of all the
files are merged into one
+YAML document. A top-level `onException` of one file then applies to the
routes of all the files when
+the result is loaded, so the command prints a warning to the standard error
when this happens (the
+standard output keeps only the YAML, so it can be redirected to a file):
+
+[source,text]
+----
+WARN: the top-level onException of b.yaml applies to the routes of a.yaml too,
as the routes of all the files are written together: use --output with a
directory to write one file per input file
+----
+
== Validating against another Camel version or runtime
Both commands validate against the CLI's own Camel version. A project that
runs another version, or Camel
diff --git
a/docs/user-manual/modules/ROOT/pages/jbang-commands/camel-jbang-validate-normalize.adoc
b/docs/user-manual/modules/ROOT/pages/jbang-commands/camel-jbang-validate-normalize.adoc
index d3a9c09b5f01..7b056a051e98 100644
---
a/docs/user-manual/modules/ROOT/pages/jbang-commands/camel-jbang-validate-normalize.adoc
+++
b/docs/user-manual/modules/ROOT/pages/jbang-commands/camel-jbang-validate-normalize.adoc
@@ -23,7 +23,7 @@ camel validate normalize [options]
| Option | Description | Default | Type
| `--download` | Whether to allow automatic downloading JAR dependencies (over
the internet) | true | boolean
| `--fresh` | Make sure we use fresh (i.e. non-cached) resources | false |
boolean
-| `--output` | File or directory to write normalized output. If not specified,
output is printed to console. | | String
+| `--output` | File or directory to write normalized output. If not specified,
output is printed to console. A directory (an existing one, or a name ending
with /) gets one file per input file, with the name of the input file; input
files with the same name are written as <parent directory>-<name> (or
<name>-<n>, with the first number no other file uses, when that clashes too),
and what was not loaded from an input file goes to normalized.yaml. A file gets
the routes of all the input fil [...]
| `--repo,--repos` | Additional maven repositories for download on-demand (Use
commas to separate multiple repositories) | | String
| `-h,--help` | Display the help and sub-commands | | boolean
|===
diff --git
a/dsl/camel-jbang/camel-jbang-plugin-validate/src/generated/resources/META-INF/camel-jbang-plugin-commands-metadata.json
b/dsl/camel-jbang/camel-jbang-plugin-validate/src/generated/resources/META-INF/camel-jbang-plugin-commands-metadata.json
index b881cb6fe2f0..95a5aa20a223 100644
---
a/dsl/camel-jbang/camel-jbang-plugin-validate/src/generated/resources/META-INF/camel-jbang-plugin-commands-metadata.json
+++
b/dsl/camel-jbang/camel-jbang-plugin-validate/src/generated/resources/META-INF/camel-jbang-plugin-commands-metadata.json
@@ -1,5 +1,5 @@
{
"commands": [
- { "name": "validate", "fullName": "validate", "description": "Validate
Camel source code", "sourceClass":
"org.apache.camel.dsl.jbang.core.commands.validate.ValidateCommand", "plugin":
"validate", "options": [ { "names": "-h,--help", "description": "Display the
help and sub-commands", "javaType": "boolean", "type": "boolean" } ],
"subcommands": [ { "name": "normalize", "fullName": "validate normalize",
"description": "Normalize YAML routes to canonical (explicit) form",
"sourceClass" [...]
+ { "name": "validate", "fullName": "validate", "description": "Validate
Camel source code", "sourceClass":
"org.apache.camel.dsl.jbang.core.commands.validate.ValidateCommand", "plugin":
"validate", "options": [ { "names": "-h,--help", "description": "Display the
help and sub-commands", "javaType": "boolean", "type": "boolean" } ],
"subcommands": [ { "name": "normalize", "fullName": "validate normalize",
"description": "Normalize YAML routes to canonical (explicit) form",
"sourceClass" [...]
]
}
diff --git
a/dsl/camel-jbang/camel-jbang-plugin-validate/src/main/java/org/apache/camel/dsl/jbang/core/commands/validate/NormalizeDumpRoutesStrategy.java
b/dsl/camel-jbang/camel-jbang-plugin-validate/src/main/java/org/apache/camel/dsl/jbang/core/commands/validate/NormalizeDumpRoutesStrategy.java
new file mode 100644
index 000000000000..198827754764
--- /dev/null
+++
b/dsl/camel-jbang/camel-jbang-plugin-validate/src/main/java/org/apache/camel/dsl/jbang/core/commands/validate/NormalizeDumpRoutesStrategy.java
@@ -0,0 +1,95 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.camel.dsl.jbang.core.commands.validate;
+
+import java.nio.file.Path;
+import java.util.Map;
+
+import org.apache.camel.impl.DefaultDumpRoutesStrategy;
+import org.apache.camel.spi.Resource;
+
+/**
+ * Dumps the routes for {@code camel validate normalize}: first one file per
input file into a directory (each file
+ * named as the command says), then all the routes merged into one file. The
merged file is written last, so once it
+ * exists the dump is complete.
+ */
+class NormalizeDumpRoutesStrategy extends DefaultDumpRoutesStrategy {
+
+ private final Path directory;
+ private final Path merged;
+ private final Map<Path, String> names;
+ private final String unattributed;
+ private boolean perFile;
+
+ /**
+ * @param directory the directory to dump one file per input file into
+ * @param merged the file to dump all the routes into
+ * @param names the file name in the directory of each input file,
by its absolute normalized path
+ * @param unattributed the file name in the directory for what was not
loaded from one of the input files
+ */
+ NormalizeDumpRoutesStrategy(Path directory, Path merged, Map<Path, String>
names, String unattributed) {
+ this.directory = directory;
+ this.merged = merged;
+ this.names = names;
+ this.unattributed = unattributed;
+ }
+
+ @Override
+ public void setOutput(String output) {
+ // the outputs are set by dumpRoutes: once the output is set to a
file, it stays a file, so the output of the
+ // main configuration (camel.main.dumpRoutesOutput) is ignored
+ }
+
+ @Override
+ public void dumpRoutes(String format) {
+ // the per-file dump first, as it needs a directory:
DefaultDumpRoutesStrategy calls resolveFileName per resource
+ // only while its outputFileName is null, and setOutput(file) sets it
for good. This holds because dumpRoutes
+ // runs once per normalize (a new strategy per command run), so the
per-file pass always comes first
+ perFile = true;
+ super.setOutput(directory.toString());
+ super.dumpRoutes(format);
+ perFile = false;
+ super.setOutput(merged.toString());
+ super.dumpRoutes(format);
+ }
+
+ @Override
+ protected String resolveFileName(String ext, Resource resource) {
+ if (!perFile) {
+ return super.resolveFileName(ext, resource);
+ }
+ Path source = sourceOf(resource);
+ String name = source != null ? names.get(source) : null;
+ return name != null ? name : unattributed;
+ }
+
+ /**
+ * The absolute normalized path of the file a resource was loaded from, or
null when it was not loaded from a file
+ */
+ static Path sourceOf(Resource resource) {
+ String location = resource != null ? resource.getLocation() : null;
+ if (location == null || !location.startsWith("file:")) {
+ return null;
+ }
+ location = location.startsWith("file://") ? location.substring(7) :
location.substring(5);
+ try {
+ return Path.of(location).toAbsolutePath().normalize();
+ } catch (Exception e) {
+ return null;
+ }
+ }
+}
diff --git
a/dsl/camel-jbang/camel-jbang-plugin-validate/src/main/java/org/apache/camel/dsl/jbang/core/commands/validate/YamlNormalizeCommand.java
b/dsl/camel-jbang/camel-jbang-plugin-validate/src/main/java/org/apache/camel/dsl/jbang/core/commands/validate/YamlNormalizeCommand.java
index 9e382c026650..f8d297c9f03b 100644
---
a/dsl/camel-jbang/camel-jbang-plugin-validate/src/main/java/org/apache/camel/dsl/jbang/core/commands/validate/YamlNormalizeCommand.java
+++
b/dsl/camel-jbang/camel-jbang-plugin-validate/src/main/java/org/apache/camel/dsl/jbang/core/commands/validate/YamlNormalizeCommand.java
@@ -16,25 +16,33 @@
*/
package org.apache.camel.dsl.jbang.core.commands.validate;
+import java.io.File;
+import java.io.IOException;
import java.io.InputStream;
+import java.io.PrintStream;
import java.nio.file.Files;
import java.nio.file.Path;
import java.util.ArrayList;
+import java.util.Collection;
import java.util.HashSet;
import java.util.LinkedHashMap;
+import java.util.LinkedHashSet;
import java.util.List;
import java.util.Locale;
import java.util.Map;
import java.util.Set;
import java.util.Stack;
+import org.apache.camel.CamelContext;
import org.apache.camel.dsl.jbang.core.commands.CamelCommand;
import org.apache.camel.dsl.jbang.core.commands.CamelJBangMain;
import org.apache.camel.dsl.jbang.core.commands.MavenResolverMixin;
import org.apache.camel.dsl.jbang.core.commands.Run;
import org.apache.camel.dsl.jbang.core.common.CamelJBangConstants;
-import org.apache.camel.dsl.jbang.core.common.CommandLineHelper;
+import org.apache.camel.main.BaseMainSupport;
import org.apache.camel.main.KameletMain;
+import org.apache.camel.main.MainListenerSupport;
+import org.apache.camel.spi.DumpRoutesStrategy;
import org.apache.camel.util.FileUtil;
import org.apache.camel.util.IOHelper;
import org.apache.camel.util.StopWatch;
@@ -45,12 +53,24 @@ import picocli.CommandLine;
public class YamlNormalizeCommand extends CamelCommand {
private static final String IGNORE_FILE = "application";
+ // the top-level elements of a file that apply to the routes of that file
only
+ private static final Set<String> SCOPED = Set.of("onException",
"errorHandler", "onCompletion", "intercept",
+ "interceptFrom", "interceptSendToEndpoint");
@CommandLine.Mixin
MavenResolverMixin mavenResolver;
+ // the warnings go to stderr, so that stdout has only the normalized YAML
+ PrintStream stderr = System.err;
+
@CommandLine.Option(names = { "--output" },
- description = "File or directory to write normalized
output. If not specified, output is printed to console.")
+ description = "File or directory to write normalized
output. If not specified, output is printed to console."
+ + " A directory (an existing one, or a
name ending with /) gets one file per input file,"
+ + " with the name of the input file;
input files with the same name are written as"
+ + " <parent directory>-<name> (or
<name>-<n>, with the first number no other file uses,"
+ + " when that clashes too), and what was
not loaded from an input file goes to"
+ + " normalized.yaml. A file gets the
routes of all the input files merged."
+ + " The output must not overwrite one of
the input files.")
private String output;
@CommandLine.Parameters(description = { "The Camel YAML source files to
normalize." },
@@ -84,9 +104,47 @@ public class YamlNormalizeCommand extends CamelCommand {
}
}
- String dump = CommandLineHelper.CAMEL_JBANG_WORK_DIR +
"/normalize-output.yaml";
- Files.deleteIfExists(Path.of(dump));
- final String target = dump;
+ // the output must not replace an input file
+ if (output != null) {
+ Path outPath = Path.of(output);
+ List<Path> targets = new ArrayList<>();
+ if (isDirectory(outPath)) {
+ Map<Path, String> names = outputNames(matched);
+ for (String name : names.values()) {
+ targets.add(outPath.resolve(name));
+ }
+ targets.add(outPath.resolve(unattributedName(names.values())));
+ } else {
+ targets.add(outPath);
+ }
+ for (Path t : targets) {
+ String input = inputAt(t, matched);
+ if (input != null) {
+ printer().printErr("--output would overwrite the input
file " + input + " with " + t
+ + ": write the normalized files to
another file or directory");
+ return 1;
+ }
+ }
+ }
+
+ // a work directory of its own, outside the one of camel run (which
run empties when it starts), so that two
+ // normalizations at the same time do not mix their dumps
+ Path workDir = Files.createTempDirectory("camel-normalize-");
+ try {
+ return normalize(matched, kamelets, workDir);
+ } finally {
+ FileUtil.removeDir(workDir.toFile());
+ }
+ }
+
+ private Integer normalize(List<String> matched, Map<String, String>
kamelets, Path workDir) throws Exception {
+ // one file per input file (the top-level onException of a file
applies to the routes of that file only),
+ // and all the routes merged into one file
+ Map<Path, String> names = outputNames(matched);
+ String unattributed = unattributedName(names.values());
+ Path perFileDir = workDir.resolve("files");
+ Path target = workDir.resolve("normalized.yaml");
+ NormalizeDumpRoutesStrategy strategy = new
NormalizeDumpRoutesStrategy(perFileDir, target, names, unattributed);
Run run = new Run(getMain()) {
@Override
@@ -97,11 +155,18 @@ public class YamlNormalizeCommand extends CamelCommand {
main.addInitialProperty("camel.main.dumpRoutesLog", "false");
main.addInitialProperty("camel.main.dumpRoutesResolvePlaceholders", "false");
main.addInitialProperty("camel.main.dumpRoutesUriAsParameters", "true");
- main.addInitialProperty("camel.main.dumpRoutesOutput", target);
main.addInitialProperty("camel.debug.enabled", "false");
main.addInitialProperty(CamelJBangConstants.TRANSFORM, "true");
main.addInitialProperty("camel.component.properties.ignoreMissingProperty",
"true");
main.addInitialProperty("camel.language.bean.validate",
"false");
+ main.addMainListener(new MainListenerSupport() {
+ @Override
+ public void beforeConfigure(BaseMainSupport main) {
+ CamelContext context = main.getCamelContext();
+ strategy.setCamelContext(context);
+
context.getCamelContextExtension().addContextPlugin(DumpRoutesStrategy.class,
strategy);
+ }
+ });
}
};
run.mavenResolver = mavenResolver;
@@ -112,7 +177,7 @@ public class YamlNormalizeCommand extends CamelCommand {
return exit;
}
- String normalized = waitForDumpFile(Path.of(target));
+ String normalized = waitForDumpFile(target);
if (normalized == null) {
printer().printErr("Error normalizing files: nothing was loaded
from " + String.join(", ", matched)
+ " (see the errors above)");
@@ -141,27 +206,55 @@ public class YamlNormalizeCommand extends CamelCommand {
}
String routes = KameletNormalizer.withoutTemplates(normalized,
kameletIds);
+ // what was dumped for each input file that is not a Kamelet
+ Map<String, String> perFile = new LinkedHashMap<>();
+ for (String n : matched) {
+ if (!kamelets.containsKey(n)) {
+ String text =
readFile(perFileDir.resolve(names.get(sourcePath(n))).toString());
+ if (text != null) {
+ perFile.put(n, KameletNormalizer.withoutTemplates(text,
kameletIds));
+ }
+ }
+ }
+
+ Path outPath = output != null ? Path.of(output) : null;
+ if (outPath != null && isDirectory(outPath)) {
+ Files.createDirectories(outPath);
+ for (Map.Entry<String, String> e : perFile.entrySet()) {
+ if (!e.getValue().isBlank()) {
+
Files.writeString(outPath.resolve(names.get(sourcePath(e.getKey()))),
e.getValue());
+ }
+ }
+ for (Map.Entry<String, String> e : kameletOutput.entrySet()) {
+
Files.writeString(outPath.resolve(names.get(sourcePath(e.getKey()))),
e.getValue());
+ }
+ // what was not loaded from one of the input files
+ String other =
readFile(perFileDir.resolve(unattributed).toString());
+ if (other != null) {
+ other = KameletNormalizer.withoutTemplates(other, kameletIds);
+ if (!other.isBlank()) {
+ Files.writeString(outPath.resolve(unattributed), other);
+ }
+ }
+ printer().println("Normalized " + matched.size() + " file(s) to "
+ output);
+ return 0;
+ }
+
+ // the routes of all the files are written together: a top-level
onException (and the like) of one file then
+ // applies to the routes of the other files
+ warnIfScopeWidens(perFile);
List<String> documents = new ArrayList<>();
if (!routes.isBlank()) {
documents.add(routes);
}
documents.addAll(kameletOutput.values());
- if (output != null) {
- Path outPath = Path.of(output);
- if (Files.isDirectory(outPath)) {
- if (!routes.isBlank()) {
- Files.writeString(outPath.resolve("normalized.yaml"),
routes);
- }
- for (Map.Entry<String, String> e : kameletOutput.entrySet()) {
-
Files.writeString(outPath.resolve(Path.of(e.getKey()).getFileName()),
e.getValue());
- }
- } else if (documents.size() > 1 && !kameletOutput.isEmpty()) {
+ if (outPath != null) {
+ if (documents.size() > 1 && !kameletOutput.isEmpty()) {
printer().printErr("--output must be a directory to normalize
Kamelets with other files:"
+ " each Kamelet is written to a file of
its own name");
return 1;
- } else {
- Files.writeString(outPath, documents.get(0));
}
+ Files.writeString(outPath, documents.get(0));
printer().println("Normalized " + matched.size() + " file(s) to "
+ output);
} else {
// more than one document: one YAML stream, separated by ---
@@ -171,6 +264,148 @@ public class YamlNormalizeCommand extends CamelCommand {
return 0;
}
+ /**
+ * Whether the output is a directory: an existing one, or a name ending
with /
+ */
+ private boolean isDirectory(Path outPath) {
+ return Files.isDirectory(outPath) || output.endsWith("/") ||
output.endsWith(File.separator);
+ }
+
+ /**
+ * Warns when the top-level onException, errorHandler, onCompletion or
intercept of an input file ends up applying
+ * to the routes of the other input files, as the routes of all the files
are written together.
+ */
+ private void warnIfScopeWidens(Map<String, String> perFile) {
+ Set<String> withRoutes = new LinkedHashSet<>();
+ for (Map.Entry<String, String> e : perFile.entrySet()) {
+ if (!topLevel(e.getValue(), Set.of("route")).isEmpty()) {
+ withRoutes.add(e.getKey());
+ }
+ }
+ for (Map.Entry<String, String> e : perFile.entrySet()) {
+ Set<String> kinds = topLevel(e.getValue(), SCOPED);
+ Set<String> others = new LinkedHashSet<>(withRoutes);
+ others.remove(e.getKey());
+ if (!kinds.isEmpty() && !others.isEmpty()) {
+ // to stderr: without --output the normalized YAML is printed
to stdout
+ stderr.println("WARN: the top-level " + String.join(", ",
kinds) + " of " + e.getKey()
+ + " applies to the routes of " + String.join(",
", others)
+ + " too, as the routes of all the files are
written together:"
+ + " use --output with a directory to write one
file per input file");
+ }
+ }
+ }
+
+ /**
+ * The kinds of the top-level elements of a YAML dump, among those given
+ */
+ static Set<String> topLevel(String yaml, Set<String> kinds) {
+ Set<String> answer = new LinkedHashSet<>();
+ for (String line : yaml.split("\n")) {
+ if (line.startsWith("- ") && line.endsWith(":")) {
+ String kind = line.substring(2, line.length() - 1).trim();
+ if (kinds.contains(kind)) {
+ answer.add(kind);
+ }
+ }
+ }
+ return answer;
+ }
+
+ /**
+ * The file name of each input file in an output directory: its own name,
or, when several input files have the same
+ * name, the name of their parent directory, a dash and their name ({@code
orders-routes.yaml}), or, when these
+ * clash too, their name with a number before the extension ({@code
routes-1.yaml}, {@code action-1.kamelet.yaml},
+ * as Camel picks the loader of a file by all its extensions). Every name
is allocated against the names of all the
+ * input files and the names already allocated, so no two input files get
the same name and no generated name is the
+ * name of another input file.
+ */
+ static Map<Path, String> outputNames(List<String> files) {
+ Map<String, List<Path>> byName = new LinkedHashMap<>();
+ for (String f : files) {
+ Path p = sourcePath(f);
+ List<Path> same =
byName.computeIfAbsent(p.getFileName().toString(), k -> new ArrayList<>());
+ if (!same.contains(p)) {
+ same.add(p);
+ }
+ }
+ // the names of the input files are reserved: a generated name must
not be one of them
+ Set<String> taken = new HashSet<>(byName.keySet());
+ Map<Path, String> answer = new LinkedHashMap<>();
+ for (Map.Entry<String, List<Path>> e : byName.entrySet()) {
+ if (e.getValue().size() == 1) {
+ answer.put(e.getValue().get(0), e.getKey());
+ }
+ }
+ for (Map.Entry<String, List<Path>> e : byName.entrySet()) {
+ String name = e.getKey();
+ List<Path> same = e.getValue();
+ if (same.size() == 1) {
+ continue;
+ }
+ Set<String> prefixed = new HashSet<>();
+ for (Path p : same) {
+ prefixed.add(withParent(p));
+ }
+ boolean unique = prefixed.size() == same.size() &&
prefixed.stream().noneMatch(taken::contains);
+ int n = 1;
+ for (Path p : same) {
+ String answerName;
+ if (unique) {
+ answerName = withParent(p);
+ } else {
+ do {
+ answerName = FileUtil.onlyName(name, false) + "-" +
n++ + "." + FileUtil.onlyExt(name, false);
+ } while (taken.contains(answerName));
+ }
+ taken.add(answerName);
+ answer.put(p, answerName);
+ }
+ }
+ return answer;
+ }
+
+ private static String withParent(Path p) {
+ Path parent = p.getParent() != null ? p.getParent().getFileName() :
null;
+ return parent != null ? parent + "-" + p.getFileName() :
p.getFileName().toString();
+ }
+
+ /**
+ * The file name in an output directory for what was not loaded from one
of the input files
+ */
+ static String unattributedName(Collection<String> names) {
+ String answer = "normalized.yaml";
+ for (int i = 1; names.contains(answer); i++) {
+ answer = "normalized-" + i + ".yaml";
+ }
+ return answer;
+ }
+
+ /**
+ * The input file that a path of the output is, or null when it is none of
them
+ */
+ private static String inputAt(Path target, List<String> inputs) {
+ if (!Files.exists(target)) {
+ return null;
+ }
+ for (String n : inputs) {
+ Path input = sourcePath(n);
+ try {
+ if (Files.exists(input) && Files.isSameFile(target, input)) {
+ return n;
+ }
+ } catch (IOException e) {
+ // not the same file
+ }
+ }
+ return null;
+ }
+
+ private static Path sourcePath(String name) {
+ String n = name.startsWith("file:") ? name.substring(5) : name;
+ return Path.of(n).toAbsolutePath().normalize();
+ }
+
private static String readFile(String name) {
try {
String n = name.startsWith("file:") ? name.substring(5) : name;
diff --git
a/dsl/camel-jbang/camel-jbang-plugin-validate/src/test/java/org/apache/camel/dsl/jbang/core/commands/validate/YamlNormalizePerFileTest.java
b/dsl/camel-jbang/camel-jbang-plugin-validate/src/test/java/org/apache/camel/dsl/jbang/core/commands/validate/YamlNormalizePerFileTest.java
new file mode 100644
index 000000000000..efd97e4c2fbe
--- /dev/null
+++
b/dsl/camel-jbang/camel-jbang-plugin-validate/src/test/java/org/apache/camel/dsl/jbang/core/commands/validate/YamlNormalizePerFileTest.java
@@ -0,0 +1,257 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.camel.dsl.jbang.core.commands.validate;
+
+import java.io.ByteArrayOutputStream;
+import java.io.PrintStream;
+import java.nio.charset.StandardCharsets;
+import java.nio.file.Files;
+import java.nio.file.Path;
+import java.util.List;
+import java.util.Map;
+import java.util.Set;
+
+import org.apache.camel.dsl.jbang.core.commands.CamelJBangMain;
+import org.apache.camel.dsl.jbang.core.common.StringPrinter;
+import org.junit.jupiter.api.BeforeEach;
+import org.junit.jupiter.api.Test;
+import org.junit.jupiter.api.io.TempDir;
+import org.junit.jupiter.api.parallel.Isolated;
+import picocli.CommandLine;
+
+import static org.assertj.core.api.Assertions.assertThat;
+
+/**
+ * CAMEL-25555: <code>camel validate normalize --output <dir></code>
writes one file per input file, so the
+ * top-level onException of a file keeps applying to the routes of that file
only.
+ */
+// camel run empties its work directory (.camel-jbang) when it starts, so two
normalizations must not run at the same time
+@Isolated
+class YamlNormalizePerFileTest {
+
+ private static final String A = """
+ - route:
+ id: fromA
+ from:
+ uri: timer:a
+ steps:
+ - log:
+ message: "A"
+ """;
+
+ private static final String B = """
+ - onException:
+ exception:
+ - java.lang.Exception
+ handled:
+ constant: "true"
+ steps:
+ - log:
+ message: "B failed"
+ - route:
+ id: fromB
+ from:
+ uri: timer:b
+ steps:
+ - log:
+ message: "B"
+ """;
+
+ @TempDir
+ Path dir;
+
+ private StringPrinter printer;
+ private String stderr;
+ private Path a;
+ private Path b;
+
+ @BeforeEach
+ void writeInputs() throws Exception {
+ a = Files.writeString(dir.resolve("a.yaml"), A);
+ b = Files.writeString(dir.resolve("b.yaml"), B);
+ }
+
+ private int normalize(String... args) throws Exception {
+ printer = new StringPrinter();
+ YamlNormalizeCommand cmd = new YamlNormalizeCommand(new
CamelJBangMain().withPrinter(printer));
+ CommandLine.populateCommand(cmd, args);
+ ByteArrayOutputStream err = new ByteArrayOutputStream();
+ cmd.stderr = new PrintStream(err, true, StandardCharsets.UTF_8);
+ int answer = cmd.doCall();
+ stderr = err.toString(StandardCharsets.UTF_8);
+ return answer;
+ }
+
+ private static String kamelet(String name) {
+ return """
+ apiVersion: camel.apache.org/v1
+ kind: Kamelet
+ metadata:
+ name: %s
+ spec:
+ definition:
+ title: %s
+ template:
+ from:
+ uri: kamelet:source
+ steps:
+ - setBody:
+ constant: %s
+ """.formatted(name, name, name);
+ }
+
+ private static String validate(Path... files) throws Exception {
+ StringPrinter out = new StringPrinter();
+ YamlValidateCommand cmd = new YamlValidateCommand(new
CamelJBangMain().withPrinter(out));
+ String[] args = new String[files.length];
+ for (int i = 0; i < files.length; i++) {
+ args[i] = files[i].toString();
+ }
+ CommandLine.populateCommand(cmd, args);
+ cmd.doCall();
+ return out.getOutput();
+ }
+
+ @Test
+ void aDirectoryGetsOneFilePerInputFile() throws Exception {
+ Path out = Files.createDirectory(dir.resolve("out"));
+ assertThat(normalize("--output=" + out, a.toString(),
b.toString())).isZero();
+
+ assertThat(out.resolve("normalized.yaml")).doesNotExist();
+ String na = Files.readString(out.resolve("a.yaml"));
+ String nb = Files.readString(out.resolve("b.yaml"));
+ assertThat(na).contains("id:
fromA").doesNotContain("onException").doesNotContain("fromB");
+ assertThat(nb).contains("id: fromB").contains("-
onException:").contains("B failed").doesNotContain("fromA");
+ // the onException stays top-level, before the route of its file
+ assertThat(nb.indexOf("- onException:")).isLessThan(nb.indexOf("-
route:"));
+ assertThat(printer.getOutput()).doesNotContain("WARN");
+
+ // and both load as YAML DSL files
+ assertThat(validate(out.resolve("a.yaml"),
out.resolve("b.yaml"))).contains("Validation success (files:2)");
+ }
+
+ @Test
+ void theMergedOutputWarnsThatTheOnExceptionAppliesToTheOtherFiles() throws
Exception {
+ Path out = dir.resolve("merged.yaml");
+ assertThat(normalize("--output=" + out, a.toString(),
b.toString())).isZero();
+
+ String merged = Files.readString(out);
+ assertThat(merged).contains("id: fromA").contains("id:
fromB").contains("- onException:");
+ assertThat(stderr)
+ .contains("WARN: the top-level onException of " + b + "
applies to the routes of " + a + " too")
+ .contains("use --output with a directory");
+ assertThat(validate(out)).contains("Validation success (files:1)");
+ }
+
+ @Test
+ void theConsoleOutputWarnsToo() throws Exception {
+ assertThat(normalize(a.toString(), b.toString())).isZero();
+ assertThat(stderr).contains("WARN: the top-level onException of " + b);
+ // stdout has only the YAML, so that it can be redirected to a file
+ String yaml = printer.getOutput();
+ assertThat(yaml).startsWith("- ").doesNotContain("WARN").contains("id:
fromA").contains("id: fromB");
+ assertThat(validate(Files.writeString(dir.resolve("stdout.yaml"),
yaml))).contains("Validation success (files:1)");
+
+ // a single file: its onException applies to its own routes only
+ assertThat(normalize(b.toString())).isZero();
+ assertThat(stderr).doesNotContain("WARN");
+ assertThat(printer.getOutput()).contains("- onException:");
+ }
+
+ @Test
+ void theFilesWithTheSameNameArePrefixedWithTheirParentDirectory() throws
Exception {
+ Path orders = Files.createDirectory(dir.resolve("orders"));
+ Path billing = Files.createDirectory(dir.resolve("billing"));
+ Path ordersRoutes = Files.writeString(orders.resolve("routes.yaml"),
A);
+ Path billingRoutes = Files.writeString(billing.resolve("routes.yaml"),
B);
+ // a directory that does not exist yet, given with a trailing /
+ Path out = dir.resolve("out");
+
+ assertThat(normalize("--output=" + out + "/", ordersRoutes.toString(),
billingRoutes.toString())).isZero();
+ assertThat(out).isDirectory();
+ assertThat(out.resolve("orders-routes.yaml")).exists();
+ assertThat(out.resolve("billing-routes.yaml")).exists();
+
assertThat(Files.readString(out.resolve("orders-routes.yaml"))).contains("id:
fromA")
+ .doesNotContain("onException");
+
assertThat(Files.readString(out.resolve("billing-routes.yaml"))).contains("id:
fromB").contains("- onException:");
+ }
+
+ @Test
+ void theInputFilesAreNotOverwritten() throws Exception {
+ String before = Files.readString(b);
+ // the directory of the inputs
+ assertThat(normalize("--output=" + dir, a.toString(),
b.toString())).isOne();
+ assertThat(printer.getOutput()).contains("--output would overwrite the
input file " + a);
+ // an input file as the merged output
+ assertThat(normalize("--output=" + b, a.toString(),
b.toString())).isOne();
+ assertThat(printer.getOutput()).contains("--output would overwrite the
input file " + b);
+ assertThat(Files.readString(a)).isEqualTo(A);
+ assertThat(Files.readString(b)).isEqualTo(before);
+ }
+
+ @Test
+ void theGeneratedNamesDoNotReplaceAnotherFile() throws Exception {
+ // review of #27818: the numbered names of two files were the name of
a third file, whose routes got lost
+ Path x = Files.createDirectories(dir.resolve("x/orders"));
+ Path y = Files.createDirectories(dir.resolve("y/orders"));
+ Path kx = Files.writeString(x.resolve("action.kamelet.yaml"),
kamelet("action-x"));
+ Path ky = Files.writeString(y.resolve("action.kamelet.yaml"),
kamelet("action-y"));
+ Path kz = Files.writeString(dir.resolve("action-1.kamelet.yaml"),
kamelet("action-z"));
+ Path rx = Files.writeString(x.resolve("routes.yaml"), A);
+ Path ry = Files.writeString(y.resolve("routes.yaml"), B);
+ Path rc = Files.writeString(dir.resolve("routes-1.yaml"),
A.replace("fromA", "routeC"));
+ Path out = dir.resolve("out");
+
+ assertThat(normalize("--output=" + out + "/", kx.toString(),
ky.toString(), kz.toString(), rx.toString(),
+ ry.toString(), rc.toString())).isZero();
+
assertThat(Files.readString(out.resolve("action-1.kamelet.yaml"))).contains("name:
action-z");
+
assertThat(Files.readString(out.resolve("action-2.kamelet.yaml"))).contains("name:
action-x");
+
assertThat(Files.readString(out.resolve("action-3.kamelet.yaml"))).contains("name:
action-y");
+
assertThat(Files.readString(out.resolve("routes-1.yaml"))).contains("id:
routeC").doesNotContain("fromA");
+
assertThat(Files.readString(out.resolve("routes-2.yaml"))).contains("id:
fromA").doesNotContain("onException");
+
assertThat(Files.readString(out.resolve("routes-3.yaml"))).contains("id:
fromB").contains("- onException:");
+ try (var files = Files.list(out)) {
+ assertThat(files).hasSize(6);
+ }
+ }
+
+ @Test
+ void theOutputNames() {
+ Map<Path, String> names = YamlNormalizeCommand.outputNames(
+ List.of("a.yaml", "x/routes.yaml", "y/routes.yaml",
"p/z/r.yml", "q/z/r.yml", "file:b.yml"));
+ assertThat(names).containsValues("a.yaml", "x-routes.yaml",
"y-routes.yaml", "r-1.yml", "r-2.yml", "b.yml");
+
assertThat(names.get(Path.of("q/z/r.yml").toAbsolutePath())).isEqualTo("r-2.yml");
+
+ // a numbered name is not the name of another input file
+ Map<Path, String> three = YamlNormalizeCommand.outputNames(
+ List.of("x/orders/routes.yaml", "y/orders/routes.yaml",
"routes-1.yaml"));
+ assertThat(three).containsValues("routes-1.yaml", "routes-2.yaml",
"routes-3.yaml");
+
assertThat(three.get(Path.of("routes-1.yaml").toAbsolutePath())).isEqualTo("routes-1.yaml");
+ // the number goes before all the extensions, and every name is
distinct
+ assertThat(YamlNormalizeCommand.outputNames(
+ List.of("x/orders/action.kamelet.yaml",
"y/orders/action.kamelet.yaml", "action.kamelet-1.yaml")))
+ .containsValues("action.kamelet-1.yaml",
"action-1.kamelet.yaml", "action-2.kamelet.yaml");
+ // nor the prefixed name of another input file
+ assertThat(YamlNormalizeCommand.outputNames(List.of("x/routes.yaml",
"y/routes.yaml", "x-routes.yaml")))
+ .containsValues("x-routes.yaml", "routes-1.yaml",
"routes-2.yaml");
+
+
assertThat(YamlNormalizeCommand.unattributedName(Set.of("a.yaml"))).isEqualTo("normalized.yaml");
+
assertThat(YamlNormalizeCommand.unattributedName(Set.of("normalized.yaml",
"normalized-1.yaml")))
+ .isEqualTo("normalized-2.yaml");
+ }
+
+}
