This is an automated email from the ASF dual-hosted git repository. davsclaus pushed a commit to branch CAMEL-26062-smooks-bodies in repository https://gitbox.apache.org/repos/asf/camel.git
commit 26f42da7c76a4c4ffbb28ad4c5b1772da85c668f Author: Claus Ibsen <[email protected]> AuthorDate: Mon Nov 2 16:03:25 2026 +0100 CAMEL-26062: camel-smooks - SAXSource, remote file, File and Path bodies are read, exports come in a fixed order A SAXSource body cast its XMLReader to a Reader, a remote file body cast its FTP/SFTP file to java.io.File, and a File or Path body found no converter to a Smooks Source when the Smooks config declares a reader. A SAXSource is now read from its InputSource (also with allowExternalEntities=false), a remote file from its downloaded content, and a File or Path from its URL. The sinks of several exports are sorted by class name, so the List body has a fixed order. A bean route with a correlation id no longer ignores a failure of its endpoint. Co-Authored-By: Claude Opus 5.5 <[email protected]> Signed-off-by: Claus Ibsen <[email protected]> --- .../camel/catalog/docs/smooks-component.adoc | 4 +- .../src/main/docs/smooks-component.adoc | 4 +- .../camel/component/smooks/SmooksProcessor.java | 69 ++++++-- .../camel/component/smooks/routing/BeanRouter.java | 7 +- .../component/smooks/SmooksReaderBodyTest.java | 177 +++++++++++++++++++++ .../component/smooks/routing/BeanRouterTest.java | 32 ++++ .../ROOT/pages/camel-4x-upgrade-guide-4_24.adoc | 6 + 7 files changed, 286 insertions(+), 13 deletions(-) diff --git a/catalog/camel-catalog/src/generated/resources/org/apache/camel/catalog/docs/smooks-component.adoc b/catalog/camel-catalog/src/generated/resources/org/apache/camel/catalog/docs/smooks-component.adoc index a3568ee63eee..82871803751d 100644 --- a/catalog/camel-catalog/src/generated/resources/org/apache/camel/catalog/docs/smooks-component.adoc +++ b/catalog/camel-catalog/src/generated/resources/org/apache/camel/catalog/docs/smooks-component.adoc @@ -14,7 +14,7 @@ The Camel Smooks component uses https://www.smooks.org/[Smooks] to break up the structured data (EDI, CSV, POJO, etc...) of a Camel message body into fragments. These fragments can be processed independently of one another from within Smooks. -The body is read by the reader of the Smooks configuration (XML by default) as text, bytes, a stream or a file. A POJO is read when the body is a `org.smooks.io.source.JavaSource` that holds it: a POJO, a `Map` or a `List` as the body itself fails. +The body is read by the reader of the Smooks configuration (XML by default) as text, bytes, a stream or a file (a `java.io.File`, a `java.nio.file.Path`, or the file of a file or remote file consumer). A `javax.xml.transform.sax.SAXSource` body is read from its `InputSource`: its `XMLReader` is not used. A POJO is read when the body is a `org.smooks.io.source.JavaSource` that holds it: a POJO, a `Map` or a `List` as the body itself fails. Common applications of Smooks include: @@ -103,6 +103,8 @@ The Smooks component is configured with a mandatory configuration file, which is The `{https://www.smooks.org/xsd/smooks/smooks-core-1.6.xsd}exports` element in this example configures Smooks to export the execution result to Camel as a string. Keep in mind that exporting the result as string means that the whole result will be kept in-memory which could cause unexpected performance issues for large payloads. +When several results are exported, the message body is a `List` of the results, ordered by the class name of their sink type (for example, the result of a `org.smooks.io.sink.JavaSink` comes before the result of a `org.smooks.io.sink.StringSink`). + === Bean routing Smooks is capable of routing fragments to Camel endpoints using the `{https://www.smooks.org/xsd/smooks/camel-1.5.xsd}route` element from the Smooks configuration. As an example, you can route to an endpoint by declaring the following in your Smooks configuration: diff --git a/components/camel-smooks/src/main/docs/smooks-component.adoc b/components/camel-smooks/src/main/docs/smooks-component.adoc index a3568ee63eee..82871803751d 100644 --- a/components/camel-smooks/src/main/docs/smooks-component.adoc +++ b/components/camel-smooks/src/main/docs/smooks-component.adoc @@ -14,7 +14,7 @@ The Camel Smooks component uses https://www.smooks.org/[Smooks] to break up the structured data (EDI, CSV, POJO, etc...) of a Camel message body into fragments. These fragments can be processed independently of one another from within Smooks. -The body is read by the reader of the Smooks configuration (XML by default) as text, bytes, a stream or a file. A POJO is read when the body is a `org.smooks.io.source.JavaSource` that holds it: a POJO, a `Map` or a `List` as the body itself fails. +The body is read by the reader of the Smooks configuration (XML by default) as text, bytes, a stream or a file (a `java.io.File`, a `java.nio.file.Path`, or the file of a file or remote file consumer). A `javax.xml.transform.sax.SAXSource` body is read from its `InputSource`: its `XMLReader` is not used. A POJO is read when the body is a `org.smooks.io.source.JavaSource` that holds it: a POJO, a `Map` or a `List` as the body itself fails. Common applications of Smooks include: @@ -103,6 +103,8 @@ The Smooks component is configured with a mandatory configuration file, which is The `{https://www.smooks.org/xsd/smooks/smooks-core-1.6.xsd}exports` element in this example configures Smooks to export the execution result to Camel as a string. Keep in mind that exporting the result as string means that the whole result will be kept in-memory which could cause unexpected performance issues for large payloads. +When several results are exported, the message body is a `List` of the results, ordered by the class name of their sink type (for example, the result of a `org.smooks.io.sink.JavaSink` comes before the result of a `org.smooks.io.sink.StringSink`). + === Bean routing Smooks is capable of routing fragments to Camel endpoints using the `{https://www.smooks.org/xsd/smooks/camel-1.5.xsd}route` element from the Smooks configuration. As an example, you can route to an endpoint by declaring the following in your Smooks configuration: diff --git a/components/camel-smooks/src/main/java/org/apache/camel/component/smooks/SmooksProcessor.java b/components/camel-smooks/src/main/java/org/apache/camel/component/smooks/SmooksProcessor.java index eefdf46f0391..bacee9499e6b 100644 --- a/components/camel-smooks/src/main/java/org/apache/camel/component/smooks/SmooksProcessor.java +++ b/components/camel-smooks/src/main/java/org/apache/camel/component/smooks/SmooksProcessor.java @@ -19,10 +19,14 @@ package org.apache.camel.component.smooks; import java.io.File; import java.io.IOException; import java.io.InputStream; +import java.io.InputStreamReader; import java.io.Reader; import java.net.MalformedURLException; import java.net.URI; import java.nio.charset.Charset; +import java.nio.file.Path; +import java.util.Arrays; +import java.util.Comparator; import java.util.HashMap; import java.util.HashSet; import java.util.List; @@ -34,6 +38,7 @@ import javax.xml.transform.sax.SAXSource; import org.w3c.dom.Node; +import org.xml.sax.InputSource; import org.xml.sax.SAXException; import org.apache.camel.CamelContext; @@ -153,6 +158,8 @@ public class SmooksProcessor extends ServiceSupport implements Processor, CamelC final Exports exports = smooks.getApplicationContext().getRegistry().lookup(new ExportsLookup()); if (exports.hasExports()) { final Sink[] sinks = exports.createSinks(); + // Smooks creates the sinks in no fixed order: sort them by type so that a List body has a stable order + Arrays.sort(sinks, Comparator.comparing(sink -> sink.getClass().getName())); smooks.filterSource(executionContext, getSource(exchange, executionContext.getContentEncoding()), sinks); setResultOnBody(exports, sinks, exchange); } else { @@ -202,6 +209,19 @@ public class SmooksProcessor extends ServiceSupport implements Processor, CamelC return new DOMSource(node); } + if (payload instanceof SAXSource saxSource) { + // The XMLReader of a SAXSource is a parser, not the input: read the input source that it holds + final Source source = toSource(saxSource, exchange); + if (!hardenXmlInput) { + return source; + } + try { + return SmooksSecuritySupport.secureXmlSource(source, contentEncoding); + } catch (IOException | SAXException e) { + throw new SmooksException(e); + } + } + if (hardenXmlInput) { // Untrusted XML: parse with a reader that does not resolve external entities before Smooks sees it try { @@ -226,24 +246,53 @@ public class SmooksProcessor extends ServiceSupport implements Processor, CamelC return new ReaderSource<>(reader); } - if (payload instanceof WrappedFile) { - final String systemId - = new javax.xml.transform.stream.StreamSource((File) exchange.getIn().getBody(WrappedFile.class).getFile()) - .getSystemId(); - try { - return new URLSource(URI.create(systemId).toURL()); - } catch (MalformedURLException e) { - throw new SmooksException(e); + if (payload instanceof WrappedFile<?> wrappedFile) { + if (wrappedFile.getFile() instanceof File file) { + return toURLSource(file.toURI().toString()); } + // A remote file (ftp, sftp, ftps) is not a java.io.File: read its downloaded content + return new StreamSource<>(exchange.getIn().getMandatoryBody(InputStream.class)); + } + + if (payload instanceof File file) { + return toURLSource(file.toURI().toString()); } - if (payload instanceof SAXSource) { - return new ReaderSource<>((Reader) ((SAXSource) payload).getXMLReader()); + if (payload instanceof Path path) { + return toURLSource(path.toUri().toString()); } return exchange.getIn().getMandatoryBody(Source.class); } + private static Source toSource(final SAXSource saxSource, final Exchange exchange) throws InvalidPayloadException { + final InputSource inputSource = saxSource.getInputSource(); + if (inputSource != null) { + if (inputSource.getCharacterStream() != null) { + return new ReaderSource<>(inputSource.getCharacterStream()); + } + if (inputSource.getByteStream() != null) { + if (inputSource.getEncoding() != null) { + return new ReaderSource<>( + new InputStreamReader(inputSource.getByteStream(), Charset.forName(inputSource.getEncoding()))); + } + return new StreamSource<>(inputSource.getByteStream()); + } + if (inputSource.getSystemId() != null) { + return toURLSource(inputSource.getSystemId()); + } + } + throw new InvalidPayloadException(exchange, Source.class); + } + + private static Source toURLSource(final String systemId) { + try { + return new URLSource(URI.create(systemId).toURL()); + } catch (MalformedURLException | IllegalArgumentException e) { + throw new SmooksException(e); + } + } + public String getSmooksConfig() { return configUri; } diff --git a/components/camel-smooks/src/main/java/org/apache/camel/component/smooks/routing/BeanRouter.java b/components/camel-smooks/src/main/java/org/apache/camel/component/smooks/routing/BeanRouter.java index e07a344ca475..2d29e1f585a2 100644 --- a/components/camel-smooks/src/main/java/org/apache/camel/component/smooks/routing/BeanRouter.java +++ b/components/camel-smooks/src/main/java/org/apache/camel/component/smooks/routing/BeanRouter.java @@ -27,6 +27,7 @@ import javax.inject.Inject; import org.w3c.dom.Element; import org.apache.camel.CamelContext; +import org.apache.camel.Exchange; import org.apache.camel.Message; import org.apache.camel.Processor; import org.apache.camel.ProducerTemplate; @@ -175,7 +176,11 @@ public class BeanRouter implements AfterVisitor, Consumer, PreExecutionLifecycle in.setHeader(correlationIdName.orElse(null), correlationIdPattern.get().apply(FreeMarkerUtils.getMergedModel(executionContext))); }; - producerTemplate.send(toEndpoint, processor); + // send does not throw: a failure of the route is set on the returned exchange + Exchange result = producerTemplate.send(toEndpoint, processor); + if (result.getException() != null) { + throw result.getException(); + } } else { producerTemplate.sendBodyAndHeaders(toEndpoint, bean, Map.of(SmooksConstants.SMOOKS_EXECUTION_CONTEXT, executionContext)); diff --git a/components/camel-smooks/src/test/java/org/apache/camel/component/smooks/SmooksReaderBodyTest.java b/components/camel-smooks/src/test/java/org/apache/camel/component/smooks/SmooksReaderBodyTest.java new file mode 100644 index 000000000000..85e504837779 --- /dev/null +++ b/components/camel-smooks/src/test/java/org/apache/camel/component/smooks/SmooksReaderBodyTest.java @@ -0,0 +1,177 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.camel.component.smooks; + +import java.io.ByteArrayInputStream; +import java.io.File; +import java.io.InputStream; +import java.io.StringReader; +import java.nio.charset.StandardCharsets; +import java.nio.file.Files; +import java.nio.file.Path; +import java.util.List; +import java.util.Map; +import java.util.Set; + +import javax.xml.transform.sax.SAXSource; + +import org.xml.sax.InputSource; + +import org.apache.camel.CamelExecutionException; +import org.apache.camel.InvalidPayloadException; +import org.apache.camel.builder.RouteBuilder; +import org.apache.camel.component.file.GenericFile; +import org.apache.camel.dataformat.smooks.Customer; +import org.apache.camel.test.junit6.CamelTestSupport; +import org.junit.jupiter.api.BeforeEach; +import org.junit.jupiter.api.Test; +import org.junit.jupiter.api.io.TempDir; +import org.smooks.Smooks; +import org.smooks.io.payload.Export; +import org.smooks.io.payload.Exports; +import org.smooks.io.sink.JavaSink; +import org.smooks.io.sink.StringSink; +import org.smooks.support.StreamUtils; +import org.xmlunit.builder.DiffBuilder; + +import static org.junit.jupiter.api.Assertions.assertEquals; +import static org.junit.jupiter.api.Assertions.assertFalse; +import static org.junit.jupiter.api.Assertions.assertInstanceOf; +import static org.junit.jupiter.api.Assertions.assertThrows; + +/** + * Message bodies that are a {@link SAXSource}, a file or a remote file are read by the reader of the Smooks + * configuration. + */ +class SmooksReaderBodyTest extends CamelTestSupport { + + @TempDir + Path tempDir; + + private String orderEdi; + private String expectedOrderXml; + + @BeforeEach + void readOrder() throws Exception { + try (InputStream edi = getClass().getResourceAsStream("/data/order.txt"); + InputStream xml = getClass().getResourceAsStream("/xml/expected-order.xml")) { + orderEdi = new String(StreamUtils.readStream(edi), StandardCharsets.UTF_8); + expectedOrderXml = new String(StreamUtils.readStream(xml), StandardCharsets.UTF_8); + } + } + + @Test + void saxSourceWithCharacterStreamIsReadByTheConfiguredReader() { + assertOrderXml(new SAXSource(new InputSource(new StringReader(orderEdi)))); + } + + @Test + void saxSourceWithByteStreamIsReadByTheConfiguredReader() { + assertOrderXml(new SAXSource(new InputSource(new ByteArrayInputStream(orderEdi.getBytes(StandardCharsets.UTF_8))))); + } + + @Test + void saxSourceWithSystemIdIsReadByTheConfiguredReader() throws Exception { + assertOrderXml(new SAXSource(new InputSource(writeOrder().toUri().toString()))); + } + + @Test + void saxSourceWithoutInputIsRejected() { + CamelExecutionException e = assertThrows(CamelExecutionException.class, + () -> template.requestBody("direct:edi", new SAXSource())); + assertInstanceOf(InvalidPayloadException.class, e.getCause().getCause()); + } + + @Test + void saxSourceIsParsedWithoutExternalEntitiesByTheDefaultReader() throws Exception { + Path entityFile = Files.writeString(tempDir.resolve("first-name.txt"), "Text of the external entity"); + String xml = "<!DOCTYPE customer [<!ENTITY name SYSTEM \"" + entityFile.toUri() + "\">]>" + + "<customer><firstName>&name;</firstName><lastName>Smith</lastName></customer>"; + + Customer customer = template.requestBody("direct:customer", new SAXSource(new InputSource(new StringReader(xml))), + Customer.class); + assertEquals("Smith", customer.getLastName()); + assertEquals("", customer.getFirstName(), "The external entity must not be read by default"); + } + + @Test + void fileBodyIsReadByTheConfiguredReader() throws Exception { + assertOrderXml(writeOrder().toFile()); + } + + @Test + void pathBodyIsReadByTheConfiguredReader() throws Exception { + assertOrderXml(writeOrder()); + } + + @Test + void localFileBodyIsReadByTheConfiguredReader() throws Exception { + GenericFile<File> localFile = new GenericFile<>(); + localFile.setFile(writeOrder().toFile()); + assertOrderXml(localFile); + } + + @Test + void remoteFileBodyIsReadByTheConfiguredReader() { + // A remote file (ftp, sftp, ftps) holds the file of its client library and its downloaded content as body + GenericFile<Object> remoteFile = new GenericFile<>(); + remoteFile.setFile(new Object()); + remoteFile.setFileName("order.txt"); + remoteFile.setBody(orderEdi.getBytes(StandardCharsets.UTF_8)); + assertOrderXml(remoteFile); + } + + @Test + void severalExportsAreReturnedInTheOrderOfTheirSinkType() { + // Smooks creates the sinks in no fixed order, so send several times + for (int i = 0; i < 20; i++) { + List<?> body = template.requestBody("direct:severalExports", "<order><id>1</id></order>", List.class); + assertEquals(2, body.size()); + assertInstanceOf(Map.class, body.get(0), "The result of the JavaSink comes first"); + assertEquals("<order><id>1</id></order>", body.get(1).toString(), "The result of the StringSink comes last"); + } + } + + private Path writeOrder() throws Exception { + return Files.writeString(tempDir.resolve("order.txt"), orderEdi); + } + + private void assertOrderXml(Object body) { + String xml = template.requestBody("direct:edi", body, String.class); + assertFalse(DiffBuilder.compare(expectedOrderXml).withTest(xml).ignoreComments().ignoreWhitespace().build() + .hasDifferences(), xml); + } + + @Override + protected RouteBuilder createRouteBuilder() { + return new RouteBuilder() { + @Override + @SuppressWarnings("resource") + // NOTE: the context will take care to clean resources + public void configure() { + // Without stream caching, as it would turn a SAXSource body into a String before Smooks reads it + from("direct:edi").noStreamCaching().to("smooks://edi-to-xml-smooks-config.xml"); + from("direct:customer").noStreamCaching().to("smooks://smooks-config.xml"); + from("direct:severalExports") + .process(new SmooksProcessor( + new Smooks().setExports( + new Exports(Set.of(new Export(StringSink.class), new Export(JavaSink.class)))), + context)); + } + }; + } +} diff --git a/components/camel-smooks/src/test/java/org/apache/camel/component/smooks/routing/BeanRouterTest.java b/components/camel-smooks/src/test/java/org/apache/camel/component/smooks/routing/BeanRouterTest.java index ab1f9eefc830..75f00ac4479f 100644 --- a/components/camel-smooks/src/test/java/org/apache/camel/component/smooks/routing/BeanRouterTest.java +++ b/components/camel-smooks/src/test/java/org/apache/camel/component/smooks/routing/BeanRouterTest.java @@ -38,6 +38,7 @@ import org.smooks.engine.lookup.LifecycleManagerLookup; import org.smooks.engine.resource.config.DefaultResourceConfig; import org.smooks.testkit.MockApplicationContext; +import static org.junit.jupiter.api.Assertions.assertInstanceOf; import static org.junit.jupiter.api.Assertions.assertNotNull; import static org.junit.jupiter.api.Assertions.assertThrows; import static org.mockito.Mockito.mock; @@ -71,6 +72,37 @@ public class BeanRouterTest extends CamelTestSupport { () -> createBeanRouter(BEAN_ID, END_POINT_URI).visitAfter(null, smooksExecutionContext)); } + @Test + public void testFailureOfEndpointIsThrown() { + endpoint.whenAnyExchangeReceived(exchange -> { + throw new IllegalStateException("Forced"); + }); + SmooksException e = assertThrows(SmooksException.class, + () -> createBeanRouter(BEAN_ID, END_POINT_URI).visitAfter(null, smooksExecutionContext)); + assertInstanceOf(IllegalStateException.class, rootCause(e)); + } + + @Test + public void testFailureOfEndpointWithCorrelationIdIsThrown() { + endpoint.whenAnyExchangeReceived(exchange -> { + throw new IllegalStateException("Forced"); + }); + BeanRouter beanRouter = createBeanRouter(BEAN_ID, END_POINT_URI); + beanRouter.setCorrelationIdName("orderId"); + beanRouter.setCorrelationIdPattern("order-1"); + SmooksException e = assertThrows(SmooksException.class, + () -> beanRouter.visitAfter(null, smooksExecutionContext)); + assertInstanceOf(IllegalStateException.class, rootCause(e)); + } + + private static Throwable rootCause(Throwable throwable) { + Throwable cause = throwable; + while (cause.getCause() != null) { + cause = cause.getCause(); + } + return cause; + } + @Test public void testBeanRouterUsingOnlyBeanId() throws Exception { endpoint.setExpectedMessageCount(1); diff --git a/docs/user-manual/modules/ROOT/pages/camel-4x-upgrade-guide-4_24.adoc b/docs/user-manual/modules/ROOT/pages/camel-4x-upgrade-guide-4_24.adoc index 1bc188419d61..49a1a402025d 100644 --- a/docs/user-manual/modules/ROOT/pages/camel-4x-upgrade-guide-4_24.adoc +++ b/docs/user-manual/modules/ROOT/pages/camel-4x-upgrade-guide-4_24.adoc @@ -1827,6 +1827,12 @@ expanded, and the declaration itself is not passed on to Smooks. Set `allowExternalEntities=true` on the endpoint or on the data format for documents that rely on external entities or an external DTD. +=== camel-smooks - a failure of a bean route with a correlation id fails the Smooks processing + +A `<camel:route>` of the Smooks configuration with a `correlationIdName` and `correlationIdPattern` ignored a failure of +the endpoint it routes the bean to. The failure now fails the Smooks processing with a `SmooksException`, as it already +did for a bean route without a correlation id. + === camel-platform-http-main - management server settings on a shared port When the HTTP server (`camel.server.enabled=true`) and the HTTP management server (`camel.management.enabled=true`)
