This is an automated email from the ASF dual-hosted git repository. davsclaus pushed a commit to branch fix/CAMEL-26115-language-bugs in repository https://gitbox.apache.org/repos/asf/camel.git
commit d0293a2eaacd793bb7fb6e4a90ee81b5767e45a4 Author: Claus Ibsen <[email protected]> AuthorDate: Sat Oct 24 11:22:00 2026 +0200 CAMEL-26115: camel-datasonnet, camel-hl7, camel-openfeature, camel-ognl, camel-python, camel-quickjs, camel-javascript, camel-python3 - small bugs of the languages - python: a script gives the Java value of its result instead of its Python text, and an error while it runs is an evaluation error - openfeature: an expression gives the value of a boolean flag - datasonnet: source is used with a null body, matches() keeps the output media type, a stream without a media type is read as text, and a null result gives null - hl7terser: a stream or a file is read as an HL7 message, and another body fails with an error that says what it is - ognl: the message variable works - quickjs: a Set, a Map and a typed array keep their values in the result - js, python3: bytes give a byte array, and a returned function fails The body types and the doc pages of these languages are updated, and the behavior changes are in the 4.24 upgrade guide. Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]> Signed-off-by: Claus Ibsen <[email protected]> --- .../camel/catalog/docs/datasonnet-language.adoc | 6 +- .../apache/camel/catalog/docs/ognl-language.adoc | 1 + .../camel/catalog/docs/openfeature-language.adoc | 18 +-- .../apache/camel/catalog/docs/python-language.adoc | 5 +- .../camel/catalog/docs/quickjs-language.adoc | 3 +- .../apache/camel/catalog/languages/datasonnet.json | 4 +- .../apache/camel/catalog/languages/hl7terser.json | 2 +- .../org/apache/camel/catalog/languages/js.json | 2 +- .../org/apache/camel/catalog/languages/ognl.json | 2 +- .../camel/catalog/languages/openfeature.json | 2 +- .../org/apache/camel/catalog/languages/python.json | 2 +- .../apache/camel/catalog/languages/python3.json | 2 +- .../apache/camel/catalog/languages/quickjs.json | 2 +- .../camel/language/datasonnet/datasonnet.json | 4 +- .../src/main/docs/datasonnet-language.adoc | 6 +- .../language/datasonnet/DatasonnetExpression.java | 29 +++-- .../language/datasonnet/DatasonnetLanguage.java | 9 +- .../language/datasonnet/DatasonnetBodyTest.java | 135 +++++++++++++++++++++ .../org/apache/camel/component/hl7/hl7terser.json | 2 +- .../camel/component/hl7/Hl7TerserLanguage.java | 49 +++++++- .../component/hl7/TerserExpressionBodyTest.java | 110 +++++++++++++++++ .../META-INF/org/apache/camel/language/js/js.json | 2 +- .../camel/language/js/JavaScriptExpression.java | 2 +- .../camel/language/js/JavaScriptLanguage.java | 38 ++++-- .../js/JavaScriptResultMaterializationTest.java | 30 +++++ .../org/apache/camel/language/ognl/ognl.json | 2 +- .../camel-ognl/src/main/docs/ognl-language.adoc | 1 + .../apache/camel/language/ognl/OgnlLanguage.java | 2 +- .../org/apache/camel/language/ognl/RootObject.java | 4 + .../org/apache/camel/language/ognl/OgnlTest.java | 8 ++ .../camel/language/openfeature/openfeature.json | 2 +- .../src/main/docs/openfeature-language.adoc | 18 +-- .../component/openfeature/OpenFeatureEndpoint.java | 40 +++++- .../language/openfeature/OpenFeatureLanguage.java | 20 +-- .../openfeature/OpenFeatureLanguageTest.java | 105 ++++++++++++++++ .../src/test/resources/log4j2.properties | 29 +++++ .../src/test/resources/openfeature/flags.json | 5 + .../org/apache/camel/language/python/python.json | 2 +- .../src/main/docs/python-language.adoc | 5 +- .../camel/language/python/PythonExpression.java | 48 +++++++- .../camel/language/python/PythonLanguage.java | 14 ++- .../camel/language/python/PythonResultTest.java | 134 ++++++++++++++++++++ .../org/apache/camel/language/python3/python3.json | 2 +- .../camel/language/python3/Python3Language.java | 50 +++++++- .../language/python3/Python3LanguageEvalTest.java | 58 +++++++++ .../org/apache/camel/language/quickjs/quickjs.json | 2 +- .../src/main/docs/quickjs-language.adoc | 3 +- .../camel/language/quickjs/QuickjsHelper.java | 82 ++++++++++++- .../camel/language/quickjs/QuickjsLanguage.java | 5 +- .../language/quickjs/QuickjsSerializationTest.java | 82 +++++++++++++ .../ROOT/pages/camel-4x-upgrade-guide-4_24.adoc | 49 ++++++++ 51 files changed, 1140 insertions(+), 99 deletions(-) diff --git a/catalog/camel-catalog/src/generated/resources/org/apache/camel/catalog/docs/datasonnet-language.adoc b/catalog/camel-catalog/src/generated/resources/org/apache/camel/catalog/docs/datasonnet-language.adoc index 827c8dff23fe..584d60eeac13 100644 --- a/catalog/camel-catalog/src/generated/resources/org/apache/camel/catalog/docs/datasonnet-language.adoc +++ b/catalog/camel-catalog/src/generated/resources/org/apache/camel/catalog/docs/datasonnet-language.adoc @@ -36,7 +36,8 @@ include::partial$language-options.adoc[] === Setting a result type -The xref:datasonnet-language.adoc[DataSonnet] expression will return a `com.datasonnet.document.Document` by default. The +The xref:datasonnet-language.adoc[DataSonnet] expression will return a `com.datasonnet.document.Document` by default, or +`null` when the result is `null`. The document preserves the content type metadata along with the contents of the transformation result. In predicates, however, the Document will be automatically unwrapped and the boolean content will be returned. Similarly, any time you want the content in a specific result type like a String. To do this, you have to instruct the @@ -118,7 +119,8 @@ And for output media type: NOTE: With the body media type `application/x-java-object`, the body is the Java object it is: JSON text is a string, not an object, so `body.foo` fails on it. Set `bodyMediaType`, or a `Content-Type` header, to read text, bytes, a -stream or a file in that format, such as `application/json`, `application/xml` or `application/csv`. +stream or a file in that format, such as `application/json`, `application/xml` or `application/csv`. Without a +media type, a stream is read as text. === Functions diff --git a/catalog/camel-catalog/src/generated/resources/org/apache/camel/catalog/docs/ognl-language.adoc b/catalog/camel-catalog/src/generated/resources/org/apache/camel/catalog/docs/ognl-language.adoc index 19f734145d2c..94d3d714796e 100644 --- a/catalog/camel-catalog/src/generated/resources/org/apache/camel/catalog/docs/ognl-language.adoc +++ b/catalog/camel-catalog/src/generated/resources/org/apache/camel/catalog/docs/ognl-language.adoc @@ -54,6 +54,7 @@ include::partial$language-options.adoc[] |exchangeId |String |the exchange id |exception |Throwable |the Exchange exception (if any) |request |Message |the message +|message |Message |the message |headers |Map |the message headers |getHeader(name) |Object |the message header by the given name |getHeader(name, type) |Type |the message header by the given name as the given type diff --git a/catalog/camel-catalog/src/generated/resources/org/apache/camel/catalog/docs/openfeature-language.adoc b/catalog/camel-catalog/src/generated/resources/org/apache/camel/catalog/docs/openfeature-language.adoc index d5ca6e7610e3..b5f06413528e 100644 --- a/catalog/camel-catalog/src/generated/resources/org/apache/camel/catalog/docs/openfeature-language.adoc +++ b/catalog/camel-catalog/src/generated/resources/org/apache/camel/catalog/docs/openfeature-language.adoc @@ -30,8 +30,9 @@ This language is provided by the `camel-openfeature` component: == Usage As a predicate, the language evaluates the given flag key as a boolean. -As an expression, it gives the string value of the flag, or the `defaultValue` of the endpoint (`false` unless set) -when the flag is missing or is not a string flag, see below. It uses the `openfeature:flags` endpoint by default, inheriting +As an expression, it gives the value of the flag: the text of a string flag or the `Boolean` of a boolean flag, or +the `defaultValue` of the endpoint (`false` unless set) when the flag is missing or is another type, see below. +It uses the `openfeature:flags` endpoint by default, inheriting component-level configuration such as `flagsResource` or `host`. The language does not modify the exchange — it reads targeting context @@ -157,7 +158,10 @@ YAML:: ==== The `language()` expression defaults to variant (string) evaluation, so -the property receives the string variant value directly. +the property receives the string variant value directly. A boolean flag is +evaluated as a boolean instead, and the property receives a `Boolean`. Number +and object flags are not supported: they give the `defaultValue` and log a +`TYPE_MISMATCH` warning. == Evaluation context @@ -253,9 +257,9 @@ from("direct:start") ---- `createExpression` defaults to variant (string) evaluation and returns an -`Expression`. When called with `evaluate(exchange, Boolean.class)`, it -automatically switches to boolean evaluation for that call without -affecting other callers: +`Expression`, which gives a `Boolean` for a boolean flag. When called with +`evaluate(exchange, Boolean.class)`, it automatically switches to boolean +evaluation for that call without affecting other callers: [source,java] ---- @@ -263,7 +267,7 @@ Expression variant = context.resolveLanguage("openfeature").createExpression( "hazmat-compliance-v2", new Object[] { null, "order-123", Map.of("customer_tier", "ENTERPRISE") }); -Object result = variant.evaluate(exchange, Object.class); // returns String +Object result = variant.evaluate(exchange, Object.class); // returns String, or Boolean for a boolean flag ---- [source,java] diff --git a/catalog/camel-catalog/src/generated/resources/org/apache/camel/catalog/docs/python-language.adoc b/catalog/camel-catalog/src/generated/resources/org/apache/camel/catalog/docs/python-language.adoc index a0e494416a4b..bd4e6a9fc015 100644 --- a/catalog/camel-catalog/src/generated/resources/org/apache/camel/catalog/docs/python-language.adoc +++ b/catalog/camel-catalog/src/generated/resources/org/apache/camel/catalog/docs/python-language.adoc @@ -54,8 +54,9 @@ The `variable` and `variables` bindings are the exchange-scoped variables (for e xref:eips:setVariable-eip.adoc[Set Variable] EIP). Global and route variables, such as `global:foo`, are not in this map; use `exchange.getVariable('global:foo')` for those. -The result of a script is its Python text, as `str()` gives it: a dict gives its Python text, and `None` gives the -text `None`. Set `resultType` to convert the text to another type. +The result of a script is its Java value: `None` is `null`, a string, a number or a boolean is the Java `String`, +number or `Boolean`, a dict, list, tuple or set is a `Map`, `List` or `Set` of Java values, and a Java object, such as +the body, is returned as it is. Set `resultType` to convert the result to another type. == Examples diff --git a/catalog/camel-catalog/src/generated/resources/org/apache/camel/catalog/docs/quickjs-language.adoc b/catalog/camel-catalog/src/generated/resources/org/apache/camel/catalog/docs/quickjs-language.adoc index ff56cae58b61..d6b600a9e229 100644 --- a/catalog/camel-catalog/src/generated/resources/org/apache/camel/catalog/docs/quickjs-language.adoc +++ b/catalog/camel-catalog/src/generated/resources/org/apache/camel/catalog/docs/quickjs-language.adoc @@ -163,7 +163,8 @@ the unsupported type. As an expression, the JSON value of the script becomes the Camel result (then converted with Camel type converters when a result type is requested): an object becomes a `Map` and an array a `List`, a `Date` becomes its ISO text, and `undefined`, `NaN` or a function becomes `null`. -A `Set` or a typed array does not keep its values. In `setBody` and `transform` this result +A `Set` becomes a `List`, a JavaScript `Map` a `Map` (its keys as text), and a typed array or an +`ArrayBuffer` a `List` of numbers, also when nested. In `setBody` and `transform` this result replaces a body the script set with `camel.setBody(...)`. As a predicate (`.when().quickjs(...)` or `.filter().quickjs(...)`), the result is converted to diff --git a/catalog/camel-catalog/src/generated/resources/org/apache/camel/catalog/languages/datasonnet.json b/catalog/camel-catalog/src/generated/resources/org/apache/camel/catalog/languages/datasonnet.json index 94621b1abcc6..949646321217 100644 --- a/catalog/camel-catalog/src/generated/resources/org/apache/camel/catalog/languages/datasonnet.json +++ b/catalog/camel-catalog/src/generated/resources/org/apache/camel/catalog/languages/datasonnet.json @@ -16,8 +16,8 @@ "modelJavaType": "org.apache.camel.model.language.DatasonnetExpression" }, "bodyTypes": { - "expressionIn": { "carriers": [ "any" ], "dependsOn": [ "bodyMediaType", "source" ], "description": "The body as it is, in the variable body. JSON, XML or CSV text is read in that format only with bodyMediaType or a Content-Type header, and a stream needs one of them" }, - "expressionOut": { "carriers": [ "pojo" ], "javaType": "com.datasonnet.document.Document", "dependsOn": [ "resultType", "outputMediaType" ], "description": "A Document with the result: a Java object, or text in outputMediaType when it is set. With resultType, the result converted to that type" } + "expressionIn": { "carriers": [ "any" ], "dependsOn": [ "bodyMediaType", "source" ], "description": "The body as it is, in the variable body. JSON, XML or CSV text is read in that format only with bodyMediaType or a Content-Type header, and without them a stream is read as text" }, + "expressionOut": { "carriers": [ "pojo", "null" ], "javaType": "com.datasonnet.document.Document", "dependsOn": [ "resultType", "outputMediaType" ], "description": "A Document with the result: a Java object, or text in outputMediaType when it is set, and null when the result is null. With resultType, the result converted to that type" } }, "properties": { "id": { "index": 0, "kind": "attribute", "displayName": "Id", "group": "common", "required": false, "type": "string", "javaType": "java.lang.String", "deprecated": false, "autowired": false, "secret": false, "description": "The id of this node." }, diff --git a/catalog/camel-catalog/src/generated/resources/org/apache/camel/catalog/languages/hl7terser.json b/catalog/camel-catalog/src/generated/resources/org/apache/camel/catalog/languages/hl7terser.json index bd1b1add84ee..4cecff9759b4 100644 --- a/catalog/camel-catalog/src/generated/resources/org/apache/camel/catalog/languages/hl7terser.json +++ b/catalog/camel-catalog/src/generated/resources/org/apache/camel/catalog/languages/hl7terser.json @@ -16,7 +16,7 @@ "modelJavaType": "org.apache.camel.model.language.Hl7TerserExpression" }, "bodyTypes": { - "expressionIn": { "carriers": [ "text", "bytes", "pojo" ], "format": "hl7", "dependsOn": [ "source" ], "description": "An HL7 message as text or bytes, in ER7 or XML, a HAPI Message, or a DOM of HL7 XML. A stream or a file fails: convert it to text first" }, + "expressionIn": { "carriers": [ "text", "bytes", "stream", "file", "pojo" ], "format": "hl7", "dependsOn": [ "source" ], "description": "An HL7 message as text or bytes, in ER7 or XML, a stream or a file of it, read in the charset of MSH-18, a HAPI Message, or a DOM of HL7 XML. Another body fails" }, "expressionOut": { "carriers": [ "text", "null" ], "javaType": "java.lang.String", "dependsOn": [ "resultType" ], "description": "The value of the field as text, null when the field is empty" } }, "properties": { diff --git a/catalog/camel-catalog/src/generated/resources/org/apache/camel/catalog/languages/js.json b/catalog/camel-catalog/src/generated/resources/org/apache/camel/catalog/languages/js.json index 697d1e15777d..17c410e60c2b 100644 --- a/catalog/camel-catalog/src/generated/resources/org/apache/camel/catalog/languages/js.json +++ b/catalog/camel-catalog/src/generated/resources/org/apache/camel/catalog/languages/js.json @@ -17,7 +17,7 @@ }, "bodyTypes": { "expressionIn": { "carriers": [ "any" ], "description": "The body as it is, in the variable body: a String is a JavaScript string, and other Java objects keep their methods" }, - "expressionOut": { "carriers": [ "unknown" ], "description": "What the script returns: an object as a Map, an array as a List, a Set as a Set and a Date as an Instant" } + "expressionOut": { "carriers": [ "unknown" ], "description": "What the script returns: an object as a Map, an array or a typed array as a List, a Set as a Set, an ArrayBuffer as bytes and a Date as an Instant. A function fails" } }, "properties": { "id": { "index": 0, "kind": "attribute", "displayName": "Id", "group": "common", "required": false, "type": "string", "javaType": "java.lang.String", "deprecated": false, "autowired": false, "secret": false, "description": "The id of this node." }, diff --git a/catalog/camel-catalog/src/generated/resources/org/apache/camel/catalog/languages/ognl.json b/catalog/camel-catalog/src/generated/resources/org/apache/camel/catalog/languages/ognl.json index 4be079c57039..33c223028370 100644 --- a/catalog/camel-catalog/src/generated/resources/org/apache/camel/catalog/languages/ognl.json +++ b/catalog/camel-catalog/src/generated/resources/org/apache/camel/catalog/languages/ognl.json @@ -16,7 +16,7 @@ "modelJavaType": "org.apache.camel.model.language.OgnlExpression" }, "bodyTypes": { - "expressionIn": { "carriers": [ "any" ], "description": "The body as it is, in request.body: there is no variable body" }, + "expressionIn": { "carriers": [ "any" ], "description": "The body as it is, in message.body or request.body: there is no variable body" }, "expressionOut": { "carriers": [ "unknown" ], "description": "What the expression returns" } }, "properties": { diff --git a/catalog/camel-catalog/src/generated/resources/org/apache/camel/catalog/languages/openfeature.json b/catalog/camel-catalog/src/generated/resources/org/apache/camel/catalog/languages/openfeature.json index 06e473388476..c71d1ebf94bd 100644 --- a/catalog/camel-catalog/src/generated/resources/org/apache/camel/catalog/languages/openfeature.json +++ b/catalog/camel-catalog/src/generated/resources/org/apache/camel/catalog/languages/openfeature.json @@ -17,7 +17,7 @@ }, "bodyTypes": { "expressionIn": { "carriers": [ "any" ], "description": "Not read: the expression is the flag key, and the targeting key and the context come from headers" }, - "expressionOut": { "carriers": [ "text" ], "javaType": "java.lang.String", "description": "The string value of the flag, or the defaultValue of the endpoint when the flag is not a string flag or is missing" } + "expressionOut": { "carriers": [ "text", "value" ], "description": "The value of a string flag as text, or of a boolean flag as a Boolean, and the defaultValue of the endpoint as text when the flag is missing or is another type" } }, "properties": { "id": { "index": 0, "kind": "attribute", "displayName": "Id", "group": "common", "required": false, "type": "string", "javaType": "java.lang.String", "deprecated": false, "autowired": false, "secret": false, "description": "The id of this node." }, diff --git a/catalog/camel-catalog/src/generated/resources/org/apache/camel/catalog/languages/python.json b/catalog/camel-catalog/src/generated/resources/org/apache/camel/catalog/languages/python.json index bfa3cd98fa59..9f2e6f91931a 100644 --- a/catalog/camel-catalog/src/generated/resources/org/apache/camel/catalog/languages/python.json +++ b/catalog/camel-catalog/src/generated/resources/org/apache/camel/catalog/languages/python.json @@ -17,7 +17,7 @@ }, "bodyTypes": { "expressionIn": { "carriers": [ "any" ], "description": "The body as it is, in the variable body: a String is a Python unicode string, and other Java objects keep their methods" }, - "expressionOut": { "carriers": [ "text" ], "javaType": "java.lang.String", "dependsOn": [ "resultType" ], "description": "The Python text, str(), of what the script returns: a dict as its Python text, and None as the text None" } + "expressionOut": { "carriers": [ "unknown" ], "description": "The Java value of what the script returns: a dict as a Map, a list or a tuple as a List, a set as a Set, and None as null" } }, "properties": { "id": { "index": 0, "kind": "attribute", "displayName": "Id", "group": "common", "required": false, "type": "string", "javaType": "java.lang.String", "deprecated": false, "autowired": false, "secret": false, "description": "The id of this node." }, diff --git a/catalog/camel-catalog/src/generated/resources/org/apache/camel/catalog/languages/python3.json b/catalog/camel-catalog/src/generated/resources/org/apache/camel/catalog/languages/python3.json index f3f647ace109..7fd5d897b4ae 100644 --- a/catalog/camel-catalog/src/generated/resources/org/apache/camel/catalog/languages/python3.json +++ b/catalog/camel-catalog/src/generated/resources/org/apache/camel/catalog/languages/python3.json @@ -17,7 +17,7 @@ }, "bodyTypes": { "expressionIn": { "carriers": [ "any" ], "description": "The body as it is, in the variable body: a Map, a List or an array can be indexed, and other Java objects have no methods unless the language is created with createWithHostAccess()" }, - "expressionOut": { "carriers": [ "unknown" ], "description": "What the script returns: a dict as a Map, a list or a tuple as a List, a set as a Set, bytes as a List of numbers, and None as null" } + "expressionOut": { "carriers": [ "unknown" ], "description": "What the script returns: a dict as a Map, a list or a tuple as a List, a set as a Set, bytes as bytes, and None as null. A function fails" } }, "properties": { "id": { "index": 0, "kind": "attribute", "displayName": "Id", "group": "common", "required": false, "type": "string", "javaType": "java.lang.String", "deprecated": false, "autowired": false, "secret": false, "description": "The id of this node." }, diff --git a/catalog/camel-catalog/src/generated/resources/org/apache/camel/catalog/languages/quickjs.json b/catalog/camel-catalog/src/generated/resources/org/apache/camel/catalog/languages/quickjs.json index f4e206c0b39f..9ad3742d017c 100644 --- a/catalog/camel-catalog/src/generated/resources/org/apache/camel/catalog/languages/quickjs.json +++ b/catalog/camel-catalog/src/generated/resources/org/apache/camel/catalog/languages/quickjs.json @@ -17,7 +17,7 @@ }, "bodyTypes": { "expressionIn": { "carriers": [ "text", "bytes", "value", "map", "list", "pojo" ], "description": "A JSON copy of the body, in the variable body: bytes as Base64 text, and a POJO through Jackson. A stream fails: convert it to text first" }, - "expressionOut": { "carriers": [ "text", "value", "map", "list", "null" ], "dependsOn": [ "resultType" ], "description": "The JSON value the script returns: an object as a Map, an array as a List, and undefined, NaN or a function as null" } + "expressionOut": { "carriers": [ "text", "value", "map", "list", "null" ], "dependsOn": [ "resultType" ], "description": "The JSON value the script returns: an object or a Map as a Map, an array, a Set or a typed array as a List, a Date as its ISO text, and undefined, NaN or a function as null" } }, "properties": { "id": { "index": 0, "kind": "attribute", "displayName": "Id", "group": "common", "required": false, "type": "string", "javaType": "java.lang.String", "deprecated": false, "autowired": false, "secret": false, "description": "The id of this node." }, diff --git a/components/camel-datasonnet/src/generated/resources/META-INF/org/apache/camel/language/datasonnet/datasonnet.json b/components/camel-datasonnet/src/generated/resources/META-INF/org/apache/camel/language/datasonnet/datasonnet.json index 94621b1abcc6..949646321217 100644 --- a/components/camel-datasonnet/src/generated/resources/META-INF/org/apache/camel/language/datasonnet/datasonnet.json +++ b/components/camel-datasonnet/src/generated/resources/META-INF/org/apache/camel/language/datasonnet/datasonnet.json @@ -16,8 +16,8 @@ "modelJavaType": "org.apache.camel.model.language.DatasonnetExpression" }, "bodyTypes": { - "expressionIn": { "carriers": [ "any" ], "dependsOn": [ "bodyMediaType", "source" ], "description": "The body as it is, in the variable body. JSON, XML or CSV text is read in that format only with bodyMediaType or a Content-Type header, and a stream needs one of them" }, - "expressionOut": { "carriers": [ "pojo" ], "javaType": "com.datasonnet.document.Document", "dependsOn": [ "resultType", "outputMediaType" ], "description": "A Document with the result: a Java object, or text in outputMediaType when it is set. With resultType, the result converted to that type" } + "expressionIn": { "carriers": [ "any" ], "dependsOn": [ "bodyMediaType", "source" ], "description": "The body as it is, in the variable body. JSON, XML or CSV text is read in that format only with bodyMediaType or a Content-Type header, and without them a stream is read as text" }, + "expressionOut": { "carriers": [ "pojo", "null" ], "javaType": "com.datasonnet.document.Document", "dependsOn": [ "resultType", "outputMediaType" ], "description": "A Document with the result: a Java object, or text in outputMediaType when it is set, and null when the result is null. With resultType, the result converted to that type" } }, "properties": { "id": { "index": 0, "kind": "attribute", "displayName": "Id", "group": "common", "required": false, "type": "string", "javaType": "java.lang.String", "deprecated": false, "autowired": false, "secret": false, "description": "The id of this node." }, diff --git a/components/camel-datasonnet/src/main/docs/datasonnet-language.adoc b/components/camel-datasonnet/src/main/docs/datasonnet-language.adoc index 827c8dff23fe..584d60eeac13 100644 --- a/components/camel-datasonnet/src/main/docs/datasonnet-language.adoc +++ b/components/camel-datasonnet/src/main/docs/datasonnet-language.adoc @@ -36,7 +36,8 @@ include::partial$language-options.adoc[] === Setting a result type -The xref:datasonnet-language.adoc[DataSonnet] expression will return a `com.datasonnet.document.Document` by default. The +The xref:datasonnet-language.adoc[DataSonnet] expression will return a `com.datasonnet.document.Document` by default, or +`null` when the result is `null`. The document preserves the content type metadata along with the contents of the transformation result. In predicates, however, the Document will be automatically unwrapped and the boolean content will be returned. Similarly, any time you want the content in a specific result type like a String. To do this, you have to instruct the @@ -118,7 +119,8 @@ And for output media type: NOTE: With the body media type `application/x-java-object`, the body is the Java object it is: JSON text is a string, not an object, so `body.foo` fails on it. Set `bodyMediaType`, or a `Content-Type` header, to read text, bytes, a -stream or a file in that format, such as `application/json`, `application/xml` or `application/csv`. +stream or a file in that format, such as `application/json`, `application/xml` or `application/csv`. Without a +media type, a stream is read as text. === Functions diff --git a/components/camel-datasonnet/src/main/java/org/apache/camel/language/datasonnet/DatasonnetExpression.java b/components/camel-datasonnet/src/main/java/org/apache/camel/language/datasonnet/DatasonnetExpression.java index c98c28e55892..84ad40798004 100644 --- a/components/camel-datasonnet/src/main/java/org/apache/camel/language/datasonnet/DatasonnetExpression.java +++ b/components/camel-datasonnet/src/main/java/org/apache/camel/language/datasonnet/DatasonnetExpression.java @@ -18,6 +18,8 @@ package org.apache.camel.language.datasonnet; import java.io.File; import java.io.IOException; +import java.io.InputStream; +import java.io.Reader; import java.nio.charset.Charset; import java.nio.file.FileVisitResult; import java.nio.file.Files; @@ -42,6 +44,7 @@ import org.apache.camel.CamelContext; import org.apache.camel.Exchange; import org.apache.camel.Expression; import org.apache.camel.RuntimeExpressionException; +import org.apache.camel.StreamCache; import org.apache.camel.spi.ExpressionResultTypeAware; import org.apache.camel.support.ExchangeHelper; import org.apache.camel.support.ExpressionAdapter; @@ -78,20 +81,28 @@ public class DatasonnetExpression extends ExpressionAdapter implements Expressio @Override public boolean matches(Exchange exchange) { - this.outputMediaType = MediaTypes.APPLICATION_JAVA; - return evaluate(exchange, Boolean.class); + // a predicate is always evaluated as a java object (without changing the output media type of this expression) + Boolean answer = doEvaluate(exchange, Boolean.class, MediaTypes.APPLICATION_JAVA); + return answer != null && answer; } - @SuppressWarnings("unchecked") @Override public <T> T evaluate(Exchange exchange, Class<T> type) { + return doEvaluate(exchange, type, outputMediaType); + } + + @SuppressWarnings("unchecked") + private <T> T doEvaluate(Exchange exchange, Class<T> type, MediaType outMediaType) { try { // pass exchange to CML lib using thread as context CML.getInstance().getExchange().set(exchange); - Document<?> result = doEvaluate(exchange); + Document<?> result = doEvaluate(exchange, outMediaType); if (!type.equals(Object.class)) { return ExchangeHelper.convertToType(exchange, type, result.getContent()); + } else if (resultType == null && result.getContent() == null) { + // a null result is a null value (not a document without content) + return null; } else if (resultType == null || resultType.equals(Document.class)) { return (T) result; } else { @@ -104,7 +115,7 @@ public class DatasonnetExpression extends ExpressionAdapter implements Expressio } } - private Document<?> doEvaluate(Exchange exchange) { + private Document<?> doEvaluate(Exchange exchange, MediaType outMediaType) { MediaType bodyMT = bodyMediaType; if (bodyMT == null && !expression.startsWith(Header.DATASONNET_HEADER)) { //Try to auto-detect input mime type if it was not explicitly set @@ -122,11 +133,13 @@ public class DatasonnetExpression extends ExpressionAdapter implements Expressio } if (doc == null) { String text = exchange.getContext().getTypeConverter().tryConvertTo(String.class, exchange, payload); - if (exchange.getMessage().getBody() == null || "".equals(text)) { + if (payload == null || "".equals(text)) { //Empty body, force type to be application/java doc = new DefaultDocument<>("", MediaTypes.APPLICATION_JAVA); } else if (MediaTypes.APPLICATION_JAVA.equalsTypeAndSubtype(bodyMT) || bodyMT == null) { - doc = new DefaultDocument<>(payload); + // a stream is read as text, as a stream is not a java object to map + boolean stream = payload instanceof InputStream || payload instanceof Reader || payload instanceof StreamCache; + doc = new DefaultDocument<>(stream ? text : payload); } else { // force using string value doc = new DefaultDocument<>(text, bodyMT); @@ -148,7 +161,7 @@ public class DatasonnetExpression extends ExpressionAdapter implements Expressio return builder.build(); }); - MediaType outMT = outputMediaType; + MediaType outMT = outMediaType; if (outMT == null) { //Try to auto-detect output mime type if it was not explicitly set String typeHeader = exchange.getProperty(DatasonnetConstants.OUTPUT_MEDIATYPE, diff --git a/components/camel-datasonnet/src/main/java/org/apache/camel/language/datasonnet/DatasonnetLanguage.java b/components/camel-datasonnet/src/main/java/org/apache/camel/language/datasonnet/DatasonnetLanguage.java index 64de43a4b8a7..4039324db5f4 100644 --- a/components/camel-datasonnet/src/main/java/org/apache/camel/language/datasonnet/DatasonnetLanguage.java +++ b/components/camel-datasonnet/src/main/java/org/apache/camel/language/datasonnet/DatasonnetLanguage.java @@ -43,12 +43,13 @@ import org.slf4j.LoggerFactory; @Language("datasonnet") @BodyTypes(expressionIn = @BodyType(value = Carrier.ANY, dependsOn = { "bodyMediaType", "source" }, description = "The body as it is, in the variable body. JSON, XML or CSV text is read in" - + " that format only with bodyMediaType or a Content-Type header, and a" - + " stream needs one of them"), - expressionOut = @BodyType(value = Carrier.POJO, javaType = "com.datasonnet.document.Document", + + " that format only with bodyMediaType or a Content-Type header, and" + + " without them a stream is read as text"), + expressionOut = @BodyType(value = { Carrier.POJO, Carrier.NULL }, javaType = "com.datasonnet.document.Document", dependsOn = { "resultType", "outputMediaType" }, description = "A Document with the result: a Java object, or text in outputMediaType" - + " when it is set. With resultType, the result converted to that type")) + + " when it is set, and null when the result is null. With resultType, the" + + " result converted to that type")) public class DatasonnetLanguage extends SingleInputTypedLanguageSupport { private static final Logger LOG = LoggerFactory.getLogger(DatasonnetLanguage.class); private final Map<String, String> classpathImports; diff --git a/components/camel-datasonnet/src/test/java/org/apache/camel/language/datasonnet/DatasonnetBodyTest.java b/components/camel-datasonnet/src/test/java/org/apache/camel/language/datasonnet/DatasonnetBodyTest.java new file mode 100644 index 000000000000..b0a6910c21ee --- /dev/null +++ b/components/camel-datasonnet/src/test/java/org/apache/camel/language/datasonnet/DatasonnetBodyTest.java @@ -0,0 +1,135 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.camel.language.datasonnet; + +import java.io.ByteArrayInputStream; +import java.nio.charset.StandardCharsets; + +import com.datasonnet.document.MediaTypes; +import org.apache.camel.Exchange; +import org.apache.camel.Expression; +import org.apache.camel.builder.RouteBuilder; +import org.apache.camel.component.mock.MockEndpoint; +import org.apache.camel.converter.stream.InputStreamCache; +import org.apache.camel.support.DefaultExchange; +import org.apache.camel.test.junit6.CamelTestSupport; +import org.junit.jupiter.api.Test; + +import static org.junit.jupiter.api.Assertions.assertEquals; +import static org.junit.jupiter.api.Assertions.assertFalse; +import static org.junit.jupiter.api.Assertions.assertNull; +import static org.junit.jupiter.api.Assertions.assertTrue; + +public class DatasonnetBodyTest extends CamelTestSupport { + + @Override + protected RouteBuilder createRouteBuilder() { + return new RouteBuilder() { + @Override + public void configure() { + from("direct:null") + .setBody(datasonnet("null")) + .setHeader("text", simple("${body}")) + .to("mock:null"); + } + }; + } + + @Test + public void testSourceWithNullBody() { + Expression exp = context.resolveLanguage("datasonnet") + .createExpression("body", new Object[] { String.class, "header:foo", null, null }); + + Exchange exchange = new DefaultExchange(context); + exchange.getMessage().setHeader("foo", "Hello"); + + assertEquals("Hello", exp.evaluate(exchange, String.class)); + } + + @Test + public void testMatchesKeepsOutputMediaType() { + DatasonnetExpression exp = new DatasonnetExpression("body"); + exp.init(context); + + assertEquals("\"Hello\"", exp.evaluate(createJsonOutputExchange(), String.class)); + + Exchange predicate = new DefaultExchange(context); + predicate.getMessage().setBody(true); + assertTrue(exp.matches(predicate)); + + assertNull(exp.getOutputMediaType()); + assertEquals("\"Hello\"", exp.evaluate(createJsonOutputExchange(), String.class)); + } + + @Test + public void testStreamBody() { + Expression exp = context.resolveLanguage("datasonnet") + .createExpression("'Hello ' + body", new Object[] { String.class, null, null, null }); + + Exchange exchange = new DefaultExchange(context); + exchange.getMessage().setBody(new ByteArrayInputStream("World".getBytes(StandardCharsets.UTF_8))); + assertEquals("Hello World", exp.evaluate(exchange, String.class)); + } + + @Test + public void testStreamCacheBody() { + Expression exp = context.resolveLanguage("datasonnet") + .createExpression("'Hello ' + body", new Object[] { String.class, null, null, null }); + + Exchange exchange = new DefaultExchange(context); + exchange.getMessage().setBody(new InputStreamCache("World".getBytes(StandardCharsets.UTF_8))); + assertEquals("Hello World", exp.evaluate(exchange, String.class)); + } + + @Test + public void testNullResult() { + Expression exp = context.resolveLanguage("datasonnet").createExpression("null"); + + Exchange exchange = new DefaultExchange(context); + exchange.getMessage().setBody("Hello"); + assertNull(exp.evaluate(exchange, Object.class)); + } + + @Test + public void testNullResultInRoute() throws Exception { + MockEndpoint mock = getMockEndpoint("mock:null"); + mock.expectedMessageCount(1); + mock.message(0).body().isNull(); + + template.sendBody("direct:null", "Hello"); + + MockEndpoint.assertIsSatisfied(context); + assertNull(mock.getReceivedExchanges().get(0).getMessage().getHeader("text")); + } + + @Test + public void testNullResultPredicate() { + DatasonnetExpression exp = new DatasonnetExpression("null"); + exp.init(context); + + Exchange exchange = new DefaultExchange(context); + exchange.getMessage().setBody("Hello"); + assertFalse(exp.matches(exchange)); + } + + private Exchange createJsonOutputExchange() { + Exchange exchange = new DefaultExchange(context); + exchange.getMessage().setBody("Hello"); + exchange.getMessage().setHeader(DatasonnetConstants.OUTPUT_MEDIATYPE, MediaTypes.APPLICATION_JSON_VALUE); + return exchange; + } +} diff --git a/components/camel-hl7/src/generated/resources/META-INF/org/apache/camel/component/hl7/hl7terser.json b/components/camel-hl7/src/generated/resources/META-INF/org/apache/camel/component/hl7/hl7terser.json index bd1b1add84ee..4cecff9759b4 100644 --- a/components/camel-hl7/src/generated/resources/META-INF/org/apache/camel/component/hl7/hl7terser.json +++ b/components/camel-hl7/src/generated/resources/META-INF/org/apache/camel/component/hl7/hl7terser.json @@ -16,7 +16,7 @@ "modelJavaType": "org.apache.camel.model.language.Hl7TerserExpression" }, "bodyTypes": { - "expressionIn": { "carriers": [ "text", "bytes", "pojo" ], "format": "hl7", "dependsOn": [ "source" ], "description": "An HL7 message as text or bytes, in ER7 or XML, a HAPI Message, or a DOM of HL7 XML. A stream or a file fails: convert it to text first" }, + "expressionIn": { "carriers": [ "text", "bytes", "stream", "file", "pojo" ], "format": "hl7", "dependsOn": [ "source" ], "description": "An HL7 message as text or bytes, in ER7 or XML, a stream or a file of it, read in the charset of MSH-18, a HAPI Message, or a DOM of HL7 XML. Another body fails" }, "expressionOut": { "carriers": [ "text", "null" ], "javaType": "java.lang.String", "dependsOn": [ "resultType" ], "description": "The value of the field as text, null when the field is empty" } }, "properties": { diff --git a/components/camel-hl7/src/main/java/org/apache/camel/component/hl7/Hl7TerserLanguage.java b/components/camel-hl7/src/main/java/org/apache/camel/component/hl7/Hl7TerserLanguage.java index d9544cf11657..fe251c9b8671 100644 --- a/components/camel-hl7/src/main/java/org/apache/camel/component/hl7/Hl7TerserLanguage.java +++ b/components/camel-hl7/src/main/java/org/apache/camel/component/hl7/Hl7TerserLanguage.java @@ -16,26 +16,35 @@ */ package org.apache.camel.component.hl7; +import java.io.File; +import java.io.InputStream; + import ca.uhn.hl7v2.HL7Exception; import ca.uhn.hl7v2.model.Message; import ca.uhn.hl7v2.util.Terser; import org.apache.camel.CamelContext; import org.apache.camel.Exchange; import org.apache.camel.Expression; +import org.apache.camel.ExpressionEvaluationException; import org.apache.camel.RuntimeCamelException; +import org.apache.camel.StreamCache; +import org.apache.camel.WrappedFile; import org.apache.camel.spi.BodyType; import org.apache.camel.spi.BodyType.Carrier; import org.apache.camel.spi.BodyTypes; import org.apache.camel.spi.annotations.Language; +import org.apache.camel.support.ExchangeHelper; import org.apache.camel.support.ExpressionAdapter; import org.apache.camel.support.SingleInputTypedLanguageSupport; import org.apache.camel.support.builder.ExpressionBuilder; import org.apache.camel.util.ObjectHelper; @Language("hl7terser") -@BodyTypes(expressionIn = @BodyType(value = { Carrier.TEXT, Carrier.BYTES, Carrier.POJO }, format = "hl7", dependsOn = "source", - description = "An HL7 message as text or bytes, in ER7 or XML, a HAPI Message, or a DOM of" - + " HL7 XML. A stream or a file fails: convert it to text first"), +@BodyTypes(expressionIn = @BodyType(value = { Carrier.TEXT, Carrier.BYTES, Carrier.STREAM, Carrier.FILE, Carrier.POJO }, + format = "hl7", dependsOn = "source", + description = "An HL7 message as text or bytes, in ER7 or XML, a stream or a file of it, read" + + " in the charset of MSH-18, a HAPI Message, or a DOM of HL7 XML. Another" + + " body fails"), expressionOut = @BodyType(value = { Carrier.TEXT, Carrier.NULL }, javaType = "java.lang.String", dependsOn = "resultType", description = "The value of the field as text, null when the field is empty")) @@ -51,7 +60,7 @@ public class Hl7TerserLanguage extends SingleInputTypedLanguageSupport { @Override public Object evaluate(Exchange exchange) { - Message message = source.evaluate(exchange, Message.class); + Message message = toMessage(exchange, source.evaluate(exchange, Object.class)); try { return new Terser(message).get(expression.trim()); } catch (HL7Exception e) { @@ -59,6 +68,38 @@ public class Hl7TerserLanguage extends SingleInputTypedLanguageSupport { } } + private Message toMessage(Exchange exchange, Object body) { + if (body instanceof Message message) { + return message; + } + Message answer = null; + if (body != null) { + try { + if (body instanceof InputStream || body instanceof StreamCache || body instanceof WrappedFile + || body instanceof File) { + // read a stream or a file as text, in the charset of MSH-18 (like the HL7 data format) + byte[] bytes = exchange.getContext().getTypeConverter().mandatoryConvertTo(byte[].class, exchange, + body); + String charsetName = HL7Charset.getCharsetName(bytes, ExchangeHelper.getCharsetName(exchange)); + body = new String(bytes, charsetName); + } + answer = exchange.getContext().getTypeConverter().convertTo(Message.class, exchange, body); + } catch (Exception e) { + throw new ExpressionEvaluationException( + this, "Cannot evaluate " + this + " as " + source + " is not a valid HL7 message", exchange, e); + } + } + if (answer == null) { + String type = body != null ? "of type " + body.getClass().getName() : "null"; + throw new ExpressionEvaluationException( + this, "Cannot evaluate " + this + " as " + source + " is " + type + + ". Accepted are text or bytes of an HL7 message in ER7 or XML (also from a stream or" + + " a file), a HAPI Message, or a DOM of HL7 XML", + exchange, null); + } + return answer; + } + @Override public void init(CamelContext context) { source.init(context); diff --git a/components/camel-hl7/src/test/java/org/apache/camel/component/hl7/TerserExpressionBodyTest.java b/components/camel-hl7/src/test/java/org/apache/camel/component/hl7/TerserExpressionBodyTest.java new file mode 100644 index 000000000000..a8bc299358af --- /dev/null +++ b/components/camel-hl7/src/test/java/org/apache/camel/component/hl7/TerserExpressionBodyTest.java @@ -0,0 +1,110 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.camel.component.hl7; + +import java.io.ByteArrayInputStream; +import java.nio.charset.StandardCharsets; +import java.nio.file.Files; +import java.nio.file.Path; +import java.util.Map; + +import ca.uhn.hl7v2.model.v24.message.ADT_A01; +import org.apache.camel.CamelExecutionException; +import org.apache.camel.ExpressionEvaluationException; +import org.apache.camel.builder.RouteBuilder; +import org.apache.camel.component.mock.MockEndpoint; +import org.apache.camel.converter.stream.InputStreamCache; +import org.apache.camel.test.junit6.CamelTestSupport; +import org.junit.jupiter.api.Test; +import org.junit.jupiter.api.io.TempDir; + +import static org.apache.camel.component.hl7.HL7.hl7terser; +import static org.junit.jupiter.api.Assertions.assertInstanceOf; +import static org.junit.jupiter.api.Assertions.assertThrows; +import static org.junit.jupiter.api.Assertions.assertTrue; + +public class TerserExpressionBodyTest extends CamelTestSupport { + + private static final String PATIENT_ID = "123456"; + + @TempDir + static Path inbox; + + @Test + public void testStreamBody() throws Exception { + MockEndpoint mock = getMockEndpoint("mock:result"); + mock.expectedBodiesReceived(PATIENT_ID); + + template.sendBody("direct:start", new ByteArrayInputStream(createADT01Message().getBytes(StandardCharsets.UTF_8))); + + MockEndpoint.assertIsSatisfied(context); + } + + @Test + public void testStreamCacheBody() throws Exception { + MockEndpoint mock = getMockEndpoint("mock:result"); + mock.expectedBodiesReceived(PATIENT_ID); + + template.sendBody("direct:start", new InputStreamCache(createADT01Message().getBytes(StandardCharsets.UTF_8))); + + MockEndpoint.assertIsSatisfied(context); + } + + @Test + public void testFileBody() throws Exception { + MockEndpoint mock = getMockEndpoint("mock:file"); + mock.expectedBodiesReceived(PATIENT_ID); + + Files.writeString(inbox.resolve("adt.hl7"), createADT01Message()); + + MockEndpoint.assertIsSatisfied(context); + } + + @Test + public void testNullBody() { + CamelExecutionException e = assertThrows(CamelExecutionException.class, + () -> template.sendBody("direct:start", null)); + ExpressionEvaluationException cause = assertInstanceOf(ExpressionEvaluationException.class, e.getCause()); + assertTrue(cause.getMessage().contains("is null"), cause.getMessage()); + } + + @Test + public void testMapBody() { + CamelExecutionException e = assertThrows(CamelExecutionException.class, + () -> template.sendBody("direct:start", Map.of("PID", PATIENT_ID))); + ExpressionEvaluationException cause = assertInstanceOf(ExpressionEvaluationException.class, e.getCause()); + assertTrue(cause.getMessage().contains("java.util."), cause.getMessage()); + assertTrue(cause.getMessage().contains("HAPI Message"), cause.getMessage()); + } + + @Override + protected RouteBuilder createRouteBuilder() { + return new RouteBuilder() { + public void configure() { + from("direct:start").transform(hl7terser("PID-3-1")).to("mock:result"); + from("file:" + inbox + "?initialDelay=0&delay=10").transform(hl7terser("PID-3-1")).to("mock:file"); + } + }; + } + + private static String createADT01Message() throws Exception { + ADT_A01 adt = new ADT_A01(); + adt.initQuickstart("ADT", "A01", "P"); + adt.getPID().getPatientIdentifierList(0).getID().setValue(PATIENT_ID); + return adt.encode(); + } +} diff --git a/components/camel-javascript/src/generated/resources/META-INF/org/apache/camel/language/js/js.json b/components/camel-javascript/src/generated/resources/META-INF/org/apache/camel/language/js/js.json index 697d1e15777d..17c410e60c2b 100644 --- a/components/camel-javascript/src/generated/resources/META-INF/org/apache/camel/language/js/js.json +++ b/components/camel-javascript/src/generated/resources/META-INF/org/apache/camel/language/js/js.json @@ -17,7 +17,7 @@ }, "bodyTypes": { "expressionIn": { "carriers": [ "any" ], "description": "The body as it is, in the variable body: a String is a JavaScript string, and other Java objects keep their methods" }, - "expressionOut": { "carriers": [ "unknown" ], "description": "What the script returns: an object as a Map, an array as a List, a Set as a Set and a Date as an Instant" } + "expressionOut": { "carriers": [ "unknown" ], "description": "What the script returns: an object as a Map, an array or a typed array as a List, a Set as a Set, an ArrayBuffer as bytes and a Date as an Instant. A function fails" } }, "properties": { "id": { "index": 0, "kind": "attribute", "displayName": "Id", "group": "common", "required": false, "type": "string", "javaType": "java.lang.String", "deprecated": false, "autowired": false, "secret": false, "description": "The id of this node." }, diff --git a/components/camel-javascript/src/main/java/org/apache/camel/language/js/JavaScriptExpression.java b/components/camel-javascript/src/main/java/org/apache/camel/language/js/JavaScriptExpression.java index 1c02b7689b30..9f4cd65e34eb 100644 --- a/components/camel-javascript/src/main/java/org/apache/camel/language/js/JavaScriptExpression.java +++ b/components/camel-javascript/src/main/java/org/apache/camel/language/js/JavaScriptExpression.java @@ -81,7 +81,7 @@ public class JavaScriptExpression extends ExpressionSupport { b.putMember("properties", properties); Value o = cx.eval(lang.source(expressionString)); - Object answer = JavaScriptLanguage.materialize(o); + Object answer = JavaScriptLanguage.materializeResult(o, this, exchange); if (type == Object.class) { return (T) answer; } diff --git a/components/camel-javascript/src/main/java/org/apache/camel/language/js/JavaScriptLanguage.java b/components/camel-javascript/src/main/java/org/apache/camel/language/js/JavaScriptLanguage.java index af10ff69f518..498e99a9d7b2 100644 --- a/components/camel-javascript/src/main/java/org/apache/camel/language/js/JavaScriptLanguage.java +++ b/components/camel-javascript/src/main/java/org/apache/camel/language/js/JavaScriptLanguage.java @@ -26,7 +26,9 @@ import java.util.Set; import java.util.concurrent.locks.Lock; import java.util.concurrent.locks.ReentrantLock; +import org.apache.camel.Exchange; import org.apache.camel.Expression; +import org.apache.camel.ExpressionEvaluationException; import org.apache.camel.NoTypeConversionAvailableException; import org.apache.camel.Predicate; import org.apache.camel.RuntimeCamelException; @@ -57,10 +59,14 @@ import org.graalvm.polyglot.Value; description = "The body as it is, in the variable body: a String is a JavaScript string," + " and other Java objects keep their methods"), expressionOut = @BodyType(value = Carrier.UNKNOWN, - description = "What the script returns: an object as a Map, an array as a List, a Set as a" - + " Set and a Date as an Instant")) + description = "What the script returns: an object as a Map, an array or a typed array as a" + + " List, a Set as a Set, an ArrayBuffer as bytes and a Date as an Instant." + + " A function fails")) public class JavaScriptLanguage extends TypedLanguageSupport implements ScriptingLanguage, Service { + static final String FUNCTION_RESULT + = "The script returned a function, which cannot be used after the script ends: return a value instead"; + // Source is not a Service, so nothing is stopped on eviction private final Map<String, Source> sourceCache = LRUCacheFactory.newLRUSoftCache(16, 1000, false); private final Lock engineLock = new ReentrantLock(); @@ -106,7 +112,7 @@ public class JavaScriptLanguage extends TypedLanguageSupport implements Scriptin bindings.forEach(b::putMember); } Value o = cx.eval(source(script)); - Object answer = materialize(o); + Object answer = materializeResult(o, null, null); if (answer == null || resultType == Object.class || resultType.isInstance(answer)) { return resultType.cast(answer); } @@ -122,12 +128,25 @@ public class JavaScriptLanguage extends TypedLanguageSupport implements Scriptin } } + /** + * Materializes the result of a script, see {@link #materialize(Value)}. A script that returns a function fails here + * with an {@link ExpressionEvaluationException}: the function is tied to the per-evaluation {@link Context}, which + * is closed as soon as the script ends. + */ + static Object materializeResult(Value value, Expression expression, Exchange exchange) { + if (value != null && value.canExecute() && !value.isHostObject() && !value.isProxyObject()) { + throw new ExpressionEvaluationException(expression, FUNCTION_RESULT, exchange, null); + } + return materialize(value); + } + /** * Copies a guest {@link Value} into ordinary Java types so the result remains usable after the per-evaluation - * {@link Context} is closed: JS arrays become {@link List}, JS objects and {@code Map} become {@link Map}, JS - * {@code Set} becomes {@link Set}, and {@code Date} becomes {@link java.time.Instant}. Nested values are copied - * recursively. Primitives, strings and host objects are returned as before; other guest objects such as functions - * are left to {@link Value#as(Class) value.as(Object.class)}. + * {@link Context} is closed: JS arrays (and typed arrays) become {@link List}, an {@code ArrayBuffer} becomes + * {@code byte[]}, JS objects and {@code Map} become {@link Map}, JS {@code Set} becomes {@link Set}, and + * {@code Date} becomes {@link java.time.Instant}. Nested values are copied recursively. Primitives, strings and + * host objects are returned as before; other guest objects such as functions are left to {@link Value#as(Class) + * value.as(Object.class)}. */ static Object materialize(Value value) { return materialize(value, new HashMap<>()); @@ -149,6 +168,11 @@ public class JavaScriptLanguage extends TypedLanguageSupport implements Scriptin if (value.isInstant()) { return value.asInstant(); } + if (value.hasBufferElements()) { + byte[] bytes = new byte[Math.toIntExact(value.getBufferSize())]; + value.readBuffer(0, bytes, 0, bytes.length); + return bytes; + } Object existing = seen.get(value); if (existing != null) { return existing; diff --git a/components/camel-javascript/src/test/java/org/apache/camel/language/js/JavaScriptResultMaterializationTest.java b/components/camel-javascript/src/test/java/org/apache/camel/language/js/JavaScriptResultMaterializationTest.java index ae57072dc3c0..077be5f91799 100644 --- a/components/camel-javascript/src/test/java/org/apache/camel/language/js/JavaScriptResultMaterializationTest.java +++ b/components/camel-javascript/src/test/java/org/apache/camel/language/js/JavaScriptResultMaterializationTest.java @@ -23,6 +23,7 @@ import java.util.Set; import org.apache.camel.CamelContext; import org.apache.camel.Exchange; +import org.apache.camel.ExpressionEvaluationException; import org.apache.camel.impl.DefaultCamelContext; import org.apache.camel.support.DefaultExchange; import org.junit.jupiter.api.AfterEach; @@ -31,6 +32,7 @@ import org.junit.jupiter.api.Test; import org.junit.jupiter.api.condition.DisabledIfSystemProperty; import static org.assertj.core.api.Assertions.assertThat; +import static org.assertj.core.api.Assertions.assertThatThrownBy; import static org.assertj.core.api.Assertions.entry; /** @@ -113,6 +115,34 @@ class JavaScriptResultMaterializationTest { assertThat(language.evaluate("n + 1", bindings, String.class)).isEqualTo("3"); } + @Test + void arrayBufferBecomesByteArray() { + assertThat(evaluate("new Uint8Array([1, 2, 255]).buffer")).isEqualTo(new byte[] { 1, 2, -1 }); + assertThat(evaluate("new ArrayBuffer(0)")).isEqualTo(new byte[0]); + assertThat(language.evaluate("new Uint8Array([n]).buffer", Map.of("n", 7), Object.class)) + .isEqualTo(new byte[] { 7 }); + } + + @Test + void typedArrayStaysList() { + // GraalJS exposes a typed array as array elements, not as a buffer + assertThat(asList(evaluate("new Uint8Array([1, 2, 255])"))).containsExactly(1, 2, 255); + } + + @Test + void returnedFunctionFailsAtEvaluation() { + String message = "The script returned a function, which cannot be used after the script ends: return a value instead"; + assertThatThrownBy(() -> evaluate("() => 1")) + .isInstanceOf(ExpressionEvaluationException.class) + .hasMessage(message); + assertThatThrownBy(() -> evaluate("(class A {})")) + .isInstanceOf(ExpressionEvaluationException.class) + .hasMessage(message); + assertThatThrownBy(() -> language.evaluate("(function f() { return 1; })", Map.of(), Object.class)) + .isInstanceOf(ExpressionEvaluationException.class) + .hasMessage(message); + } + private Object evaluate(String script) { return language.createExpression(script).evaluate(new DefaultExchange(context), Object.class); } diff --git a/components/camel-ognl/src/generated/resources/META-INF/org/apache/camel/language/ognl/ognl.json b/components/camel-ognl/src/generated/resources/META-INF/org/apache/camel/language/ognl/ognl.json index 4be079c57039..33c223028370 100644 --- a/components/camel-ognl/src/generated/resources/META-INF/org/apache/camel/language/ognl/ognl.json +++ b/components/camel-ognl/src/generated/resources/META-INF/org/apache/camel/language/ognl/ognl.json @@ -16,7 +16,7 @@ "modelJavaType": "org.apache.camel.model.language.OgnlExpression" }, "bodyTypes": { - "expressionIn": { "carriers": [ "any" ], "description": "The body as it is, in request.body: there is no variable body" }, + "expressionIn": { "carriers": [ "any" ], "description": "The body as it is, in message.body or request.body: there is no variable body" }, "expressionOut": { "carriers": [ "unknown" ], "description": "What the expression returns" } }, "properties": { diff --git a/components/camel-ognl/src/main/docs/ognl-language.adoc b/components/camel-ognl/src/main/docs/ognl-language.adoc index 19f734145d2c..94d3d714796e 100644 --- a/components/camel-ognl/src/main/docs/ognl-language.adoc +++ b/components/camel-ognl/src/main/docs/ognl-language.adoc @@ -54,6 +54,7 @@ include::partial$language-options.adoc[] |exchangeId |String |the exchange id |exception |Throwable |the Exchange exception (if any) |request |Message |the message +|message |Message |the message |headers |Map |the message headers |getHeader(name) |Object |the message header by the given name |getHeader(name, type) |Type |the message header by the given name as the given type diff --git a/components/camel-ognl/src/main/java/org/apache/camel/language/ognl/OgnlLanguage.java b/components/camel-ognl/src/main/java/org/apache/camel/language/ognl/OgnlLanguage.java index 580abfea74d0..61051e29fc20 100644 --- a/components/camel-ognl/src/main/java/org/apache/camel/language/ognl/OgnlLanguage.java +++ b/components/camel-ognl/src/main/java/org/apache/camel/language/ognl/OgnlLanguage.java @@ -33,7 +33,7 @@ import org.apache.camel.support.TypedLanguageSupport; @Language("ognl") @BodyTypes(expressionIn = @BodyType(value = Carrier.ANY, - description = "The body as it is, in request.body: there is no variable body"), + description = "The body as it is, in message.body or request.body: there is no variable body"), expressionOut = @BodyType(value = Carrier.UNKNOWN, description = "What the expression returns")) public class OgnlLanguage extends TypedLanguageSupport implements ScriptingLanguage { diff --git a/components/camel-ognl/src/main/java/org/apache/camel/language/ognl/RootObject.java b/components/camel-ognl/src/main/java/org/apache/camel/language/ognl/RootObject.java index d323912cfa0a..ab50618d7ba3 100644 --- a/components/camel-ognl/src/main/java/org/apache/camel/language/ognl/RootObject.java +++ b/components/camel-ognl/src/main/java/org/apache/camel/language/ognl/RootObject.java @@ -49,6 +49,10 @@ public class RootObject { return exchange.getIn(); } + public Message getMessage() { + return exchange.getMessage(); + } + @Deprecated public Message getResponse() { return exchange.getOut(); diff --git a/components/camel-ognl/src/test/java/org/apache/camel/language/ognl/OgnlTest.java b/components/camel-ognl/src/test/java/org/apache/camel/language/ognl/OgnlTest.java index 23fa14f41a1e..c70cdc24ee62 100644 --- a/components/camel-ognl/src/test/java/org/apache/camel/language/ognl/OgnlTest.java +++ b/components/camel-ognl/src/test/java/org/apache/camel/language/ognl/OgnlTest.java @@ -36,6 +36,14 @@ public class OgnlTest extends LanguageTestSupport { assertExpression("request.headers.foo", "abc"); } + @Test + public void testMessageExpressions() { + assertExpression("message.body", "<hello id='m123'>world!</hello>"); + assertExpression("message.headers.foo", "abc"); + assertExpression("getMessage().headers['foo']", "abc"); + assertExpression("message == exchange.message", true); + } + @Test public void testClassMethodExpression() { try { diff --git a/components/camel-openfeature/src/generated/resources/META-INF/org/apache/camel/language/openfeature/openfeature.json b/components/camel-openfeature/src/generated/resources/META-INF/org/apache/camel/language/openfeature/openfeature.json index 06e473388476..c71d1ebf94bd 100644 --- a/components/camel-openfeature/src/generated/resources/META-INF/org/apache/camel/language/openfeature/openfeature.json +++ b/components/camel-openfeature/src/generated/resources/META-INF/org/apache/camel/language/openfeature/openfeature.json @@ -17,7 +17,7 @@ }, "bodyTypes": { "expressionIn": { "carriers": [ "any" ], "description": "Not read: the expression is the flag key, and the targeting key and the context come from headers" }, - "expressionOut": { "carriers": [ "text" ], "javaType": "java.lang.String", "description": "The string value of the flag, or the defaultValue of the endpoint when the flag is not a string flag or is missing" } + "expressionOut": { "carriers": [ "text", "value" ], "description": "The value of a string flag as text, or of a boolean flag as a Boolean, and the defaultValue of the endpoint as text when the flag is missing or is another type" } }, "properties": { "id": { "index": 0, "kind": "attribute", "displayName": "Id", "group": "common", "required": false, "type": "string", "javaType": "java.lang.String", "deprecated": false, "autowired": false, "secret": false, "description": "The id of this node." }, diff --git a/components/camel-openfeature/src/main/docs/openfeature-language.adoc b/components/camel-openfeature/src/main/docs/openfeature-language.adoc index d5ca6e7610e3..b5f06413528e 100644 --- a/components/camel-openfeature/src/main/docs/openfeature-language.adoc +++ b/components/camel-openfeature/src/main/docs/openfeature-language.adoc @@ -30,8 +30,9 @@ This language is provided by the `camel-openfeature` component: == Usage As a predicate, the language evaluates the given flag key as a boolean. -As an expression, it gives the string value of the flag, or the `defaultValue` of the endpoint (`false` unless set) -when the flag is missing or is not a string flag, see below. It uses the `openfeature:flags` endpoint by default, inheriting +As an expression, it gives the value of the flag: the text of a string flag or the `Boolean` of a boolean flag, or +the `defaultValue` of the endpoint (`false` unless set) when the flag is missing or is another type, see below. +It uses the `openfeature:flags` endpoint by default, inheriting component-level configuration such as `flagsResource` or `host`. The language does not modify the exchange — it reads targeting context @@ -157,7 +158,10 @@ YAML:: ==== The `language()` expression defaults to variant (string) evaluation, so -the property receives the string variant value directly. +the property receives the string variant value directly. A boolean flag is +evaluated as a boolean instead, and the property receives a `Boolean`. Number +and object flags are not supported: they give the `defaultValue` and log a +`TYPE_MISMATCH` warning. == Evaluation context @@ -253,9 +257,9 @@ from("direct:start") ---- `createExpression` defaults to variant (string) evaluation and returns an -`Expression`. When called with `evaluate(exchange, Boolean.class)`, it -automatically switches to boolean evaluation for that call without -affecting other callers: +`Expression`, which gives a `Boolean` for a boolean flag. When called with +`evaluate(exchange, Boolean.class)`, it automatically switches to boolean +evaluation for that call without affecting other callers: [source,java] ---- @@ -263,7 +267,7 @@ Expression variant = context.resolveLanguage("openfeature").createExpression( "hazmat-compliance-v2", new Object[] { null, "order-123", Map.of("customer_tier", "ENTERPRISE") }); -Object result = variant.evaluate(exchange, Object.class); // returns String +Object result = variant.evaluate(exchange, Object.class); // returns String, or Boolean for a boolean flag ---- [source,java] diff --git a/components/camel-openfeature/src/main/java/org/apache/camel/component/openfeature/OpenFeatureEndpoint.java b/components/camel-openfeature/src/main/java/org/apache/camel/component/openfeature/OpenFeatureEndpoint.java index 9cdcf57dcb86..2b360113d93c 100644 --- a/components/camel-openfeature/src/main/java/org/apache/camel/component/openfeature/OpenFeatureEndpoint.java +++ b/components/camel-openfeature/src/main/java/org/apache/camel/component/openfeature/OpenFeatureEndpoint.java @@ -31,6 +31,7 @@ import dev.openfeature.contrib.providers.flagd.Config; import dev.openfeature.contrib.providers.flagd.FlagdOptions; import dev.openfeature.contrib.providers.flagd.FlagdProvider; import dev.openfeature.sdk.Client; +import dev.openfeature.sdk.ErrorCode; import dev.openfeature.sdk.FeatureProvider; import dev.openfeature.sdk.FlagEvaluationDetails; import dev.openfeature.sdk.MutableContext; @@ -144,29 +145,56 @@ public class OpenFeatureEndpoint extends DefaultEndpoint { * side effects on headers or properties. */ public Object evaluate(String flagKey, String evaluationType, String targetingKey, Map<String, Object> contextMap) { + return evaluate(flagKey, evaluationType, targetingKey, contextMap, false); + } + + /** + * Evaluate a flag from the language path with explicit parameters. Does not read from or write to the exchange — no + * side effects on headers or properties. + * + * @param booleanFallback when the string evaluation reports that the flag is not a string flag + * ({@link ErrorCode#TYPE_MISMATCH}), evaluate the flag again as a boolean, so a boolean flag + * gives its value + */ + public Object evaluate( + String flagKey, String evaluationType, String targetingKey, Map<String, Object> contextMap, + boolean booleanFallback) { MutableContext ctx = buildMutableContext(targetingKey, contextMap); - return evaluateDetails(flagKey, evaluationType, ctx).getValue(); + FlagEvaluationDetails<?> details = resolveDetails(flagKey, evaluationType, ctx); + if (booleanFallback && details.getErrorCode() == ErrorCode.TYPE_MISMATCH && !isBooleanEvaluation(evaluationType)) { + FlagEvaluationDetails<?> booleanDetails = resolveDetails(flagKey, "boolean", ctx); + if (booleanDetails.getErrorCode() == null) { + details = booleanDetails; + } + } + logEvaluationError(flagKey, details); + return details.getValue(); } private FlagEvaluationDetails<?> evaluateDetails(String flagKey, String evaluationType, MutableContext ctx) { + FlagEvaluationDetails<?> details = resolveDetails(flagKey, evaluationType, ctx); + logEvaluationError(flagKey, details); + return details; + } + + private FlagEvaluationDetails<?> resolveDetails(String flagKey, String evaluationType, MutableContext ctx) { Client c = client; if (c == null) { throw new IllegalStateException("OpenFeature endpoint is not started"); } - FlagEvaluationDetails<?> details; if (isBooleanEvaluation(evaluationType)) { boolean defaultVal = Boolean.parseBoolean(configuration.getDefaultValue()); - details = c.getBooleanDetails(flagKey, defaultVal, ctx); - } else { - details = c.getStringDetails(flagKey, configuration.getDefaultValue(), ctx); + return c.getBooleanDetails(flagKey, defaultVal, ctx); } + return c.getStringDetails(flagKey, configuration.getDefaultValue(), ctx); + } + private static void logEvaluationError(String flagKey, FlagEvaluationDetails<?> details) { if (details.getErrorCode() != null) { LOG.warn("OpenFeature evaluation error for flag '{}': {} - {}", flagKey, details.getErrorCode(), details.getErrorMessage()); } - return details; } @SuppressWarnings("unchecked") diff --git a/components/camel-openfeature/src/main/java/org/apache/camel/language/openfeature/OpenFeatureLanguage.java b/components/camel-openfeature/src/main/java/org/apache/camel/language/openfeature/OpenFeatureLanguage.java index 8ce1fe1639d1..f87a03596ab3 100644 --- a/components/camel-openfeature/src/main/java/org/apache/camel/language/openfeature/OpenFeatureLanguage.java +++ b/components/camel-openfeature/src/main/java/org/apache/camel/language/openfeature/OpenFeatureLanguage.java @@ -41,9 +41,10 @@ import org.apache.camel.support.LanguageSupport; @BodyTypes(expressionIn = @BodyType(value = Carrier.ANY, description = "Not read: the expression is the flag key, and the targeting key and the" + " context come from headers"), - expressionOut = @BodyType(value = Carrier.TEXT, javaType = "java.lang.String", - description = "The string value of the flag, or the defaultValue of the endpoint when the" - + " flag is not a string flag or is missing")) + expressionOut = @BodyType(value = { Carrier.TEXT, Carrier.VALUE }, + description = "The value of a string flag as text, or of a boolean flag as a Boolean, and" + + " the defaultValue of the endpoint as text when the flag is missing or is" + + " another type")) public class OpenFeatureLanguage extends LanguageSupport { private String endpoint = "openfeature:flags"; @@ -62,7 +63,8 @@ public class OpenFeatureLanguage extends LanguageSupport { property(String.class, properties, 0, endpoint), property(String.class, properties, 1, null), property(Map.class, properties, 2, null), - property(String.class, properties, 3, "boolean")); + property(String.class, properties, 3, "boolean"), + false); if (getCamelContext() != null) { answer.init(getCamelContext()); } @@ -83,7 +85,8 @@ public class OpenFeatureLanguage extends LanguageSupport { property(String.class, properties, 0, endpoint), property(String.class, properties, 1, null), property(Map.class, properties, 2, null), - property(String.class, properties, 3, "variant")); + property(String.class, properties, 3, "variant"), + true); if (getCamelContext() != null) { answer.init(getCamelContext()); } @@ -112,15 +115,18 @@ public class OpenFeatureLanguage extends LanguageSupport { private final String targetingKey; private final Map<String, Object> contextMap; private final String evaluationType; + // an expression gives the value of a boolean flag too, when the string evaluation reports a type mismatch + private final boolean booleanFallback; private OpenFeatureEndpoint endpoint; private Evaluation(String flagKey, String endpointUri, String targetingKey, - Map<String, Object> contextMap, String evaluationType) { + Map<String, Object> contextMap, String evaluationType, boolean booleanFallback) { this.flagKey = flagKey; this.endpointUri = endpointUri; this.targetingKey = targetingKey; this.contextMap = contextMap; this.evaluationType = evaluationType; + this.booleanFallback = booleanFallback; } @Override @@ -165,7 +171,7 @@ public class OpenFeatureLanguage extends LanguageSupport { ctx = (Map<String, Object>) ctxHeader; } } - return endpoint.evaluate(flagKey, evalType, tk, ctx); + return endpoint.evaluate(flagKey, evalType, tk, ctx, booleanFallback); } catch (Exception e) { throw RuntimeCamelException.wrapRuntimeCamelException(e); } diff --git a/components/camel-openfeature/src/test/java/org/apache/camel/component/openfeature/OpenFeatureLanguageTest.java b/components/camel-openfeature/src/test/java/org/apache/camel/component/openfeature/OpenFeatureLanguageTest.java index ba1d246b11a2..040fcc2a4753 100644 --- a/components/camel-openfeature/src/test/java/org/apache/camel/component/openfeature/OpenFeatureLanguageTest.java +++ b/components/camel-openfeature/src/test/java/org/apache/camel/component/openfeature/OpenFeatureLanguageTest.java @@ -16,7 +16,9 @@ */ package org.apache.camel.component.openfeature; +import java.util.List; import java.util.Map; +import java.util.concurrent.CopyOnWriteArrayList; import org.apache.camel.CamelContext; import org.apache.camel.Exchange; @@ -27,6 +29,14 @@ import org.apache.camel.component.mock.MockEndpoint; import org.apache.camel.language.openfeature.OpenFeatureLanguage; import org.apache.camel.test.junit6.CamelTestSupport; import org.apache.camel.test.junit6.TestSupport; +import org.apache.logging.log4j.Level; +import org.apache.logging.log4j.LogManager; +import org.apache.logging.log4j.core.LogEvent; +import org.apache.logging.log4j.core.Logger; +import org.apache.logging.log4j.core.appender.AbstractAppender; +import org.apache.logging.log4j.core.config.Property; +import org.junit.jupiter.api.AfterEach; +import org.junit.jupiter.api.BeforeEach; import org.junit.jupiter.api.Test; import static org.assertj.core.api.Assertions.assertThat; @@ -36,6 +46,35 @@ class OpenFeatureLanguageTest extends CamelTestSupport { private static final String FLAGS_RESOURCE = "classpath:openfeature/flags.json"; + private final List<String> warnings = new CopyOnWriteArrayList<>(); + private final AbstractAppender appender + = new AbstractAppender("OpenFeatureEvaluationWarning", null, null, true, Property.EMPTY_ARRAY) { + @Override + public void append(LogEvent event) { + if (event.getLevel() == Level.WARN) { + warnings.add(event.getMessage().getFormattedMessage()); + } + } + }; + + @BeforeEach + void captureWarnings() { + appender.start(); + endpointLogger().addAppender(appender); + // addAppender creates a non-additive logger config, which would hide this logger from the root appenders + endpointLogger().setAdditive(true); + } + + @AfterEach + void removeAppender() { + endpointLogger().removeAppender(appender); + appender.stop(); + } + + private static Logger endpointLogger() { + return (Logger) LogManager.getLogger(OpenFeatureEndpoint.class); + } + @Override protected CamelContext createCamelContext() throws Exception { CamelContext camelContext = super.createCamelContext(); @@ -80,6 +119,21 @@ class OpenFeatureLanguageTest extends CamelTestSupport { .to("mock:lang-filtered") .end() .to("openfeature:flags/variant?flagKey=hazmat-compliance-v2&resultProperty=variant"); + + from("direct:set-body-boolean") + .setBody().language("openfeature", "enrichment-enabled"); + + from("direct:set-header-boolean") + .setHeader("flag").language("openfeature", "targeted-boolean"); + + from("direct:set-property-string") + .setProperty("flag").language("openfeature", "hazmat-compliance-v2"); + + from("direct:set-body-number") + .setBody().language("openfeature", "max-retries"); + + from("direct:set-body-missing") + .setBody().language("openfeature", "no-such-flag"); } }; } @@ -227,6 +281,57 @@ class OpenFeatureLanguageTest extends CamelTestSupport { assertThat(result).isEqualTo(true); } + @Test + void testSetBodyWithBooleanFlag() { + Object body = template.requestBody("direct:set-body-boolean", "test-message"); + + assertThat(body).isInstanceOf(Boolean.class).isEqualTo(Boolean.TRUE); + assertThat(warnings).isEmpty(); + } + + @Test + void testSetHeaderWithTargetedBooleanFlag() { + Exchange enterprise = template.request("direct:set-header-boolean", e -> { + e.getMessage().setHeader(OpenFeatureConstants.TARGETING_KEY, "user-123"); + e.getMessage().setHeader(OpenFeatureConstants.EVALUATION_CONTEXT, Map.of("customer_tier", "ENTERPRISE")); + }); + Exchange standard = template.request("direct:set-header-boolean", e -> { + e.getMessage().setHeader(OpenFeatureConstants.TARGETING_KEY, "user-456"); + e.getMessage().setHeader(OpenFeatureConstants.EVALUATION_CONTEXT, Map.of("customer_tier", "STANDARD")); + }); + + assertThat(enterprise.getMessage().getHeader("flag")).isInstanceOf(Boolean.class).isEqualTo(Boolean.TRUE); + assertThat(standard.getMessage().getHeader("flag")).isInstanceOf(Boolean.class).isEqualTo(Boolean.FALSE); + assertThat(warnings).isEmpty(); + } + + @Test + void testSetPropertyWithStringFlag() { + Exchange exchange = template.request("direct:set-property-string", e -> { + e.getMessage().setHeader(OpenFeatureConstants.TARGETING_KEY, "order-123"); + e.getMessage().setHeader(OpenFeatureConstants.EVALUATION_CONTEXT, Map.of("customer_tier", "ENTERPRISE")); + }); + + assertThat(exchange.getProperty("flag")).isEqualTo("v2"); + assertThat(warnings).isEmpty(); + } + + @Test + void testSetBodyWithMissingFlagGivesDefaultValue() { + Object body = template.requestBody("direct:set-body-missing", "test-message"); + + assertThat(body).isEqualTo("false"); + assertThat(warnings).hasSize(1).first().asString().contains("no-such-flag", "FLAG_NOT_FOUND"); + } + + @Test + void testSetBodyWithNumberFlagGivesDefaultValue() { + Object body = template.requestBody("direct:set-body-number", "test-message"); + + assertThat(body).isEqualTo("false"); + assertThat(warnings).hasSize(1).first().asString().contains("max-retries", "TYPE_MISMATCH"); + } + @Test void testValidatesBlankFlagKey() { OpenFeatureLanguage language = new OpenFeatureLanguage(); diff --git a/components/camel-openfeature/src/test/resources/log4j2.properties b/components/camel-openfeature/src/test/resources/log4j2.properties new file mode 100644 index 000000000000..70e002c20000 --- /dev/null +++ b/components/camel-openfeature/src/test/resources/log4j2.properties @@ -0,0 +1,29 @@ +## --------------------------------------------------------------------------- +## Licensed to the Apache Software Foundation (ASF) under one or more +## contributor license agreements. See the NOTICE file distributed with +## this work for additional information regarding copyright ownership. +## The ASF licenses this file to You under the Apache License, Version 2.0 +## (the "License"); you may not use this file except in compliance with +## the License. You may obtain a copy of the License at +## +## http://www.apache.org/licenses/LICENSE-2.0 +## +## Unless required by applicable law or agreed to in writing, software +## distributed under the License is distributed on an "AS IS" BASIS, +## WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +## See the License for the specific language governing permissions and +## limitations under the License. +## --------------------------------------------------------------------------- + +appender.file.type = File +appender.file.name = file +appender.file.fileName = target/camel-openfeature-test.log +appender.file.layout.type = PatternLayout +appender.file.layout.pattern = %d %-5p %c{1} - %m %n +appender.out.type = Console +appender.out.name = out +appender.out.layout.type = PatternLayout +appender.out.layout.pattern = [%30.30t] %-30.30c{1} %-5p %m%n + +rootLogger.level = INFO +rootLogger.appenderRef.file.ref = file diff --git a/components/camel-openfeature/src/test/resources/openfeature/flags.json b/components/camel-openfeature/src/test/resources/openfeature/flags.json index cc1fff2f034a..025cb1696f2b 100644 --- a/components/camel-openfeature/src/test/resources/openfeature/flags.json +++ b/components/camel-openfeature/src/test/resources/openfeature/flags.json @@ -32,6 +32,11 @@ ] } }, + "max-retries": { + "state": "ENABLED", + "variants": { "low": 3, "high": 5 }, + "defaultVariant": "low" + }, "disabled-flag": { "state": "DISABLED", "variants": { "on": true, "off": false }, diff --git a/components/camel-python/src/generated/resources/META-INF/org/apache/camel/language/python/python.json b/components/camel-python/src/generated/resources/META-INF/org/apache/camel/language/python/python.json index bfa3cd98fa59..9f2e6f91931a 100644 --- a/components/camel-python/src/generated/resources/META-INF/org/apache/camel/language/python/python.json +++ b/components/camel-python/src/generated/resources/META-INF/org/apache/camel/language/python/python.json @@ -17,7 +17,7 @@ }, "bodyTypes": { "expressionIn": { "carriers": [ "any" ], "description": "The body as it is, in the variable body: a String is a Python unicode string, and other Java objects keep their methods" }, - "expressionOut": { "carriers": [ "text" ], "javaType": "java.lang.String", "dependsOn": [ "resultType" ], "description": "The Python text, str(), of what the script returns: a dict as its Python text, and None as the text None" } + "expressionOut": { "carriers": [ "unknown" ], "description": "The Java value of what the script returns: a dict as a Map, a list or a tuple as a List, a set as a Set, and None as null" } }, "properties": { "id": { "index": 0, "kind": "attribute", "displayName": "Id", "group": "common", "required": false, "type": "string", "javaType": "java.lang.String", "deprecated": false, "autowired": false, "secret": false, "description": "The id of this node." }, diff --git a/components/camel-python/src/main/docs/python-language.adoc b/components/camel-python/src/main/docs/python-language.adoc index a0e494416a4b..bd4e6a9fc015 100644 --- a/components/camel-python/src/main/docs/python-language.adoc +++ b/components/camel-python/src/main/docs/python-language.adoc @@ -54,8 +54,9 @@ The `variable` and `variables` bindings are the exchange-scoped variables (for e xref:eips:setVariable-eip.adoc[Set Variable] EIP). Global and route variables, such as `global:foo`, are not in this map; use `exchange.getVariable('global:foo')` for those. -The result of a script is its Python text, as `str()` gives it: a dict gives its Python text, and `None` gives the -text `None`. Set `resultType` to convert the text to another type. +The result of a script is its Java value: `None` is `null`, a string, a number or a boolean is the Java `String`, +number or `Boolean`, a dict, list, tuple or set is a `Map`, `List` or `Set` of Java values, and a Java object, such as +the body, is returned as it is. Set `resultType` to convert the result to another type. == Examples diff --git a/components/camel-python/src/main/java/org/apache/camel/language/python/PythonExpression.java b/components/camel-python/src/main/java/org/apache/camel/language/python/PythonExpression.java index af1ab5077744..3a50c9ce489f 100644 --- a/components/camel-python/src/main/java/org/apache/camel/language/python/PythonExpression.java +++ b/components/camel-python/src/main/java/org/apache/camel/language/python/PythonExpression.java @@ -16,9 +16,16 @@ */ package org.apache.camel.language.python; +import java.util.ArrayList; +import java.util.Collection; +import java.util.LinkedHashMap; +import java.util.LinkedHashSet; +import java.util.List; import java.util.Map; +import java.util.Set; import org.apache.camel.Exchange; +import org.apache.camel.ExpressionEvaluationException; import org.apache.camel.ExpressionIllegalSyntaxException; import org.apache.camel.Message; import org.apache.camel.support.ExpressionSupport; @@ -85,18 +92,53 @@ public class PythonExpression extends ExpressionSupport { compiler.set("properties", properties); PyObject out = compiler.eval(compiledExpression); - if (out != null) { - String value = out.toString(); + Object value = toJava(out); + if (value != null) { return exchange.getContext().getTypeConverter().convertTo(type, value); } } catch (Exception e) { - throw new ExpressionIllegalSyntaxException(expressionString, e); + // the script compiled in the constructor, so this is an error while it runs + throw new ExpressionEvaluationException(this, exchange, e); } finally { compiler.cleanup(); } return null; } + /** + * The Java value of what a script returns: None is null, a Python string, number or boolean is the Java String, + * number or Boolean, and a Java object bound to the script (such as the body) is that object. A dict, list, tuple + * or set is copied into a LinkedHashMap, ArrayList or LinkedHashSet of Java values, as the Jython collections are + * live views that give their Python text from toString() and keep nested Python collections. + */ + static Object toJava(Object value) { + if (!(value instanceof PyObject py)) { + return value; + } + Object answer = py.__tojava__(Object.class); + if (!(answer instanceof PyObject)) { + // None, a Python string, number or boolean, or a Java object + return answer; + } + if (answer instanceof Map<?, ?> map) { + Map<Object, Object> copy = new LinkedHashMap<>(); + map.forEach((k, v) -> copy.put(toJava(k), toJava(v))); + return copy; + } else if (answer instanceof List<?> list) { + return copyInto(list, new ArrayList<>(list.size())); + } else if (answer instanceof Set<?> set) { + return copyInto(set, new LinkedHashSet<>()); + } + return answer; + } + + private static <C extends Collection<Object>> C copyInto(Collection<?> source, C target) { + for (Object element : source) { + target.add(toJava(element)); + } + return target; + } + public Class<?> getType() { return type; } diff --git a/components/camel-python/src/main/java/org/apache/camel/language/python/PythonLanguage.java b/components/camel-python/src/main/java/org/apache/camel/language/python/PythonLanguage.java index 0b0b2298567e..c3739da09eac 100644 --- a/components/camel-python/src/main/java/org/apache/camel/language/python/PythonLanguage.java +++ b/components/camel-python/src/main/java/org/apache/camel/language/python/PythonLanguage.java @@ -21,6 +21,7 @@ import java.util.HashMap; import java.util.Map; import org.apache.camel.Expression; +import org.apache.camel.ExpressionEvaluationException; import org.apache.camel.ExpressionIllegalSyntaxException; import org.apache.camel.Predicate; import org.apache.camel.spi.BodyType; @@ -38,9 +39,9 @@ import org.python.util.PythonInterpreter; @BodyTypes(expressionIn = @BodyType(value = Carrier.ANY, description = "The body as it is, in the variable body: a String is a Python unicode" + " string, and other Java objects keep their methods"), - expressionOut = @BodyType(value = Carrier.TEXT, javaType = "java.lang.String", dependsOn = "resultType", - description = "The Python text, str(), of what the script returns: a dict as its Python" - + " text, and None as the text None")) + expressionOut = @BodyType(value = Carrier.UNKNOWN, + description = "The Java value of what the script returns: a dict as a Map, a list or a" + + " tuple as a List, a set as a Set, and None as null")) public class PythonLanguage extends TypedLanguageSupport implements ScriptingLanguage { private final Map<String, PyCode> compiledScriptsCache; @@ -92,12 +93,13 @@ public class PythonLanguage extends TypedLanguageSupport implements ScriptingLan bindings.forEach(compiler::set); } PyObject out = compiler.eval(code); - if (out != null) { - String value = out.toString(); + Object value = PythonExpression.toJava(out); + if (value != null) { return getCamelContext().getTypeConverter().convertTo(resultType, value); } } catch (Exception e) { - throw new ExpressionIllegalSyntaxException(script, e); + // the script compiled above, so this is an error while it runs + throw new ExpressionEvaluationException(null, "Error evaluating python script: " + script, null, e); } finally { compiler.cleanup(); } diff --git a/components/camel-python/src/test/java/org/apache/camel/language/python/PythonResultTest.java b/components/camel-python/src/test/java/org/apache/camel/language/python/PythonResultTest.java new file mode 100644 index 000000000000..6229999a6bd9 --- /dev/null +++ b/components/camel-python/src/test/java/org/apache/camel/language/python/PythonResultTest.java @@ -0,0 +1,134 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.camel.language.python; + +import java.util.ArrayList; +import java.util.LinkedHashMap; +import java.util.List; +import java.util.Map; +import java.util.Set; + +import org.apache.camel.ExpressionEvaluationException; +import org.apache.camel.ExpressionIllegalSyntaxException; +import org.apache.camel.spi.Language; +import org.apache.camel.spi.ScriptingLanguage; +import org.apache.camel.test.junit6.LanguageTestSupport; +import org.junit.jupiter.api.Test; + +import static org.junit.jupiter.api.Assertions.assertEquals; +import static org.junit.jupiter.api.Assertions.assertInstanceOf; +import static org.junit.jupiter.api.Assertions.assertNull; +import static org.junit.jupiter.api.Assertions.assertSame; +import static org.junit.jupiter.api.Assertions.assertThrows; +import static org.junit.jupiter.api.Assertions.assertTrue; + +/** + * A script returns the Java value of its result, not its Python text, and an error while it runs is an evaluation + * error, not a syntax error. + */ +class PythonResultTest extends LanguageTestSupport { + + private Object evaluate(String script) { + Language language = context.resolveLanguage("python"); + return language.createExpression(script).evaluate(exchange, Object.class); + } + + private ScriptingLanguage scripting() { + return (ScriptingLanguage) context.resolveLanguage("python"); + } + + @Test + void pythonValuesAreJavaValues() { + assertNull(evaluate("None")); + assertEquals(3, evaluate("1 + 2")); + assertEquals(1.5, evaluate("3 / 2.0")); + assertEquals(Boolean.TRUE, evaluate("1 == 1")); + assertEquals("abc", evaluate("'abc'")); + assertEquals("abc", evaluate("u'abc'")); + } + + @Test + void bodyIsReturnedAsItIs() { + byte[] data = { 1, 2, 3 }; + exchange.getMessage().setBody(data); + + assertSame(data, evaluate("body")); + } + + @Test + void collectionsAreJavaCollections() { + Object map = evaluate("{'a': 1, 'b': [1, None, {'c': 'd'}]}"); + assertInstanceOf(LinkedHashMap.class, map); + Map<String, Object> nested = new LinkedHashMap<>(); + nested.put("c", "d"); + List<Object> list = new ArrayList<>(); + list.add(1); + list.add(null); + list.add(nested); + assertEquals(Map.of("a", 1, "b", list), map); + assertInstanceOf(LinkedHashMap.class, ((List<?>) ((Map<?, ?>) map).get("b")).get(2)); + + assertEquals(List.of(1, "x"), evaluate("(1, 'x')")); + assertEquals(Set.of(2), evaluate("set([2])")); + } + + @Test + void resultIsConvertedToTheRequestedType() { + assertExpression("1 + 2", "3"); + assertExpression("'5'", 5); + } + + @Test + void noneIsAFalsePredicate() { + assertPredicate("None", false); + assertPredicate("1 == 2", false); + assertPredicate("1 == 1", true); + } + + @Test + void scriptingReturnsJavaValues() { + assertNull(scripting().evaluate("None", null, Object.class)); + assertEquals(3, scripting().evaluate("1 + 2", null, Object.class)); + assertEquals(Map.of("a", 1), scripting().evaluate("{'a': 1}", null, Object.class)); + } + + @Test + void runtimeErrorIsAnEvaluationError() { + Language language = context.resolveLanguage("python"); + + ExpressionEvaluationException e = assertThrows(ExpressionEvaluationException.class, + () -> language.createExpression("unknown + 1").evaluate(exchange, Object.class)); + assertTrue(e.getCause().toString().contains("NameError")); + + e = assertThrows(ExpressionEvaluationException.class, + () -> scripting().evaluate("1 / 0", null, Object.class)); + assertTrue(e.getCause().toString().contains("ZeroDivisionError")); + } + + @Test + void syntaxErrorIsASyntaxError() { + Language language = context.resolveLanguage("python"); + + assertThrows(ExpressionIllegalSyntaxException.class, () -> language.createExpression("1 +")); + assertThrows(ExpressionIllegalSyntaxException.class, () -> scripting().evaluate("1 +", null, Object.class)); + } + + @Override + protected String getLanguageName() { + return "python"; + } +} diff --git a/components/camel-python3/src/generated/resources/META-INF/org/apache/camel/language/python3/python3.json b/components/camel-python3/src/generated/resources/META-INF/org/apache/camel/language/python3/python3.json index f3f647ace109..7fd5d897b4ae 100644 --- a/components/camel-python3/src/generated/resources/META-INF/org/apache/camel/language/python3/python3.json +++ b/components/camel-python3/src/generated/resources/META-INF/org/apache/camel/language/python3/python3.json @@ -17,7 +17,7 @@ }, "bodyTypes": { "expressionIn": { "carriers": [ "any" ], "description": "The body as it is, in the variable body: a Map, a List or an array can be indexed, and other Java objects have no methods unless the language is created with createWithHostAccess()" }, - "expressionOut": { "carriers": [ "unknown" ], "description": "What the script returns: a dict as a Map, a list or a tuple as a List, a set as a Set, bytes as a List of numbers, and None as null" } + "expressionOut": { "carriers": [ "unknown" ], "description": "What the script returns: a dict as a Map, a list or a tuple as a List, a set as a Set, bytes as bytes, and None as null. A function fails" } }, "properties": { "id": { "index": 0, "kind": "attribute", "displayName": "Id", "group": "common", "required": false, "type": "string", "javaType": "java.lang.String", "deprecated": false, "autowired": false, "secret": false, "description": "The id of this node." }, diff --git a/components/camel-python3/src/main/java/org/apache/camel/language/python3/Python3Language.java b/components/camel-python3/src/main/java/org/apache/camel/language/python3/Python3Language.java index dd921ce8c2b5..3fcac2e23ff7 100644 --- a/components/camel-python3/src/main/java/org/apache/camel/language/python3/Python3Language.java +++ b/components/camel-python3/src/main/java/org/apache/camel/language/python3/Python3Language.java @@ -80,9 +80,19 @@ import org.graalvm.polyglot.Value; + " created with createWithHostAccess()"), expressionOut = @BodyType(value = Carrier.UNKNOWN, description = "What the script returns: a dict as a Map, a list or a tuple as a List, a set" - + " as a Set, bytes as a List of numbers, and None as null")) + + " as a Set, bytes as bytes, and None as null. A function fails")) public class Python3Language extends TypedLanguageSupport implements ScriptingLanguage, Service { + static final String FUNCTION_RESULT + = "The script returned a function, which cannot be used after the script ends: return a value instead"; + + /** + * The built-in Python types of functions and methods: a def or lambda, a bound method, a built-in function or bound + * built-in method, and the method wrappers of built-in types. + */ + private static final Set<String> PYTHON_FUNCTION_TYPES + = Set.of("function", "method", "builtin_function_or_method", "method-wrapper", "method_descriptor"); + private final HostAccess hostAccess; /** * When true, also bind the host objects {@code exchange}, {@code camelContext}, {@code message}, {@code request} @@ -265,6 +275,10 @@ public class Python3Language extends TypedLanguageSupport implements ScriptingLa */ @SuppressWarnings("unchecked") static <T> T convert(Value value, Class<T> resultType, CamelContext camelContext, Exchange exchange) { + if (isPythonFunction(value)) { + // the function is tied to the per-evaluation Context, which is closed as soon as the script ends + throw new ExpressionEvaluationException(null, FUNCTION_RESULT, exchange, null); + } Object obj = materialize(value, new HashMap<>()); if (resultType == null || resultType == Object.class) { return (T) obj; @@ -287,8 +301,9 @@ public class Python3Language extends TypedLanguageSupport implements ScriptingLa /** * Copies guest values into ordinary Java types so the result remains usable after {@link Context#close()}. * <p> - * Python lists and tuples become {@link List}, dicts become {@link Map}, and sets become {@link Set}. Nested values - * are copied recursively. Other guest objects (including custom types) are left as {@link Value#as(Class) + * Python lists and tuples become {@link List}, dicts become {@link Map}, sets become {@link Set}, and bytes, + * bytearray and memoryview (any value with buffer elements) become {@code byte[]}. Nested values are copied + * recursively. Other guest objects (including custom types) are left as {@link Value#as(Class) * value.as(Object.class)}. * <p> * GraalPy returns the {@code __main__} module (not {@link Value#isNull()}) for {@code None}, {@code pass}, and @@ -326,6 +341,11 @@ public class Python3Language extends TypedLanguageSupport implements ScriptingLa if (isMainModule(value)) { return null; } + if (value.hasBufferElements()) { + byte[] bytes = new byte[Math.toIntExact(value.getBufferSize())]; + value.readBuffer(0, bytes, 0, bytes.length); + return bytes; + } if (isPythonSet(value)) { Set<Object> set = new LinkedHashSet<>(); seen.put(value, set); @@ -376,6 +396,30 @@ public class Python3Language extends TypedLanguageSupport implements ScriptingLa return isPythonType(value, "tuple"); } + /** + * Whether the value is a Python function or method. {@link Value#canExecute()} alone also matches a Java host + * object, a class and an instance of a class with {@code __call__}, which are data; so the type must also be one of + * the built-in function types (a class of the script, even one named {@code function}, has another module). + */ + private static boolean isPythonFunction(Value value) { + if (value == null || !value.canExecute() || value.isHostObject() || value.isMetaObject()) { + return false; + } + Value meta = value.getMetaObject(); + if (meta == null || !meta.isMetaObject()) { + return false; + } + try { + if (!PYTHON_FUNCTION_TYPES.contains(meta.getMetaSimpleName())) { + return false; + } + Value module = meta.getMember("__module__"); + return module != null && module.isString() && "builtins".equals(module.asString()); + } catch (UnsupportedOperationException e) { + return false; + } + } + private static boolean isPythonType(Value value, String simpleName) { Value meta = value.getMetaObject(); if (meta == null || !meta.isMetaObject()) { diff --git a/components/camel-python3/src/test/java/org/apache/camel/language/python3/Python3LanguageEvalTest.java b/components/camel-python3/src/test/java/org/apache/camel/language/python3/Python3LanguageEvalTest.java index 1448fe9c0b12..90f262137ea2 100644 --- a/components/camel-python3/src/test/java/org/apache/camel/language/python3/Python3LanguageEvalTest.java +++ b/components/camel-python3/src/test/java/org/apache/camel/language/python3/Python3LanguageEvalTest.java @@ -30,6 +30,7 @@ import java.util.concurrent.atomic.AtomicInteger; import org.apache.camel.CamelContext; import org.apache.camel.Exchange; import org.apache.camel.Expression; +import org.apache.camel.ExpressionEvaluationException; import org.apache.camel.Predicate; import org.apache.camel.ProducerTemplate; import org.apache.camel.builder.RouteBuilder; @@ -44,6 +45,7 @@ import org.junit.jupiter.api.condition.DisabledIfSystemProperty; import static org.apache.camel.language.python3.Python3Language.python3; import static org.assertj.core.api.Assertions.assertThat; +import static org.assertj.core.api.Assertions.assertThatThrownBy; @DisabledIfSystemProperty(named = "os.arch", matches = "(?i)(s390x|ppc64le)") class Python3LanguageEvalTest { @@ -199,6 +201,62 @@ class Python3LanguageEvalTest { assertThat(inner).containsEntry("k", 2); } + @Test + void bytesResultIsByteArray() { + Language language = context.resolveLanguage("python3"); + Exchange exchange = exchangeWithBody(null); + assertThat(language.createExpression("b'ab\\xff'").evaluate(exchange, Object.class)) + .isEqualTo(new byte[] { 'a', 'b', -1 }); + assertThat(language.createExpression("bytearray(b'xy')").evaluate(exchange, Object.class)) + .isEqualTo(new byte[] { 'x', 'y' }); + assertThat(language.createExpression("memoryview(b'abcd')[1:3]").evaluate(exchange, Object.class)) + .isEqualTo(new byte[] { 'b', 'c' }); + assertThat(language.createExpression("b''").evaluate(exchange, Object.class)).isEqualTo(new byte[0]); + Map<Object, Object> map = castMap(language.createExpression("{'data': b'z'}").evaluate(exchange, Object.class)); + assertThat(map.get("data")).isEqualTo(new byte[] { 'z' }); + assertThat(language.createExpression("b'hi'").evaluate(exchange, String.class)).isEqualTo("hi"); + } + + @Test + void byteArrayBodyIsReturnedAsIs() { + byte[] body = { 1, 2 }; + Object result + = context.resolveLanguage("python3").createExpression("body").evaluate(exchangeWithBody(body), Object.class); + assertThat(result).isSameAs(body); + } + + @Test + void returnedFunctionFailsAtEvaluation() { + String message = "The script returned a function, which cannot be used after the script ends: return a value instead"; + Language language = context.resolveLanguage("python3"); + Exchange exchange = exchangeWithBody(null); + for (String script : List.of("lambda x: x", "def f():\n return 1\nf", + "class C:\n def m(self):\n return 1\nC().m", + "len", "[].append")) { + assertThatThrownBy(() -> language.createExpression(script).evaluate(exchange, Object.class)) + .as(script) + .isInstanceOf(ExpressionEvaluationException.class) + .hasMessage(message); + } + assertThatThrownBy(() -> ((ScriptingLanguage) language).evaluate("lambda: 1", null, Object.class)) + .isInstanceOf(ExpressionEvaluationException.class) + .hasMessage(message); + } + + @Test + void callableDataIsNotTakenForAFunction() { + Language language = context.resolveLanguage("python3"); + Runnable task = () -> { + }; + Map<String, Object> bindings = new HashMap<>(); + bindings.put("task", task); + assertThat(((ScriptingLanguage) language).evaluate("task", bindings, Object.class)).isSameAs(task); + // an instance of a class with __call__, even one named function, is data and not rejected + Object instance = language.createExpression("class function:\n def __call__(self):\n return 1\nfunction()") + .evaluate(exchangeWithBody(null), Object.class); + assertThat(instance).isNotNull(); + } + @Test void resultTypeListAndMap() { Language language = context.resolveLanguage("python3"); diff --git a/components/camel-quickjs/src/generated/resources/META-INF/org/apache/camel/language/quickjs/quickjs.json b/components/camel-quickjs/src/generated/resources/META-INF/org/apache/camel/language/quickjs/quickjs.json index f4e206c0b39f..9ad3742d017c 100644 --- a/components/camel-quickjs/src/generated/resources/META-INF/org/apache/camel/language/quickjs/quickjs.json +++ b/components/camel-quickjs/src/generated/resources/META-INF/org/apache/camel/language/quickjs/quickjs.json @@ -17,7 +17,7 @@ }, "bodyTypes": { "expressionIn": { "carriers": [ "text", "bytes", "value", "map", "list", "pojo" ], "description": "A JSON copy of the body, in the variable body: bytes as Base64 text, and a POJO through Jackson. A stream fails: convert it to text first" }, - "expressionOut": { "carriers": [ "text", "value", "map", "list", "null" ], "dependsOn": [ "resultType" ], "description": "The JSON value the script returns: an object as a Map, an array as a List, and undefined, NaN or a function as null" } + "expressionOut": { "carriers": [ "text", "value", "map", "list", "null" ], "dependsOn": [ "resultType" ], "description": "The JSON value the script returns: an object or a Map as a Map, an array, a Set or a typed array as a List, a Date as its ISO text, and undefined, NaN or a function as null" } }, "properties": { "id": { "index": 0, "kind": "attribute", "displayName": "Id", "group": "common", "required": false, "type": "string", "javaType": "java.lang.String", "deprecated": false, "autowired": false, "secret": false, "description": "The id of this node." }, diff --git a/components/camel-quickjs/src/main/docs/quickjs-language.adoc b/components/camel-quickjs/src/main/docs/quickjs-language.adoc index ff56cae58b61..d6b600a9e229 100644 --- a/components/camel-quickjs/src/main/docs/quickjs-language.adoc +++ b/components/camel-quickjs/src/main/docs/quickjs-language.adoc @@ -163,7 +163,8 @@ the unsupported type. As an expression, the JSON value of the script becomes the Camel result (then converted with Camel type converters when a result type is requested): an object becomes a `Map` and an array a `List`, a `Date` becomes its ISO text, and `undefined`, `NaN` or a function becomes `null`. -A `Set` or a typed array does not keep its values. In `setBody` and `transform` this result +A `Set` becomes a `List`, a JavaScript `Map` a `Map` (its keys as text), and a typed array or an +`ArrayBuffer` a `List` of numbers, also when nested. In `setBody` and `transform` this result replaces a body the script set with `camel.setBody(...)`. As a predicate (`.when().quickjs(...)` or `.filter().quickjs(...)`), the result is converted to diff --git a/components/camel-quickjs/src/main/java/org/apache/camel/language/quickjs/QuickjsHelper.java b/components/camel-quickjs/src/main/java/org/apache/camel/language/quickjs/QuickjsHelper.java index ac5f97bca2d4..c9a37aad5a23 100644 --- a/components/camel-quickjs/src/main/java/org/apache/camel/language/quickjs/QuickjsHelper.java +++ b/components/camel-quickjs/src/main/java/org/apache/camel/language/quickjs/QuickjsHelper.java @@ -152,6 +152,78 @@ final class QuickjsHelper { */ static final String CAMEL_FACADE = camelFacade(); + /** + * JavaScript that prepares the script result for the JSON hand-over to Java, where a {@code Set} or a {@code Map} + * would become an empty object and a typed array an object of index to value: a {@code Set} becomes an array, a + * {@code Map} an object (its keys as strings), and a typed array or an {@code ArrayBuffer} an array of numbers, + * also when nested in objects and arrays. A primitive is returned after one type check, and an object or array is + * only copied when something in it is converted. A value with {@code toJSON} (such as a {@code Date}) and a cyclic + * reference are left to JSON as before. The built-ins are captured once so a script cannot replace them. + */ + private static final String RESULT_CONVERTER = """ + const __camelResult = (() => { + const SetType = Set; + const MapType = Map; + const ArrayBufferType = ArrayBuffer; + const DataViewType = DataView; + const Uint8ArrayType = Uint8Array; + const isView = ArrayBuffer.isView; + const isArray = Array.isArray; + const from = Array.from; + const keys = Object.keys; + const fromEntries = Object.fromEntries; + const define = Object.defineProperty; + const isObject = v => typeof v === "object" && v !== null; + const put = (o, k, v) => define(o, k, { value: v, writable: true, enumerable: true, configurable: true }); + const convert = (v, path) => { + if (typeof v.toJSON === "function" || path.includes(v)) { + return v; + } + if (v instanceof ArrayBufferType) { + return from(new Uint8ArrayType(v)); + } + if (isView(v)) { + return v instanceof DataViewType ? v : from(v); + } + const element = x => isObject(x) ? convert(x, path) : x; + path.push(v); + let copy; + if (v instanceof SetType) { + copy = from(v, element); + } else if (v instanceof MapType) { + copy = fromEntries(from(v, ([k, x]) => [String(k), element(x)])); + } else if (isArray(v)) { + for (let i = 0; i < v.length; i++) { + const x = v[i]; + const c = element(x); + if (c !== x) { + copy = copy || v.slice(); + copy[i] = c; + } + } + } else { + const names = keys(v); + for (let i = 0; i < names.length; i++) { + const x = v[names[i]]; + const c = element(x); + if (c !== x && !copy) { + copy = {}; + for (let j = 0; j < i; j++) { + put(copy, names[j], v[names[j]]); + } + } + if (copy) { + put(copy, names[i], c); + } + } + } + path.pop(); + return copy || v; + }; + return v => isObject(v) ? convert(v, []) : v; + })(); + """; + /** * Prologue and epilogue shared by every script function: QuickJS4J host identifiers on {@code globalThis} are * replaced with stubs while the user script runs and restored afterwards so result delivery still works on a reused @@ -186,7 +258,7 @@ final class QuickjsHelper { * declarations do not leak into {@code globalThis} between calls. Binding names must be valid JavaScript * identifiers. */ - static final String EVAL_WRAPPER = CAMEL_FACADE + static final String EVAL_WRAPPER = CAMEL_FACADE + RESULT_CONVERTER + """ export function camelEval(bindings, script) { const names = Object.keys(bindings); @@ -195,7 +267,7 @@ final class QuickjsHelper { '"use strict"; return eval(__camel_quickjs_script);'); """ + indent(SANDBOX_ENTER, 2) - + " return fn(...values, __camelFacade, __camel_stub_invoke, undefined, script);\n" + + " return __camelResult(fn(...values, __camelFacade, __camel_stub_invoke, undefined, script));\n" + indent(SANDBOX_EXIT, 2) + "}\n"; @@ -235,7 +307,7 @@ final class QuickjsHelper { * exactly as before. */ static String scriptLibrary(String script, boolean expressionForm) { - StringBuilder sb = new StringBuilder(CAMEL_FACADE); + StringBuilder sb = new StringBuilder(CAMEL_FACADE).append(RESULT_CONVERTER); sb.append("function __camelScript(").append(String.join(", ", EXCHANGE_BINDINGS)) .append(", camel, java_invoke, quickjs4j_engine) {\n \"use strict\";\n"); if (expressionForm) { @@ -246,11 +318,11 @@ final class QuickjsHelper { } sb.append("export function camelEval(b, s) {\n"); sb.append(indent(SANDBOX_ENTER, 1)); - sb.append(" return __camelScript("); + sb.append(" return __camelResult(__camelScript("); for (String name : EXCHANGE_BINDINGS) { sb.append("b.").append(name).append(", "); } - sb.append("__camelFacade, __camel_stub_invoke, undefined);\n"); + sb.append("__camelFacade, __camel_stub_invoke, undefined));\n"); sb.append(indent(SANDBOX_EXIT, 1)); sb.append("}\n"); return sb.toString(); diff --git a/components/camel-quickjs/src/main/java/org/apache/camel/language/quickjs/QuickjsLanguage.java b/components/camel-quickjs/src/main/java/org/apache/camel/language/quickjs/QuickjsLanguage.java index e45dcb558728..a54d1b0c244e 100644 --- a/components/camel-quickjs/src/main/java/org/apache/camel/language/quickjs/QuickjsLanguage.java +++ b/components/camel-quickjs/src/main/java/org/apache/camel/language/quickjs/QuickjsLanguage.java @@ -65,8 +65,9 @@ import run.endive.runtime.Memory; + " POJO through Jackson. A stream fails: convert it to text first"), expressionOut = @BodyType(value = { Carrier.TEXT, Carrier.VALUE, Carrier.MAP, Carrier.LIST, Carrier.NULL }, dependsOn = "resultType", - description = "The JSON value the script returns: an object as a Map, an array as a List," - + " and undefined, NaN or a function as null")) + description = "The JSON value the script returns: an object or a Map as a Map, an array, a" + + " Set or a typed array as a List, a Date as its ISO text, and undefined, NaN" + + " or a function as null")) public class QuickjsLanguage extends TypedLanguageSupport implements ScriptingLanguage, Service { /** diff --git a/components/camel-quickjs/src/test/java/org/apache/camel/language/quickjs/QuickjsSerializationTest.java b/components/camel-quickjs/src/test/java/org/apache/camel/language/quickjs/QuickjsSerializationTest.java index 2e04ed3fb7f9..539435b92bca 100644 --- a/components/camel-quickjs/src/test/java/org/apache/camel/language/quickjs/QuickjsSerializationTest.java +++ b/components/camel-quickjs/src/test/java/org/apache/camel/language/quickjs/QuickjsSerializationTest.java @@ -19,6 +19,9 @@ package org.apache.camel.language.quickjs; import java.io.ByteArrayInputStream; import java.io.StringReader; import java.nio.charset.StandardCharsets; +import java.util.Collections; +import java.util.List; +import java.util.Map; import org.apache.camel.CamelContext; import org.apache.camel.Exchange; @@ -26,12 +29,14 @@ import org.apache.camel.ExpressionEvaluationException; import org.apache.camel.converter.stream.InputStreamCache; import org.apache.camel.impl.DefaultCamelContext; import org.apache.camel.spi.Language; +import org.apache.camel.spi.ScriptingLanguage; import org.apache.camel.support.DefaultExchange; import org.junit.jupiter.api.AfterAll; import org.junit.jupiter.api.BeforeAll; import org.junit.jupiter.api.Test; import static org.assertj.core.api.Assertions.assertThat; +import static org.assertj.core.api.Assertions.assertThatCode; import static org.assertj.core.api.Assertions.assertThatThrownBy; class QuickjsSerializationTest { @@ -123,4 +128,81 @@ class QuickjsSerializationTest { Exchange exchange = new DefaultExchange(context); assertThat(language().createExpression("void 0").evaluate(exchange, Object.class)).isNull(); } + + @Test + void setResultIsList() { + Exchange exchange = new DefaultExchange(context); + assertThat(language().createExpression("new Set([1, 'a', 2])").evaluate(exchange, Object.class)) + .isEqualTo(List.of(1, "a", 2)); + } + + @Test + void mapResultKeepsEntries() { + Exchange exchange = new DefaultExchange(context); + assertThat(language().createExpression("new Map([['a', 1], [2, new Set(['x'])]])").evaluate(exchange, Object.class)) + .isEqualTo(Map.of("a", 1, "2", List.of("x"))); + } + + @Test + void typedArrayResultIsListOfNumbers() { + Exchange exchange = new DefaultExchange(context); + assertThat(language().createExpression("new Uint8Array([1, 2, 255])").evaluate(exchange, Object.class)) + .isEqualTo(List.of(1, 2, 255)); + assertThat(language().createExpression("new Float64Array([1.5])").evaluate(exchange, Object.class)) + .isEqualTo(List.of(1.5)); + assertThat(language().createExpression("new Uint8Array([7, 8]).buffer").evaluate(exchange, Object.class)) + .isEqualTo(List.of(7, 8)); + } + + @Test + void nestedSetMapAndTypedArrayAreConverted() { + Exchange exchange = new DefaultExchange(context); + Object result = language().createExpression( + "({ tags: new Set(['a', 'b']), list: [new Map([['k', 'v']]), new Int8Array([-1])], n: 1, s: 'x' })") + .evaluate(exchange, Object.class); + assertThat(result).isEqualTo(Map.of( + "tags", List.of("a", "b"), + "list", List.of(Map.of("k", "v"), List.of(-1)), + "n", 1, + "s", "x")); + } + + @Test + void statementScriptConvertsSet() { + Exchange exchange = new DefaultExchange(context); + assertThat(language().createExpression("const s = new Set([3, 4]); s;").evaluate(exchange, Object.class)) + .isEqualTo(List.of(3, 4)); + } + + @Test + void scriptingEvaluateConvertsSet() { + ScriptingLanguage scripting = (ScriptingLanguage) language(); + assertThat(scripting.evaluate("({ s: new Set([x, 2]) })", Map.of("x", 1), Object.class)) + .isEqualTo(Map.of("s", List.of(1, 2))); + } + + @Test + void plainResultsAndJsonRulesAreKept() { + Exchange exchange = new DefaultExchange(context); + Language language = language(); + assertThat(language.createExpression("({ a: [1, { b: 'c' }], d: 'e' })").evaluate(exchange, Object.class)) + .isEqualTo(Map.of("a", List.of(1, Map.of("b", "c")), "d", "e")); + assertThat(language.createExpression("NaN").evaluate(exchange, Object.class)).isNull(); + assertThat(language.createExpression("() => 1").evaluate(exchange, Object.class)).isNull(); + assertThat(language.createExpression("({ f: () => 1, u: undefined, n: NaN })").evaluate(exchange, Object.class)) + .isEqualTo(Collections.singletonMap("n", null)); + assertThat(language.createExpression("new Date(0)").evaluate(exchange, Object.class)) + .isEqualTo("1970-01-01T00:00:00.000Z"); + assertThat(language.createExpression("({ when: new Date(0), tags: new Set([1]) })").evaluate(exchange, Object.class)) + .isEqualTo(Map.of("when", "1970-01-01T00:00:00.000Z", "tags", List.of(1))); + } + + @Test + void cyclicResultWithSetDoesNotRecurseForever() { + Exchange exchange = new DefaultExchange(context); + // the cycle is left to JSON as before; the conversion must not follow it into a stack overflow + assertThatCode(() -> language().createExpression("const o = { s: new Set([1]) }; o.self = o; o;") + .evaluate(exchange, Object.class)) + .doesNotThrowAnyException(); + } } diff --git a/docs/user-manual/modules/ROOT/pages/camel-4x-upgrade-guide-4_24.adoc b/docs/user-manual/modules/ROOT/pages/camel-4x-upgrade-guide-4_24.adoc index 97d2460e984d..938de13c6201 100644 --- a/docs/user-manual/modules/ROOT/pages/camel-4x-upgrade-guide-4_24.adoc +++ b/docs/user-manual/modules/ROOT/pages/camel-4x-upgrade-guide-4_24.adoc @@ -2661,3 +2661,52 @@ Node producer creates it, from the options of the Edge Node endpoint (`deviceIds Device route that starts before the Edge Node route, or while it is stopped, therefore no longer fails with a `NullPointerException` when the Device URI has no `deviceIds`. Sending to a Device endpoint while no producer of its Edge Node endpoint is started now fails the exchange with an `IllegalStateException`; before, the data was dropped. + +=== camel-python - a script gives the Java value of its result + +The `python` language (Jython) now gives the Java value of what a script returns, instead of its Python text from +`str()`. `None` is `null`, a string, a number or a boolean is the Java `String`, number or `Boolean`, a dict, list, +tuple or set is a `Map`, `List` or `Set` of Java values, and a Java object bound to the script, such as the body, is +returned as it is. For example `1 + 2` gives the `Integer` 3 instead of the text `3`, and `body` on a byte array gives +the byte array instead of the text `array('b', [...])`. Set `resultType` to `java.lang.String` to get text. + +This also changes predicates: `None` was the text `None`, which counted as true, and is now `null`, which is false. An +empty list was the text `[]`, which counted as true, and is now an empty collection, which is false. + +An error while a script runs, such as a `NameError`, now fails with `ExpressionEvaluationException` instead of +`ExpressionIllegalSyntaxException`, which is kept for a script that does not compile. + +=== camel-openfeature - an expression gives the value of a boolean flag + +Used as an expression, for example in `setBody`, `setHeader` or `setProperty`, the `openfeature` language now gives the +`Boolean` value of a boolean flag. Before, it evaluated every flag as a string flag, so a boolean flag gave the +`defaultValue` of the endpoint as text (`false` unless set) and logged a `TYPE_MISMATCH` warning. String flags give +their text as before, and number and object flags still give the `defaultValue`. Predicates are unchanged. + +=== camel-datasonnet - a null result and a stream body + +An expression whose result is `null` now gives `null` instead of a `Document` without content, unless `resultType` is +`com.datasonnet.document.Document`. A predicate whose result is `null` is false; before, it failed with a +`NullPointerException`. + +A stream body without a body media type is now read as text. Before, it failed while the stream was mapped as a Java +object. + +=== camel-hl7 - the hl7terser language reads a stream or a file + +The `hl7terser` language now reads a stream or a file body as an HL7 message, in the charset of MSH-18 as the HL7 +data format does. A body that is not an HL7 message, such as a `Map`, or no body, now fails with an +`ExpressionEvaluationException` that says what the body is, instead of a `NullPointerException`; text that is not a +valid HL7 message fails with an `ExpressionEvaluationException` that has the parse error as its cause. + +=== camel-javascript, camel-python3 - bytes and functions in the result of a script + +A Python `bytes`, `bytearray` or `memoryview` returned by a `python3` script, and an `ArrayBuffer` returned by a `js` +script, now give a `byte[]` instead of a `List` of numbers. A script that returns a function now fails with an +`ExpressionEvaluationException`; before, it gave a value that failed when it was used later. + +=== camel-quickjs - a Set, a Map or a typed array in the result of a script + +A JavaScript `Set` returned by a `quickjs` script now gives a `List`, a JavaScript `Map` a `Map` with its keys as text, +and a typed array or an `ArrayBuffer` a `List` of numbers, also when nested. Before, a `Set` and a `Map` gave an empty +`Map`, and a typed array a `Map` of index to value.
