This is an automated email from the ASF dual-hosted git repository.
davsclaus pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/camel.git
The following commit(s) were added to refs/heads/main by this push:
new 3b3e0ae7bb26 CAMEL-25481: camel-dataweave - convert XML namespaces,
attributes and the DataSonnet XML model (#27612)
3b3e0ae7bb26 is described below
commit 3b3e0ae7bb2693ad92b7bf514205845f69fa7197
Author: Claus Ibsen <[email protected]>
AuthorDate: Fri Oct 9 21:52:18 2026 +0200
CAMEL-25481: camel-dataweave - convert XML namespaces, attributes and the
DataSonnet XML model (#27612)
* CAMEL-25481: camel-dataweave - convert XML namespaces, attributes and the
DataSonnet XML model
The DataWeave to DataSonnet conversion now follows the DataWeave XML
semantics.
XML input: a selector matches an element by its local name whatever its
namespace
prefix (such as payload.Envelope.Body of a SOAP message), .ns#name only in
the namespace
of an ns header directive, .@ gives all the attributes and ..* all the
descendants, an
empty element is null, and the object functions see the child elements in
document
order. An output other than XML writes the elements without the attributes,
namespace
declarations and positions of the DataSonnet XML model (dw.output).
XML output: attributes (key @(name: value)), namespace-qualified keys
(ns0#key),
null as an empty element, arrays, object spreads and keys that repeat as
elements that
repeat (in the order of the script), elements of the input with their
attributes and
namespaces, and the writeDeclaration and skipNullOn writer properties.
DataSonnet does not keep the value of a function argument (each use
evaluates it
again), so nested calls of the camel-dataweave.libsonnet functions took
exponential time
in the depth of the nesting. The functions now evaluate their arguments
once.
The corpus has 29 new XML entries, verified with the DataWeave CLI, and
compares XML
outputs without formatting.
Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
Signed-off-by: Claus Ibsen <[email protected]>
* CAMEL-25481: parenthesize mixed && / || conditions, rewrap the upgrade
guide note
Review feedback.
Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
Signed-off-by: Claus Ibsen <[email protected]>
* CAMEL-25481: camel-dataweave - fix CDATA next to text, namespaces with
the same local name, empty arrays and nested lambdas
Review fixes:
- The text of an element with text and CDATA segments was read twice (the
reader also
gives it in '$'); adjacent text and CDATA in mixed content are one __text.
- .*name and ..*name give the elements of every namespace with that local
name, in
document order (and .name the first in document order).
- An empty array is an empty element also in an object with a spread or a
repeated key.
- A lambda parameter kept as an XML element for its attributes is tracked
in its scope,
so an inner lambda with a parameter of the same name does not change it.
Five corpus entries for these, verified with the DataWeave CLI.
Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
Signed-off-by: Claus Ibsen <[email protected]>
---------
Signed-off-by: Claus Ibsen <[email protected]>
Co-authored-by: Claude Opus 5.5 (1M context) <[email protected]>
---
.../camel/catalog/docs/datasonnet-language.adoc | 31 +-
.../src/main/docs/datasonnet-language.adoc | 31 +-
.../src/main/resources/camel-dataweave.libsonnet | 549 ++++++++++++++-------
.../language/datasonnet/DataWeaveCorpusTest.java | 58 ++-
.../src/test/resources/dataweave-corpus/corpus.txt | 242 ++++++++-
.../camel/component/dataweave/DataWeaveAst.java | 37 +-
.../component/dataweave/DataWeaveConverter.java | 250 ++++++++--
.../camel/component/dataweave/DataWeaveLexer.java | 38 +-
.../camel/component/dataweave/DataWeaveParser.java | 94 ++--
.../dataweave/DataWeaveConverterTest.java | 113 ++++-
.../component/dataweave/DataWeaveParserTest.java | 34 +-
.../ROOT/pages/camel-4x-upgrade-guide-4_23.adoc | 10 +-
12 files changed, 1191 insertions(+), 296 deletions(-)
diff --git
a/catalog/camel-catalog/src/generated/resources/org/apache/camel/catalog/docs/datasonnet-language.adoc
b/catalog/camel-catalog/src/generated/resources/org/apache/camel/catalog/docs/datasonnet-language.adoc
index af3222598ad6..9a4894c4de70 100644
---
a/catalog/camel-catalog/src/generated/resources/org/apache/camel/catalog/docs/datasonnet-language.adoc
+++
b/catalog/camel-catalog/src/generated/resources/org/apache/camel/catalog/docs/datasonnet-language.adoc
@@ -542,10 +542,13 @@ The conversion either produces a script that gives the
same result as the DataWe
=== Supported Constructs
- *Header*: `%dw`, `output` (JSON, XML, CSV, Java and text; for JSON the
writer properties `skipNullOn`, `indent` and
- `encoding`), `input`, `import` of the `dw::core` modules, `var` and `fun`
declarations (also with default parameters,
- recursive and calling each other), and `type` declarations (ignored).
+ `encoding`, for XML `writeDeclaration`, `skipNullOn` (`elements`,
`attributes` or `everywhere`), `indent`,
+ `encoding` and `inlineCloseOn`), `input`, `ns` namespace declarations,
`import` of the `dw::core` modules, `var` and
+ `fun` declarations (also with default parameters, recursive and calling each
other), and `type` declarations
+ (ignored).
- *Selectors*: `.field`, `."quoted field"`, `[index]` (negative from the end),
`[from to to]`, `["key"]`, `.*field`,
- `..field`, `.@attribute`, `field?` and `[?(condition)]`.
+ `..field`, `..*field`, `.@attribute`, `.@` (all the attributes), `.ns#field`
(in the namespace declared as `ns`),
+ `field?` and `[?(condition)]`.
- *Mule variables and attributes*: `payload` is the message body, `vars.name`
(or `vars["name"]`) an exchange variable,
and `attributes.headers`, `attributes.queryParams` and
`attributes.uriParams` message headers. `attributes.method`,
`requestPath`, `requestUri`, `queryString`, `statusCode` and `reasonPhrase`
are the corresponding Camel HTTP headers.
@@ -564,6 +567,17 @@ The conversion either produces a script that gives the
same result as the DataWe
`partition`, `firstWith` and `mergeWith`.
- *Objects*: dynamic keys `(key): value`, conditional fields `(key: value) if
condition`, and object spread
`{ (expression) }` of an object or an array of objects.
+- *XML input*: as in DataWeave, a selector matches an element by its local
name whatever its namespace prefix (such as
+ `payload.Envelope.Body` of a SOAP message), and `.ns#field` only in the
given namespace. The value of an element
+ without child elements is its text (`null` when empty), `.field` of an
element that repeats is the first one and
+ `.*field` all of them, and the attributes are read with `.@name` and `.@`
(also of an element in a variable or
+ given to a lambda). The object functions (`keysOf`, `sizeOf`, `mapObject`,
`pluck`, `entriesOf`, ...) see the child
+ elements in document order, without the attributes. In an output other than
XML, an element is written as its text,
+ or as an object of its child elements, without its attributes and namespace
prefixes.
+- *XML output*: attributes `key @(name: value)`, namespace-qualified keys
`ns0#key` (declared on the root element),
+ `null` as an empty element, and arrays, object spreads and keys that repeat
as elements that repeat, in the order of
+ the script. An element of the input is written with its child elements,
attributes and namespaces; under another key
+ it gets the attributes of that key, as in DataWeave.
- *Strings*: interpolation (`"Hello $(expression)"`, `"Hello $name"` and `$`
of a lambda), and regular expressions
`/.../` in `replace` (also with a function of the matched groups: `replace
/(\d+)/ with ($[1] ++ "!")`), `contains`,
`splitBy`, `matches`, `scan` and `find`.
@@ -577,10 +591,13 @@ The conversion either produces a script that gives the
same result as the DataWe
The expected output of the DataWeave scripts in the tests of the conversion is
verified with the DataWeave CLI.
These constructs are not converted (the conversion fails): the `update`
operator, `try` (`dw::Runtime`), overloaded
-functions, custom modules, `as` with a custom type, XML namespaces, XML
attributes in the output, and writer properties
-other than the ones listed above. The XML support is limited: an element with
text is read as its text (as in
-DataWeave), but the attributes of such an element are only kept when they are
selected directly (`payload.order.@id`)
-or in a lambda (`payload.orders.*order map $.@id`).
+functions, custom modules, `as` with a custom type, the `..@` selector, a
namespace prefix that is not declared, and
+writer properties other than the ones listed above.
+
+A DataSonnet object can not repeat a key, so where DataWeave repeats a key in
a JSON or Java output, the converted
+script gives an array of the values: for elements that repeat under the same
name (in an XML input, such as
+`payload.order` with two `item` elements), and for the text of mixed content
(`__text`). An XML output does repeat the
+elements.
== Dependencies
diff --git a/components/camel-datasonnet/src/main/docs/datasonnet-language.adoc
b/components/camel-datasonnet/src/main/docs/datasonnet-language.adoc
index af3222598ad6..9a4894c4de70 100644
--- a/components/camel-datasonnet/src/main/docs/datasonnet-language.adoc
+++ b/components/camel-datasonnet/src/main/docs/datasonnet-language.adoc
@@ -542,10 +542,13 @@ The conversion either produces a script that gives the
same result as the DataWe
=== Supported Constructs
- *Header*: `%dw`, `output` (JSON, XML, CSV, Java and text; for JSON the
writer properties `skipNullOn`, `indent` and
- `encoding`), `input`, `import` of the `dw::core` modules, `var` and `fun`
declarations (also with default parameters,
- recursive and calling each other), and `type` declarations (ignored).
+ `encoding`, for XML `writeDeclaration`, `skipNullOn` (`elements`,
`attributes` or `everywhere`), `indent`,
+ `encoding` and `inlineCloseOn`), `input`, `ns` namespace declarations,
`import` of the `dw::core` modules, `var` and
+ `fun` declarations (also with default parameters, recursive and calling each
other), and `type` declarations
+ (ignored).
- *Selectors*: `.field`, `."quoted field"`, `[index]` (negative from the end),
`[from to to]`, `["key"]`, `.*field`,
- `..field`, `.@attribute`, `field?` and `[?(condition)]`.
+ `..field`, `..*field`, `.@attribute`, `.@` (all the attributes), `.ns#field`
(in the namespace declared as `ns`),
+ `field?` and `[?(condition)]`.
- *Mule variables and attributes*: `payload` is the message body, `vars.name`
(or `vars["name"]`) an exchange variable,
and `attributes.headers`, `attributes.queryParams` and
`attributes.uriParams` message headers. `attributes.method`,
`requestPath`, `requestUri`, `queryString`, `statusCode` and `reasonPhrase`
are the corresponding Camel HTTP headers.
@@ -564,6 +567,17 @@ The conversion either produces a script that gives the
same result as the DataWe
`partition`, `firstWith` and `mergeWith`.
- *Objects*: dynamic keys `(key): value`, conditional fields `(key: value) if
condition`, and object spread
`{ (expression) }` of an object or an array of objects.
+- *XML input*: as in DataWeave, a selector matches an element by its local
name whatever its namespace prefix (such as
+ `payload.Envelope.Body` of a SOAP message), and `.ns#field` only in the
given namespace. The value of an element
+ without child elements is its text (`null` when empty), `.field` of an
element that repeats is the first one and
+ `.*field` all of them, and the attributes are read with `.@name` and `.@`
(also of an element in a variable or
+ given to a lambda). The object functions (`keysOf`, `sizeOf`, `mapObject`,
`pluck`, `entriesOf`, ...) see the child
+ elements in document order, without the attributes. In an output other than
XML, an element is written as its text,
+ or as an object of its child elements, without its attributes and namespace
prefixes.
+- *XML output*: attributes `key @(name: value)`, namespace-qualified keys
`ns0#key` (declared on the root element),
+ `null` as an empty element, and arrays, object spreads and keys that repeat
as elements that repeat, in the order of
+ the script. An element of the input is written with its child elements,
attributes and namespaces; under another key
+ it gets the attributes of that key, as in DataWeave.
- *Strings*: interpolation (`"Hello $(expression)"`, `"Hello $name"` and `$`
of a lambda), and regular expressions
`/.../` in `replace` (also with a function of the matched groups: `replace
/(\d+)/ with ($[1] ++ "!")`), `contains`,
`splitBy`, `matches`, `scan` and `find`.
@@ -577,10 +591,13 @@ The conversion either produces a script that gives the
same result as the DataWe
The expected output of the DataWeave scripts in the tests of the conversion is
verified with the DataWeave CLI.
These constructs are not converted (the conversion fails): the `update`
operator, `try` (`dw::Runtime`), overloaded
-functions, custom modules, `as` with a custom type, XML namespaces, XML
attributes in the output, and writer properties
-other than the ones listed above. The XML support is limited: an element with
text is read as its text (as in
-DataWeave), but the attributes of such an element are only kept when they are
selected directly (`payload.order.@id`)
-or in a lambda (`payload.orders.*order map $.@id`).
+functions, custom modules, `as` with a custom type, the `..@` selector, a
namespace prefix that is not declared, and
+writer properties other than the ones listed above.
+
+A DataSonnet object can not repeat a key, so where DataWeave repeats a key in
a JSON or Java output, the converted
+script gives an array of the values: for elements that repeat under the same
name (in an XML input, such as
+`payload.order` with two `item` elements), and for the text of mixed content
(`__text`). An XML output does repeat the
+elements.
== Dependencies
diff --git
a/components/camel-datasonnet/src/main/resources/camel-dataweave.libsonnet
b/components/camel-datasonnet/src/main/resources/camel-dataweave.libsonnet
index 88c068bb7953..93d1d94518b8 100644
--- a/components/camel-datasonnet/src/main/resources/camel-dataweave.libsonnet
+++ b/components/camel-datasonnet/src/main/resources/camel-dataweave.libsonnet
@@ -20,55 +20,180 @@
// In DataWeave a missing field, an index out of range and a selector on null
give null, a field selector on an
// array selects the field of every element, and most functions accept null.
In Jsonnet these are errors.
// Functions that are given a function call it with all the arguments
DataWeave passes (such as item and index).
+//
+// DataSonnet evaluates the argument of a function each time the function uses
it, and an element of an array each
+// time it is used: only the value of an object field is kept. A function that
uses an argument more than once starts
+// with strict, else nested calls are evaluated over and over (exponentially
in the depth of the nesting).
{
local dw = self,
local indexes(a) = std.range(0, std.length(a) - 1),
+ // f(x), with x evaluated once (the initial value of std.foldl is evaluated
once, and kept)
+ local strict(x, f) = std.foldl(function(v, _) f(v), [0], x),
- // -- Selectors
-
- // An XML element with only text (and attributes) is read as an object with
the text in '$'
- local isText(x) = std.isObject(x) && std.objectHas(x, '$')
- && std.all([k == '$' || k == '~' || std.startsWith(k, '@')
for k in std.objectFields(x)]),
- local text(x) = if isText(x) then x['$'] else x,
+ // -- XML
+ //
+ // The DataSonnet XML reader gives an element as an object with its position
in '~', its attributes in '@name' keys,
+ // its namespace declarations in '@xmlns', its text in '$' (mixed content in
'$1', '$2', ..., CDATA also in '#1',
+ // ...), and its child elements by name (with the prefix of their namespace,
such as 'soapenv:Body'); an element
+ // repeated under the same name is an array. In DataWeave the value of an
element without child elements is its text
+ // (null when empty), a selector matches the local name of an element
whatever its namespace, and the attributes are
+ // metadata that only the .@ selector and an XML output see.
- // An XML element is read as an object with its position in '~'
local isElement(x) = std.isObject(x) && std.objectHas(x, '~'),
+ // the reader gives an array only for a repeated element, so the first
element tells
+ local isRepeated(v) = std.isArray(v) && std.length(v) > 0 && isElement(v[0]),
+ // an element, or a document (the object with the root element)
+ local isXml(x) =
+ isElement(x) || std.isObject(x) && (local fs = std.objectFields(x);
+ std.length(fs) == 1 &&
(isElement(x[fs[0]]) || isRepeated(x[fs[0]]))),
+ local isMeta(k) = k == '~' || std.startsWith(k, '@'),
+ local isTextKey(k) = std.startsWith(k, '$') || std.startsWith(k, '#'),
+ local isText(x) = isElement(x) && std.all([isMeta(k) || isTextKey(k) for k
in std.objectFields(x)]),
+ // the text segments of an element in document order: text ($1, $3, ...) and
CDATA (#2, ...)
+ local segments(x) = [k for k in std.objectFields(x) if std.length(k) > 1 &&
isTextKey(k)],
+ // the text of an element (the reader also gives all its text in '$' when it
has no child elements)
+ local textOf(x) = strict(if std.objectHas(x, '$') then x['$'] else
std.join('', [x[k] for k in segments(x)]),
+ function(t) if t == '' then null else t),
+ // (an element with child elements is marked as selected: written as XML
under another key, its attributes are left
+ // out, as they are those of its key in the input)
+ local text(x0) = strict(x0, function(x)
+ if isText(x) then textOf(x) else if isElement(x) then x + { '~selected'::
true } else x),
+ local localName(k) = local i = std.findSubstr(':', k); if std.length(i) == 0
then k else k[i[std.length(i) - 1] + 1:],
+ // the keys of the children k of an object: of an XML element those with the
local name k, whatever their namespace
+ // prefix (in the order of their first element)
+ local childKeys(x, k) =
+ if !isXml(x) then (if std.objectHas(x, k) then [k] else [])
+ else [f for f in std.objectFields(x) if !isMeta(f) && !isTextKey(f) &&
localName(f) == k],
+ local childKey(x, k) = strict(childKeys(x, k), function(m) if std.length(m)
== 0 then null else m[0]),
+ // the children of the keys (an element repeated under a key is an array),
in document order
+ local children(x, keys) =
+ strict(std.flattenArrays([strict(x[key], function(v) if std.isArray(v)
then v else [v]) for key in keys]),
+ function(vs) if std.length(keys) > 1 then std.sort(vs, function(e)
e['~']) else vs),
+ local first(v) = if isRepeated(v) then v[0] else v,
+ // a child element with the namespace declarations in scope of its parent,
so its namespace can be resolved (and it
+ // can be written as XML) without its ancestors
+ local inScope(x, v0) = strict(v0, function(v)
+ if !std.isObject(x) || !std.objectHas(x, '@xmlns') then v
+ else if isElement(v) then v { '@xmlns': x['@xmlns'] + (if std.objectHas(v,
'@xmlns') then v['@xmlns'] else {}) }
+ else if isRepeated(v) then [inScope(x, e) for e in v]
+ else v),
+ // the position of a child in its element: the order of child elements, or
the number of a text key ($2)
+ local position(k, v) =
+ if isElement(v) then v['~']
+ else if std.length(k) > 1 && isTextKey(k) then std.parseInt(k[1:])
+ else 0,
+ local attrsOf(x0) = strict(x0, function(x)
+ if !isElement(x) then {}
+ else { [k[1:]]: x[k] for k in std.objectFields(x) if std.startsWith(k,
'@') && k != '@xmlns' }),
+
+ // The value of an XML element as DataWeave sees it: its text, or its child
elements
+ text(x):: text(x),
+ // The entries of an object as DataWeave sees them: for an XML element
without its attributes and order, with the
+ // local names, an entry for each element repeated under a name, and the
text of the elements without children
+ local entry(k, e) = { k: localName(k), v: text(e), a: attrsOf(e), p:
position(k, e) },
+ local entries(o0) = strict(o0, function(o)
+ if !isXml(o) then [{ k: k, v: o[k], a: {} } for k in std.objectFields(o)]
+ else
+ // the child elements and the text segments (else the text in '$')
+ local texts = segments(o);
+ local keys = [k for k in std.objectFields(o)
+ if !isMeta(k) && (!isTextKey(k) || std.length(k) > 1 ||
std.length(texts) == 0)];
+ local es = std.flattenArrays([
+ if isTextKey(k) then [{ k: '__text', v: o[k], a: {}, p: position(k,
o[k]) }]
+ else strict(inScope(o, o[k]), function(v) if isRepeated(v) then
[entry(k, e) for e in v] else [entry(k, v)])
+ for k in keys
+ ]);
+ // in document order, which the keys are unless an element repeats with
others in between, or text is mixed in
+ strict(if std.length(keys) > 1 && std.any([std.isArray(o[k]) ||
isTextKey(k) for k in keys])
+ then std.sort(es, function(e) e.p)
+ else es, mergeText)),
+ // adjacent text segments (text and CDATA) are one text
+ local mergeText(es) =
+ if !std.any([e.k == '__text' for e in es]) then es
+ else std.foldl(function(acc, e)
+ local n = std.length(acc);
+ if e.k == '__text' && n > 0 && acc[n - 1].k == '__text'
+ then acc[:n - 1] + [acc[n - 1] { v: acc[n - 1].v + e.v }]
+ else acc + [e],
+ es, []),
+ // An object of entries; the values of a key that repeats become an array
(an object can not repeat a key)
+ local fromEntries(es0) = strict(es0, function(es)
+ local keys = std.foldl(function(acc, e) if std.member(acc, e.k) then acc
else acc + [e.k], es, []);
+ { [k]: strict([e.v for e in es if e.k == k], function(vs) if
std.length(vs) == 1 then vs[0] else vs) for k in keys }),
+ entries(o):: entries(o),
+ // The DataWeave value of something read from XML, for writing it in another
format than XML: elements as their text
+ // or as objects of their child elements, without attributes, order and
namespace prefixes
+ fromXml(x0):: strict(x0, function(x)
+ if isText(x) then textOf(x)
+ else if std.isArray(x) then std.map(dw.fromXml, x)
+ else if isXml(x) then fromEntries([{ k: e.k, v: dw.fromXml(e.v) } for e in
entries(x)])
+ else if std.isObject(x) then { [k]: dw.fromXml(x[k]) for k in
std.objectFields(x) }
+ else x),
+ // The result of a script written in another format than XML: when the input
is XML, as DataWeave would write it
+ output(result, input):: if isXml(input) then dw.fromXml(result) else result,
+
+ // -- Selectors
// x.k: null when missing; on an array the field of every element that has
it; of an XML element repeated under
- // the same name (read as an array) the first
- selRaw(x, k)::
- if std.isObject(x) then (
- if !std.objectHas(x, k) then null
- else
- local v = x[k];
- if std.isArray(v) && std.length(v) > 0 && std.all(std.map(isElement,
v)) then v[0] else v
- )
- else if std.isArray(x) then [e[k] for e in x if std.isObject(e) &&
std.objectHas(e, k)]
- else null,
- sel(x, k)::
- local v = dw.selRaw(x, k);
- if std.isArray(x) then std.map(text, v) else text(v),
+ // the same name the first; selRaw keeps an XML element as an object (for
its attributes)
+ selRaw(x0, k):: strict(x0, function(x)
+ if std.isObject(x) then strict(childKey(x, k), function(key) if key ==
null then null else inScope(x, first(x[key])))
+ else if std.isArray(x) then std.flattenArrays([
+ strict(e, function(e)
+ if !std.isObject(e) then []
+ else strict(childKey(e, k), function(key)
+ if key == null then [] else if isRepeated(e[key]) then inScope(e,
e[key]) else [inScope(e, e[key])]))
+ for e in x
+ ])
+ else null),
+ sel(x0, k):: strict(x0, function(x)
+ strict(dw.selRaw(x, k), function(v) if std.isArray(x) then std.map(text,
v) else text(v))),
+ // x.ns#k: the child k in the namespace with the given URI; a child whose
prefix is not declared in scope (as when
+ // it was not selected from its parent) is taken to be in that namespace
+ selNsRaw(x0, uri, k):: strict(x0, function(x)
+ local ns = if std.isObject(x) && std.objectHas(x, '@xmlns') then
x['@xmlns'] else {};
+ local inNamespace(f) =
+ local i = std.findSubstr(':', f);
+ local prefix = if std.length(i) == 0 then '$' else f[:i[0]];
+ strict(first(x[f]), function(v)
+ strict(ns + (if isElement(v) && std.objectHas(v, '@xmlns') then
v['@xmlns'] else {}), function(declarations)
+ !std.objectHas(declarations, prefix) || declarations[prefix] ==
uri));
+ if !isXml(x) then dw.selRaw(x, k)
+ else strict([f for f in std.objectFields(x) if !isMeta(f) && !isTextKey(f)
&& localName(f) == k && inNamespace(f)],
+ function(m) if std.length(m) == 0 then null else inScope(x,
first(x[m[0]])))),
+ selNs(x, uri, k):: text(dw.selNsRaw(x, uri, k)),
// x.a.b.c
pathRaw(x, ks):: std.foldl(function(v, k) dw.selRaw(v, k), ks, x),
path(x, ks):: std.foldl(function(v, k) dw.sel(v, k), ks, x),
// x.@a
- attr(x, a):: dw.selRaw(x, '@' + a),
+ attr(x0, a):: strict(x0, function(x)
+ if std.isArray(x) then [dw.attr(e, a) for e in x]
+ else if std.isObject(x) && std.objectHas(x, '@' + a) then x['@' + a]
+ else null),
// x.*k: all the values of k (an XML element repeated under the same name is
read as an array)
- multiRaw(x, k)::
- if std.isObject(x) then (if !std.objectHas(x, k) then null else if
std.isArray(x[k]) then x[k] else [x[k]])
- else if std.isArray(x) then std.flattenArrays([dw.multiRaw(e, k) for e in
x if dw.multiRaw(e, k) != null])
- else null,
- multi(x, k):: local v = dw.multiRaw(x, k); if v == null then null else
std.map(text, v),
- // x..k: the values of k at any depth
- desc(x, k)::
+ multiRaw(x0, k):: strict(x0, function(x)
+ if std.isObject(x) then strict(childKeys(x, k), function(keys)
+ if std.length(keys) == 0 then null else [inScope(x, e) for e in
children(x, keys)])
+ else if std.isArray(x) then
+ std.flattenArrays([strict(dw.multiRaw(e, k), function(v) if v == null
then [] else v) for e in x])
+ else null),
+ multi(x, k):: strict(dw.multiRaw(x, k), function(v) if v == null then null
else std.map(text, v)),
+ // x..k: the values of k at any depth (of an XML element repeated under the
same name the first), x..*k all of them
+ local descendants(x0, k, all) = strict(x0, function(x)
local walk(v) =
if std.isObject(v) then
- (if std.objectHas(v, k) then [text(v[k])] else []) +
std.flattenArrays([walk(v[f]) for f in std.objectFields(v)])
+ strict(childKeys(v, k), function(keys)
+ if std.length(keys) == 0 then []
+ else if all && isXml(v) then std.map(text, children(v, keys))
+ else [text(first(v[keys[0]]))])
+ + std.flattenArrays([walk(v[f]) for f in std.objectFields(v) if
!isMeta(f)])
else if std.isArray(v) then std.flattenArrays([walk(e) for e in v])
else [];
- if x == null then null else walk(x),
+ if x == null then null else walk(x)),
+ desc(x, k):: descendants(x, k, false),
+ descAll(x, k):: descendants(x, k, true),
// x[i]: from the end when negative, null when out of range; x["k"] is x.k
- idx(x, i)::
+ idx(x0, i):: strict(x0, function(x)
if x == null || i == null then null
else if std.isString(i) then dw.sel(x, i)
else if std.isArray(x) || std.isString(x) then (
@@ -81,9 +206,9 @@
local j = if i < 0 then std.length(ks) + i else i;
if j < 0 || j >= std.length(ks) then null else x[ks[j]]
)
- else null,
+ else null),
// x[a to b]: inclusive, from the end when negative, reversed when a > b
- slice(x, a, b)::
+ slice(x0, a, b):: strict(x0, function(x)
if x == null then null
else
local n = std.length(x);
@@ -94,210 +219,304 @@
local part = if std.isString(x) then std.substr(x, lo, hi - lo + 1) else
x[lo:hi + 1];
if from <= to then part
else if std.isString(x) then std.join('',
std.reverse(std.stringChars(part)))
- else std.reverse(part),
+ else std.reverse(part)),
// a to b
range(a, b):: if a <= b then std.range(a, b) else std.reverse(std.range(b,
a)),
// x.k?
- has(x, k):: std.isObject(x) && std.objectHas(x, k),
+ has(x0, k):: strict(x0, function(x) std.isObject(x) && (std.objectHas(x, k)
|| childKey(x, k) != null)),
// -- Values
- default(v, fallback):: if v == null then fallback else v,
+ default(v0, fallback):: strict(v0, function(v) if v == null then fallback
else v),
// a value in a string (interpolation, dynamic key)
- str(v):: if std.isString(v) then v else std.toString(v),
- toString(v):: if v == null || std.isString(v) then v else std.toString(v),
- nullSafe(f, x):: if x == null then null else f(x),
+ str(v0):: strict(v0, function(v) if std.isString(v) then v else
std.toString(v)),
+ toString(v0):: strict(v0, function(v) if v == null || std.isString(v) then v
else std.toString(v)),
+ nullSafe(f, x0):: strict(x0, function(x) if x == null then null else f(x)),
// a function given as a value, called with up to three arguments
- fn(f)::
+ fn(f0):: strict(f0, function(f)
local n = std.length(f);
if n == 0 then function(a=null, b=null, c=null) f()
else if n == 1 then function(a, b=null, c=null) f(a)
else if n == 2 then function(a, b=null, c=null) f(a, b)
- else f,
- typeOf(x)::
+ else f),
+ typeOf(x0):: strict(x0, function(x)
if x == null then 'Null'
else if std.isString(x) then 'String'
else if std.isNumber(x) then 'Number'
else if std.isBoolean(x) then 'Boolean'
else if std.isArray(x) then 'Array'
else if std.isObject(x) then 'Object'
- else 'Function',
+ else 'Function'),
+ // key @(attributes): value, written as XML (a null attribute as "null", or
left out with skipNull); the attributes
+ // of an element of the input are those of its key in the input, so they are
replaced
+ withAttributes(v0, attrs0, skipNull=false):: strict(v0, function(v)
strict(attrs0, function(attrs)
+ local a = { ['@' + k]: dw.str(attrs[k]) for k in std.objectFields(attrs)
if !(skipNull && attrs[k] == null) };
+ if std.isArray(v) then [dw.withAttributes(e, attrs, skipNull) for e in v]
+ else if isElement(v) then withoutAttributes(v) + a + { '~keyAttributes'::
true }
+ else if std.isObject(v) then v + a
+ else if v == null then a
+ else a { '$': dw.str(v) })),
+ // The DataSonnet XML writer writes the child elements in the order of their
position in '~' (else of their field):
+ // the elements of an object made by the script are given their position in
it, as an element of the input has its
+ // position in the input (the output of the script is not read anymore, so a
value is written as an element)
+ // an element of the input without its attributes (but with its namespace
declarations), as the value of a key
+ local withoutAttributes(v) =
+ std.foldl(function(acc, k) acc + { [k]:: null }, [k for k in
std.objectFields(v) if std.startsWith(k, '@') && k != '@xmlns'], v),
+ local positioned(v, i) =
+ if std.isObject(v) then v { '~': i }
+ else if v == null then { '~': i }
+ else { '$': dw.str(v), '~': i },
+ local ordered(x0) = strict(x0, function(x)
+ if std.isArray(x) then std.map(ordered, x)
+ else if !std.isObject(x) then x
+ // an element of the input selected under a key of the script: written
with the attributes of that key (none or
+ // those of withAttributes)
+ else if isElement(x) then
+ (if std.objectHasAll(x, '~selected') && !std.objectHasAll(x,
'~keyAttributes') then withoutAttributes(x) else x)
+ else
+ local fields = [k for k in std.objectFields(x) if !isMeta(k) &&
!isTextKey(k)];
+ // an empty array is written as an empty element
+ local count(k) = if std.isArray(x[k]) then std.max(1, std.length(x[k]))
else 1;
+ local fieldOrder = std.flattenArrays([[k for _ in std.range(1,
count(k))] for k in fields]);
+ // the keys of the elements in the order of the object, or as kept by
xmlObject
+ strict(if std.objectHasAll(x, '~order') && std.length(x['~order']) ==
std.length(fieldOrder)
+ && std.all([std.length([o for o in x['~order'] if o == k]) ==
count(k) for k in fields])
+ then x['~order'] else fieldOrder, function(order)
+ { [k]: x[k] for k in std.objectFields(x) if isMeta(k) || isTextKey(k) }
+ + { [k]: strict([i for i in indexes(order) if order[i] == k],
function(ps)
+ if std.isArray(x[k]) && std.length(x[k]) > 0 then
[positioned(ordered(x[k][j]), ps[j]) for j in indexes(x[k])]
+ else positioned(if std.isArray(x[k]) then null else
ordered(x[k]), ps[0]))
+ for k in fields })),
+ // The result of a script written as XML: an element of the input (such as
payload.Envelope.Body) is written as its
+ // content; the namespaces of the header, and those of the input in scope,
are declared on the root element
+ xmlOutput(doc0, ns):: strict(doc0, function(doc)
+ local inherited = if isElement(doc) && std.objectHas(doc, '@xmlns') then
doc['@xmlns'] else {};
+ strict(if isElement(doc) then { [k]: doc[k] for k in std.objectFields(doc)
if !isMeta(k) && !isTextKey(k) } else doc,
+ function(content)
+ if !std.isObject(content) ||
std.length(std.objectFields(content)) != 1 then content
+ else
+ local root = std.objectFields(content)[0];
+ strict(ordered(content[root]), function(v)
+ if inherited + ns == {} then { [root]: v }
+ else
+ local own = if std.isObject(v) && std.objectHas(v,
'@xmlns') then v['@xmlns'] else {};
+ { [root]: (if std.isObject(v) then v else if v == null then
{} else { '$': dw.str(v) })
+ + { '@xmlns': inherited + ns + own } }))),
+ // x.@: the attributes of an XML element
+ attrs(x0):: strict(x0, function(x) if std.isArray(x) then [attrsOf(e) for e
in x] else attrsOf(first(x))),
// a ~= b
- similar(a, b):: a == b || (a != null && b != null && dw.str(a) == dw.str(b)),
+ similar(a0, b0):: strict(a0, function(a) strict(b0, function(b)
+ a == b || (a != null && b != null && dw.str(a) == dw.str(b)))),
// an object spread { (x) }: an object, or an array of objects
- toObject(x)::
+ toObject(x0):: strict(x0, function(x)
if x == null then {}
else if std.isArray(x) then std.foldl(function(acc, o) acc +
dw.toObject(o), x, {})
- else x,
+ else if isXml(x) then fromEntries(entries(x))
+ else x),
+ // An object written as XML, of its parts (objects of its fields, and object
spreads): a key that repeats is an
+ // element that repeats (DataWeave keeps every field of a key that repeats,
where Jsonnet keeps the last)
+ local fieldsOf(x0) = strict(x0, function(x)
+ if x == null then []
+ else if std.isArray(x) then std.flattenArrays([fieldsOf(e) for e in x])
+ else if isElement(x) then [{ k: k, v: x[k] } for k in std.objectFields(x)
if !isMeta(k) && !isTextKey(k)]
+ else [{ k: k, v: x[k] } for k in std.objectFields(x)]),
+ // (the items of an array are elements that repeat); the order of the
elements is kept in the hidden field '~order'
+ xmlObject(parts):: strict(std.flattenArrays([fieldsOf(p) for p in parts]),
function(fs)
+ strict(std.flattenArrays([strict(f.v, function(v)
+ if !std.isArray(v) then [{ k: f.k, v: v }]
+ // an empty array is an empty element
+ else if std.length(v) == 0 then [{ k: f.k, v:
null }]
+ else [{ k: f.k, v: e } for e in v])
+ for f in fs]), function(es)
+ local keys = std.foldl(function(acc, e) if std.member(acc, e.k) then acc
else acc + [e.k], es, []);
+ { [k]: strict([e.v for e in es if e.k == k], function(vs) if
std.length(vs) == 1 then vs[0] else vs) for k in keys }
+ + { '~order':: [e.k for e in es] })),
// obj - "key", array - element
- minus(a, b)::
- if std.isObject(a) then std.foldl(function(acc, k) if k == b then acc else
acc { [k]: a[k] }, std.objectFields(a), {})
+ minus(a0, b):: strict(a0, function(a)
+ if std.isObject(a) then fromEntries([e for e in entries(a) if e.k != b])
else if std.isArray(a) then [e for e in a if e != b]
- else a - b,
+ else a - b),
// obj -- ["k1", "k2"] (or an object with the keys), array -- elements
- removeAll(a, b)::
- local keys = if std.isObject(b) then std.objectFields(b) else b;
- if a == null then null
- else if std.isObject(a) then std.foldl(function(acc, k) if
std.member(keys, k) then acc else acc { [k]: a[k] },
- std.objectFields(a), {})
- else [e for e in a if !std.member(keys, e)],
- // the skipNullOn writer property: everywhere, objects or arrays
- skipNulls(v, where)::
- local objects = where == 'everywhere' || where == 'objects';
+ removeAll(a0, b0):: strict(a0, function(a) strict(b0, function(b)
+ strict(if std.isObject(b) then [e.k for e in entries(b)] else b,
function(keys)
+ if a == null then null
+ else if std.isObject(a) then fromEntries([e for e in entries(a) if
!std.member(keys, e.k)])
+ else [e for e in a if !std.member(keys, e)]))),
+ // the skipNullOn writer property: everywhere, objects or arrays (JSON),
elements (XML; attributes are left out
+ // by withAttributes)
+ skipNulls(v0, where):: strict(v0, function(v)
+ local objects = where == 'everywhere' || where == 'objects' || where ==
'elements';
local arrays = where == 'everywhere' || where == 'arrays';
if std.isObject(v) then
- std.foldl(function(acc, k) if objects && v[k] == null then acc else acc
{ [k]: dw.skipNulls(v[k], where) },
- std.objectFields(v), {})
- else if std.isArray(v) then [dw.skipNulls(e, where) for e in v if !(arrays
&& e == null)]
- else v,
+ { [k]: dw.skipNulls(v[k], where) for k in std.objectFields(v) if
!(objects && v[k] == null) }
+ + (if std.objectHasAll(v, '~order') then { '~order':: [k for k in
v['~order'] if !(objects && v[k] == null)] }
+ else {})
+ else if std.isArray(v) then
+ std.flattenArrays([strict(e, function(e) if arrays && e == null then []
else [dw.skipNulls(e, where)]) for e in v])
+ else v),
// -- Strings
- upper(s):: if s == null then null else std.asciiUpper(s),
- lower(s):: if s == null then null else std.asciiLower(s),
- trim(s):: if s == null then null else std.stripChars(s, ' \t\n\r'),
- isBlank(s):: s == null || std.length(std.stripChars(s, ' \t\n\r')) == 0,
- contains(a, b)::
+ upper(s0):: strict(s0, function(s) if s == null then null else
std.asciiUpper(s)),
+ lower(s0):: strict(s0, function(s) if s == null then null else
std.asciiLower(s)),
+ trim(s0):: strict(s0, function(s) if s == null then null else
std.stripChars(s, ' \t\n\r')),
+ isBlank(s0):: strict(s0, function(s) s == null ||
std.length(std.stripChars(s, ' \t\n\r')) == 0),
+ contains(a0, b):: strict(a0, function(a)
if a == null then false
else if std.isArray(a) then std.member(a, b)
- else std.length(std.findSubstr(b, a)) > 0,
+ else std.length(std.findSubstr(b, a)) > 0),
// contains /regex/, with scan the ds.scan function
- containsMatch(s, scan, regex):: s != null && std.length(scan(s, regex)) > 0,
+ containsMatch(s0, scan, regex):: strict(s0, function(s) s != null &&
std.length(scan(s, regex)) > 0),
// splitBy /regex/, with split the ds.splitBy function
- splitByMatch(s, split, regex):: if s == null then null else split(s, regex),
- startsWith(s, prefix):: s != null && std.startsWith(s, prefix),
- endsWith(s, suffix):: s != null && std.endsWith(s, suffix),
- splitBy(s, separator)::
+ splitByMatch(s0, split, regex):: strict(s0, function(s) if s == null then
null else split(s, regex)),
+ startsWith(s0, prefix):: strict(s0, function(s) s != null &&
std.startsWith(s, prefix)),
+ endsWith(s0, suffix):: strict(s0, function(s) s != null && std.endsWith(s,
suffix)),
+ splitBy(s0, separator):: strict(s0, function(s)
if s == null then null
else if separator == '' then std.stringChars(s)
- else std.split(s, separator),
- joinBy(a, separator):: if a == null then null else std.join(separator,
[dw.str(e) for e in a]),
- replace(s, target, replacement):: if s == null then null else
std.strReplace(s, target, replacement),
+ else std.split(s, separator)),
+ joinBy(a0, separator):: strict(a0, function(a)
+ if a == null then null else std.join(separator, [dw.str(e) for e in a])),
+ replace(s0, target, replacement):: strict(s0, function(s)
+ if s == null then null else std.strReplace(s, target, replacement)),
// replace /regex/ with a function of the match (the matched text and its
groups), with starts and matches as given
// by ds.find and ds.scan
- replaceMatches(s, starts, matches, f)::
- if s == null then null
- else
- local r = std.foldl(function(acc, i) {
- out: acc.out + std.substr(s, acc.pos, starts[i] - acc.pos) +
dw.str(f(matches[i])),
- pos: starts[i] + std.length(matches[i][0]),
- }, indexes(matches), { out: '', pos: 0 });
- r.out + std.substr(s, r.pos, std.length(s) - r.pos),
+ replaceMatches(s0, starts0, matches0, f):: strict(s0, function(s)
strict(starts0, function(starts)
+ strict(matches0, function(matches)
+ if s == null then null
+ else
+ strict(std.foldl(function(acc, i) {
+ out: acc.out + std.substr(s, acc.pos, starts[i] - acc.pos) +
dw.str(f(matches[i])),
+ pos: starts[i] + std.length(matches[i][0]),
+ }, indexes(matches), { out: '', pos: 0 }), function(r) r.out +
std.substr(s, r.pos, std.length(s) - r.pos))))),
// -- Arrays and objects
+ //
+ // A function given to them is called with the item evaluated once (an item
of an array computed by another
+ // function is evaluated each time it is used).
- sizeOf(x):: if x == null then null else std.length(x),
- isEmpty(x):: x == null || std.length(x) == 0,
- map(a, f):: if a == null then null else std.mapWithIndex(function(i, x) f(x,
i), a),
- filter(a, f)::
+ sizeOf(x0):: strict(x0, function(x)
+ if x == null then null else if isXml(x) then std.length(entries(x)) else
std.length(text(x))),
+ isEmpty(x0):: strict(x0, function(x)
+ x == null || strict(text(x), function(t) t == null || std.length(if
isXml(x) then entries(x) else t) == 0)),
+ map(a0, f):: strict(a0, function(a)
+ if a == null then null else std.mapWithIndex(function(i, x) strict(x,
function(x) f(x, i)), a)),
+ filter(a0, f):: strict(a0, function(a)
if a == null then null
else if std.isString(a) then (
local cs = std.stringChars(a);
std.join('', [cs[i] for i in indexes(cs) if f(cs[i], i)])
)
else if std.isObject(a) then dw.filterObject(a, function(v, k, i) f(v, k))
- else [a[i] for i in indexes(a) if f(a[i], i)],
- flatMap(a, f):: if a == null then null else std.flattenArrays(dw.map(a, f)),
+ else std.flattenArrays([strict(a[i], function(x) if f(x, i) then [x] else
[]) for i in indexes(a)])),
+ flatMap(a, f):: strict(dw.map(a, f), function(m) if m == null then null else
std.flattenArrays(m)),
// reduce with an initial accumulator, and without (starting with the first
item)
- reduce(a, f, init):: if a == null then null else std.foldl(function(acc, x)
f(x, acc), a, init),
- reduce1(a, f)::
+ reduce(a0, f, init):: strict(a0, function(a)
+ if a == null then null else std.foldl(function(acc, x) strict(x,
function(x) f(x, acc)), a, init)),
+ reduce1(a0, f):: strict(a0, function(a)
if a == null || std.length(a) == 0 then null
- else std.foldl(function(acc, x) f(x, acc), a[1:], a[0]),
- distinctBy(a, f)::
+ else std.foldl(function(acc, x) strict(x, function(x) f(x, acc)), a[1:],
a[0])),
+ distinctBy(a0, f):: strict(a0, function(a)
if a == null then null
else if std.isObject(a) then
- std.foldl(function(acc, k)
- local g = f(a[k], k);
- if std.member(acc.keys, g) then acc else { keys: acc.keys +
[g], obj: acc.obj { [k]: a[k] } },
- std.objectFields(a), { keys: [], obj: {} }).obj
+ fromEntries(std.foldl(function(acc, e)
+ strict(f(e.v, e.k), function(g)
+ if std.member(acc.keys, g) then acc else {
keys: acc.keys + [g], es: acc.es + [e] }),
+ entries(a), { keys: [], es: [] }).es)
else
std.foldl(function(acc, i)
- local g = f(a[i], i);
- if std.member(acc.keys, g) then acc else { keys: acc.keys +
[g], items: acc.items + [a[i]] },
- indexes(a), { keys: [], items: [] }).items,
- groupBy(a, f)::
+ strict(a[i], function(x) strict(f(x, i), function(g)
+ if std.member(acc.keys, g) then acc else { keys: acc.keys
+ [g], items: acc.items + [x] })),
+ indexes(a), { keys: [], items: [] }).items),
+ groupBy(a0, f):: strict(a0, function(a)
if a == null then null
else if std.isObject(a) then
- std.foldl(function(acc, k)
- local g = dw.str(f(a[k], k));
- acc { [g]: (if std.objectHas(acc, g) then acc[g] else {}) +
{ [k]: a[k] } },
- std.objectFields(a), {})
+ strict([e { g: dw.str(f(e.v, e.k)) } for e in entries(a)], function(es)
+ { [g]: fromEntries([e for e in es if e.g == g])
+ for g in std.foldl(function(acc, e) if std.member(acc, e.g) then acc
else acc + [e.g], es, []) })
else
std.foldl(function(acc, i)
- local g = dw.str(f(a[i], i));
- acc { [g]: (if std.objectHas(acc, g) then acc[g] else []) +
[a[i]] },
- indexes(a), {}),
- // a stable sort by key (std.sort of DataSonnet ignores keyF for an array of
numbers or strings)
- local sortBy(xs, key) =
- local keyed = [{ k: key(x), x: x } for x in xs];
- std.flattenArrays([[e.x for e in keyed if e.k == k] for k in std.set([e.k
for e in keyed])]),
- orderBy(a, f)::
+ strict(a[i], function(x) strict(dw.str(f(x, i)), function(g)
+ acc { [g]: (if std.objectHas(acc, g) then acc[g] else [])
+ [x] })),
+ indexes(a), {})),
+ // a stable sort by key, with the key of each item computed once (std.sort
of DataSonnet ignores keyF for an array
+ // of numbers or strings, so it sorts objects with the indexes)
+ local sortBy(xs0, key) = strict(xs0, function(xs) {
+ ks: { [std.toString(i)]: key(xs[i], i) for i in indexes(xs) },
+ r: [xs[e.i] for e in std.sort([{ i: i } for i in indexes(xs)], function(e)
self.ks[std.toString(e.i)])],
+ }.r),
+ orderBy(a0, f):: strict(a0, function(a)
if a == null then null
- else if std.isObject(a) then
- dw.toObject(sortBy([{ [k]: a[k] } for k in std.objectFields(a)],
- function(e) local k = std.objectFields(e)[0]; f(e[k],
k)))
- else std.map(function(e) e.x, sortBy([{ i: i, x: a[i] } for i in
indexes(a)], function(e) f(e.x, e.i))),
- mapObject(o, f)::
+ else if std.isObject(a) then fromEntries(sortBy(entries(a), function(e, i)
f(e.v, e.k)))
+ else sortBy(a, function(x, i) strict(x, function(x) f(x, i)))),
+ mapObject(o0, f):: strict(o0, function(o)
if o == null then null
- else
- local ks = std.objectFields(o);
- std.foldl(function(acc, i) acc + f(o[ks[i]], ks[i], i), indexes(ks), {}),
- filterObject(o, f)::
+ else strict(entries(o), function(es)
+ fromEntries(std.flattenArrays([strict(es[i], function(e) entries(f(e.v,
e.k, i))) for i in indexes(es)])))),
+ filterObject(o0, f):: strict(o0, function(o)
if o == null then null
- else
- local ks = std.objectFields(o);
- std.foldl(function(acc, i) if f(o[ks[i]], ks[i], i) then acc { [ks[i]]:
o[ks[i]] } else acc, indexes(ks), {}),
- pluck(o, f)::
+ else strict(entries(o), function(es)
+ fromEntries(std.flattenArrays([strict(es[i], function(e) if f(e.v, e.k,
i) then [e] else [])
+ for i in indexes(es)])))),
+ pluck(o0, f):: strict(o0, function(o)
if o == null then null
- else
- local ks = std.objectFields(o);
- [f(o[ks[i]], ks[i], i) for i in indexes(ks)],
- keysOf(o):: if o == null then null else std.objectFields(o),
+ else strict(entries(o), function(es) [strict(es[i], function(e) f(e.v,
e.k, i)) for i in indexes(es)])),
+ keysOf(o0):: strict(o0, function(o) if o == null then null else [e.k for e
in entries(o)]),
namesOf(o):: dw.keysOf(o),
- valuesOf(o):: if o == null then null else [o[k] for k in
std.objectFields(o)],
- entriesOf(o):: if o == null then null else [{ key: k, value: o[k],
attributes: {} } for k in std.objectFields(o)],
- flatten(a):: if a == null then null else std.flattenArrays([if
std.isArray(e) then e else [e] for e in a]),
- zip(a, b)::
+ valuesOf(o0):: strict(o0, function(o) if o == null then null else [e.v for e
in entries(o)]),
+ entriesOf(o0):: strict(o0, function(o)
+ if o == null then null else [{ key: e.k, value: e.v, attributes: e.a } for
e in entries(o)]),
+ flatten(a0):: strict(a0, function(a)
+ if a == null then null else std.flattenArrays([strict(e, function(e) if
std.isArray(e) then e else [e]) for e in a])),
+ zip(a0, b0):: strict(a0, function(a) strict(b0, function(b)
if a == null || b == null then null
- else [[a[i], b[i]] for i in std.range(0, std.min(std.length(a),
std.length(b)) - 1)],
+ else [[a[i], b[i]] for i in std.range(0, std.min(std.length(a),
std.length(b)) - 1)])),
// x then f
- pipe(v, f):: f(v),
- sum(a):: if a == null then null else std.foldl(function(acc, x) acc + x, a,
0),
- avg(a):: if a == null then null else dw.sum(a) / std.length(a),
- min(a):: if a == null || std.length(a) == 0 then null else
std.foldl(function(m, x) if x < m then x else m, a[1:], a[0]),
- max(a):: if a == null || std.length(a) == 0 then null else
std.foldl(function(m, x) if x > m then x else m, a[1:], a[0]),
- maxBy(a, f)::
+ pipe(v, f):: strict(v, f),
+ sum(a0):: strict(a0, function(a) if a == null then null else
std.foldl(function(acc, x) acc + x, a, 0)),
+ avg(a0):: strict(a0, function(a) if a == null then null else dw.sum(a) /
std.length(a)),
+ min(a0):: strict(a0, function(a)
+ if a == null || std.length(a) == 0 then null
+ else std.foldl(function(m, x) strict(x, function(x) if x < m then x else
m), a[1:], a[0])),
+ max(a0):: strict(a0, function(a)
if a == null || std.length(a) == 0 then null
- else std.foldl(function(m, x) if f(x) > f(m) then x else m, a[1:], a[0]),
- minBy(a, f)::
+ else std.foldl(function(m, x) strict(x, function(x) if x > m then x else
m), a[1:], a[0])),
+ maxBy(a0, f):: strict(a0, function(a)
if a == null || std.length(a) == 0 then null
- else std.foldl(function(m, x) if f(x) < f(m) then x else m, a[1:], a[0]),
- sumBy(a, f):: if a == null then null else std.foldl(function(acc, x) acc +
f(x), a, 0),
- countBy(a, f):: if a == null then null else std.length([x for x in a if
f(x)]),
- every(a, f):: a == null || std.all([f(x) for x in a]),
- some(a, f):: a != null && std.any([f(x) for x in a]),
- firstWith(a, f):: if a == null then null else (local m = [x for x in a if
f(x)]; if std.length(m) == 0 then null else m[0]),
- partition(a, f):: if a == null then null else { success: [x for x in a if
f(x)], failure: [x for x in a if !f(x)] },
- indexOf(a, x)::
+ else std.foldl(function(m, x) strict(x, function(x) if f(x) > f(m) then x
else m), a[1:], a[0])),
+ minBy(a0, f):: strict(a0, function(a)
+ if a == null || std.length(a) == 0 then null
+ else std.foldl(function(m, x) strict(x, function(x) if f(x) < f(m) then x
else m), a[1:], a[0])),
+ sumBy(a0, f):: strict(a0, function(a) if a == null then null else
std.foldl(function(acc, x) acc + f(x), a, 0)),
+ countBy(a0, f):: strict(a0, function(a) if a == null then null else
std.length([x for x in a if f(x)])),
+ every(a0, f):: strict(a0, function(a) a == null || std.all([f(x) for x in
a])),
+ some(a0, f):: strict(a0, function(a) a != null && std.any([f(x) for x in
a])),
+ firstWith(a0, f):: strict(a0, function(a)
+ if a == null then null
+ else strict([x for x in a if f(x)], function(m) if std.length(m) == 0 then
null else m[0])),
+ partition(a0, f):: strict(a0, function(a)
+ if a == null then null else { success: [x for x in a if f(x)], failure: [x
for x in a if !f(x)] }),
+ indexOf(a0, x):: strict(a0, function(a)
if a == null then -1
- else if std.isString(a) then (local found = std.findSubstr(x, a); if
std.length(found) == 0 then -1 else found[0])
- else (local found = [i for i in indexes(a) if a[i] == x]; if
std.length(found) == 0 then -1 else found[0]),
- lastIndexOf(a, x)::
+ else strict(if std.isString(a) then std.findSubstr(x, a) else [i for i in
indexes(a) if a[i] == x],
+ function(found) if std.length(found) == 0 then -1 else
found[0])),
+ lastIndexOf(a0, x):: strict(a0, function(a)
if a == null then -1
- else if std.isString(a) then (local found = std.findSubstr(x, a); if
std.length(found) == 0 then -1 else found[std.length(found) - 1])
- else (local found = [i for i in indexes(a) if a[i] == x]; if
std.length(found) == 0 then -1 else found[std.length(found) - 1]),
- take(a, n):: if a == null then null else a[0:std.max(0, std.min(n,
std.length(a)))],
- drop(a, n):: if a == null then null else a[std.max(0, std.min(n,
std.length(a))):],
- splitAt(a, n):: if a == null then null else { l: dw.take(a, n), r:
dw.drop(a, n) },
- divideBy(a, n):: if a == null then null else [a[i:std.min(i + n,
std.length(a))] for i in std.range(0, std.length(a) - 1) if i % n == 0],
+ else strict(if std.isString(a) then std.findSubstr(x, a) else [i for i in
indexes(a) if a[i] == x],
+ function(found) if std.length(found) == 0 then -1 else
found[std.length(found) - 1])),
+ take(a0, n):: strict(a0, function(a) if a == null then null else
a[0:std.max(0, std.min(n, std.length(a)))]),
+ drop(a0, n):: strict(a0, function(a) if a == null then null else
a[std.max(0, std.min(n, std.length(a))):]),
+ splitAt(a0, n):: strict(a0, function(a) if a == null then null else { l:
dw.take(a, n), r: dw.drop(a, n) }),
+ divideBy(a0, n):: strict(a0, function(a)
+ if a == null then null else [a[i:std.min(i + n, std.length(a))] for i in
std.range(0, std.length(a) - 1) if i % n == 0]),
// -- Numbers
- abs(x):: if x == null then null else if x < 0 then -x else x,
+ abs(x0):: strict(x0, function(x) if x == null then null else if x < 0 then
-x else x),
// half up, as DataWeave
- round(x):: if x == null then null else if x < 0 then -std.floor(-x + 0.5)
else std.floor(x + 0.5),
+ round(x0):: strict(x0, function(x) if x == null then null else if x < 0 then
-std.floor(-x + 0.5) else std.floor(x + 0.5)),
isEven(x):: x % 2 == 0,
isOdd(x):: x % 2 != 0,
- isInteger(x):: std.isNumber(x) && std.floor(x) == x,
- isDecimal(x):: std.isNumber(x) && std.floor(x) != x,
+ isInteger(x0):: strict(x0, function(x) std.isNumber(x) && std.floor(x) == x),
+ isDecimal(x0):: strict(x0, function(x) std.isNumber(x) && std.floor(x) != x),
}
diff --git
a/components/camel-datasonnet/src/test/java/org/apache/camel/language/datasonnet/DataWeaveCorpusTest.java
b/components/camel-datasonnet/src/test/java/org/apache/camel/language/datasonnet/DataWeaveCorpusTest.java
index e15ccbb0dfa6..8885cc025994 100644
---
a/components/camel-datasonnet/src/test/java/org/apache/camel/language/datasonnet/DataWeaveCorpusTest.java
+++
b/components/camel-datasonnet/src/test/java/org/apache/camel/language/datasonnet/DataWeaveCorpusTest.java
@@ -18,6 +18,7 @@ package org.apache.camel.language.datasonnet;
import java.io.IOException;
import java.io.InputStream;
+import java.io.StringReader;
import java.nio.charset.StandardCharsets;
import java.util.ArrayList;
import java.util.LinkedHashMap;
@@ -25,8 +26,19 @@ import java.util.List;
import java.util.Locale;
import java.util.Map;
import java.util.Properties;
+import java.util.TreeMap;
import java.util.stream.Stream;
+import javax.xml.XMLConstants;
+import javax.xml.parsers.DocumentBuilderFactory;
+
+import org.w3c.dom.Element;
+import org.w3c.dom.NamedNodeMap;
+import org.w3c.dom.Node;
+import org.w3c.dom.Text;
+
+import org.xml.sax.InputSource;
+
import com.fasterxml.jackson.databind.ObjectMapper;
import org.apache.camel.Exchange;
import org.apache.camel.Expression;
@@ -40,6 +52,7 @@ import org.junit.jupiter.params.provider.Arguments;
import org.junit.jupiter.params.provider.MethodSource;
import org.skyscreamer.jsonassert.JSONAssert;
+import static org.junit.jupiter.api.Assertions.assertEquals;
import static org.junit.jupiter.api.Assertions.assertThrows;
import static org.junit.jupiter.api.Assertions.fail;
@@ -163,11 +176,54 @@ class DataWeaveCorpusTest extends CamelTestSupport {
fail("Converted without error but fails when evaluated: " +
exchange.getException().getMessage(),
exchange.getException());
}
- if (!"?".equals(expected)) {
+ if ("?".equals(expected)) {
+ return;
+ }
+ if (entry.directive("outType", JSON).endsWith("xml")) {
+ assertEquals(canonicalXml(expected), canonicalXml(actual), actual);
+ } else {
JSONAssert.assertEquals("{\"v\":" + expected + "}", "{\"v\":" +
actual + "}", true);
}
}
+ // An XML document without its formatting: whether it has an XML
declaration, and its elements with their namespace,
+ // attributes (in order of name, without the namespace declarations, which
can be on another element) and text
+ private static String canonicalXml(String xml) throws Exception {
+ DocumentBuilderFactory factory = DocumentBuilderFactory.newInstance();
+ factory.setNamespaceAware(true);
+
factory.setFeature("http://apache.org/xml/features/disallow-doctype-decl";,
true);
+ String document = xml.strip();
+ StringBuilder sb = new StringBuilder(document.startsWith("<?xml") ?
"<?xml?>" : "");
+ canonicalXml(factory.newDocumentBuilder().parse(new InputSource(new
StringReader(document))).getDocumentElement(),
+ sb);
+ return sb.toString();
+ }
+
+ private static void canonicalXml(Element element, StringBuilder sb) {
+ sb.append('<').append(element.getTagName());
+ if (element.getNamespaceURI() != null) {
+ sb.append(" {").append(element.getNamespaceURI()).append('}');
+ }
+ Map<String, String> attributes = new TreeMap<>();
+ NamedNodeMap map = element.getAttributes();
+ for (int i = 0; i < map.getLength(); i++) {
+ Node attribute = map.item(i);
+ if
(!XMLConstants.XMLNS_ATTRIBUTE_NS_URI.equals(attribute.getNamespaceURI())) {
+ attributes.put(attribute.getNodeName(),
attribute.getNodeValue());
+ }
+ }
+ attributes.forEach((name, value) -> sb.append('
').append(name).append("=\"").append(value).append('"'));
+ sb.append('>');
+ for (Node child = element.getFirstChild(); child != null; child =
child.getNextSibling()) {
+ if (child instanceof Element e) {
+ canonicalXml(e, sb);
+ } else if (child instanceof Text text &&
!text.getData().isBlank()) {
+ sb.append(text.getData().strip());
+ }
+ }
+ sb.append("</").append(element.getTagName()).append('>');
+ }
+
private Expression createExpression(Entry entry, String dataWeave) {
return
context.resolveLanguage("datasonnet").createExpression(dataWeave,
new Object[] {
diff --git
a/components/camel-datasonnet/src/test/resources/dataweave-corpus/corpus.txt
b/components/camel-datasonnet/src/test/resources/dataweave-corpus/corpus.txt
index 33592c0f4102..219854cdfc99 100644
--- a/components/camel-datasonnet/src/test/resources/dataweave-corpus/corpus.txt
+++ b/components/camel-datasonnet/src/test/resources/dataweave-corpus/corpus.txt
@@ -17,7 +17,8 @@
# snippet is the output of the DataWeave CLI (dw 2.12, English locale); vars
and attributes are given to dw as inputs.
# Each entry starts with "=== <id> <title>", followed by directives and then
the DataWeave script:
#
-# @exp <json> the output DataWeave produces (as JSON); "?" = any
output, PARSE-ERROR = invalid DataWeave
+# @exp <json> the output DataWeave produces (as JSON, or the XML of
an XML output, compared without formatting
+# and namespace declarations); "?" = any output,
PARSE-ERROR = invalid DataWeave
# @in <text> the body (default: default.json)
# @inType / @outType the body and output media types (default:
application/json)
# @vars / @headers exchange variables / message headers as a JSON object
@@ -122,7 +123,7 @@ fun fullName(c) = c.first ++ " " ++ c.last
}
=== H02 output xml
@outType application/xml
-@exp ?
+@exp <?xml version='1.0'
encoding='UTF-8'?><order><id>7</id><name>Ann</name></order>
%dw 2.0
output application/xml
---
@@ -163,9 +164,8 @@ output application/json
---
countBy(payload.nums, (n) -> n > 2)
=== H07 ns declaration and ns-qualified key
-@unsupported XML namespaces
@outType application/xml
-@exp ?
+@exp <?xml version='1.0' encoding='UTF-8'?><ns0:order
xmlns:ns0="http://example.com/order";><id>7</id></ns0:order>
%dw 2.0
output application/xml
ns ns0 http://example.com/order
@@ -681,10 +681,9 @@ payload.customer ++ { vip: true }
=== O11 quoted keys
@exp {"first-name":"Ann"}
{ "first-name": payload.customer.first }
-=== O12 object key with ns/attrs
-@unsupported XML attributes in output
+=== O12 object key with attributes
@outType application/xml
-@exp ?
+@exp <?xml version='1.0' encoding='UTF-8'?><order
id="7"><name>Ann</name></order>
%dw 2.0
output application/xml
---
@@ -719,11 +718,238 @@ payload.order.item
payload.orders.*order map $.@id
=== XM04 xml output with list
@outType application/xml
-@exp ?
+@exp <?xml version='1.0'
encoding='UTF-8'?><items><item>A1</item><item>B2</item><item>A1</item></items>
%dw 2.0
output application/xml
---
{ items: { (payload.items map ((i) -> { item: i.sku })) } }
+=== XM05 SOAP selectors ignore namespace prefixes
+@inType application/xml
+@in <soapenv:Envelope
xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/";
xmlns:ns0="http://example.com/orders";><soapenv:Body><ns0:getOrderResponse><ns0:order
id="7"><ns0:item>apple</ns0:item></ns0:order></ns0:getOrderResponse></soapenv:Body></soapenv:Envelope>
+@exp ["7","apple"]
+[payload.Envelope.Body.getOrderResponse.order.@id,
payload.Envelope.Body.getOrderResponse.order.item]
+=== XM06 namespace-qualified selectors
+@inType application/xml
+@in <soapenv:Envelope
xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/";
xmlns:ns0="http://example.com/orders";><soapenv:Body><ns0:getOrderResponse><ns0:order
id="7"><ns0:item>apple</ns0:item></ns0:order></ns0:getOrderResponse></soapenv:Body></soapenv:Envelope>
+@exp ["apple","7",null]
+%dw 2.0
+output application/json
+ns o http://example.com/orders
+ns s http://schemas.xmlsoap.org/soap/envelope/
+---
+[payload.s#Envelope.s#Body.o#getOrderResponse.o#order.o#item,
payload.Envelope.Body.o#getOrderResponse.order.@id, payload.s#Envelope.o#Body]
+=== XM07 element with child elements written as JSON
+@inType application/xml
+@in <order id="1"><item>apple</item><note>n</note><empty/></order>
+@exp {"order":{"item":"apple","note":"n","empty":null}}
+payload
+=== XM08 empty element, keys, size and attributes of an element
+@inType application/xml
+@in <order id="1"
status="new"><item>apple</item><item>pear</item><note>n</note><empty/></order>
+@exp
[null,["item","item","note","empty"],4,["apple","pear"],{"id":"1","status":"new"}]
+[payload.order.empty, keysOf(payload.order), sizeOf(payload.order),
payload.order.*item, payload.order.@]
+=== XM09 elements repeated with others in between keep the document order
+@inType application/xml
+@in <a><x>1</x><y>2</y><x>3</x></a>
+@exp [["x","y","x"],["1","2","3"]]
+[keysOf(payload.a), valuesOf(payload.a)]
+=== XM10 attributes and text of the items of an implicit lambda
+@inType application/xml
+@in <orders><order id="1">a</order><order id="2">b</order></orders>
+@exp
[{"id":"1","text":"a","all":{"id":"1"}},{"id":"2","text":"b","all":{"id":"2"}}]
+payload.orders.*order map { id: $.@id, text: $, all: $.@ }
+=== XM11 attributes of a lambda parameter
+@inType application/xml
+@in <orders><order id="1">a</order><order id="2">b</order></orders>
+@exp [{"id":"1","v":"a","i":0},{"id":"2","v":"b","i":1}]
+payload.orders.*order map ((o, i) -> { id: o.@id, v: o, i: i })
+=== XM12 existence of elements and attributes
+@inType application/xml
+@in <order id="1"><item>apple</item><empty/></order>
+@exp [true,false,true,true,false]
+[payload.order.item?, payload.order.missing?, payload.order.empty?,
payload.order.@id?, payload.order.@missing?]
+=== XM13 object functions on an element
+@inType application/xml
+@in <order id="1"><item>apple</item><note>n</note><empty/></order>
+@exp
[{"ITEM":"apple","NOTE":"n","EMPTY":null},["item","note","empty"],{"item":"apple","note":"n"},{"item":"apple","note":"n","empty":null},{"item":"apple","empty":null},{"item":"apple"},["apple","n",null]]
+[payload.order mapObject ((v, k) -> { (upper(k)): v }), payload.order pluck
$$, payload.order filterObject ((v) -> v != null), { (payload.order) },
payload.order - "note", payload.order -- ["note", "empty"],
valuesOf(payload.order)]
+=== XM14 entriesOf with the attributes of the elements
+@inType application/xml
+@in <order><item sku="A1">apple</item><note>n</note></order>
+@exp
[{"key":"item","value":"apple","attributes":{"sku":"A1"}},{"key":"note","value":"n","attributes":{}}]
+entriesOf(payload.order)
+=== XM15 CDATA
+@inType application/xml
+@in <a><![CDATA[x < y]]></a>
+@exp "x < y"
+payload.a
+=== XM16 descendants
+@inType application/xml
+@in
<orders><order><item>a</item></order><order><item>b</item><item>c</item></order></orders>
+@exp [["a","b"],["a","b","c"]]
+[payload..item, payload..*item]
+=== XM17 attributes of a variable
+@inType application/xml
+@in <order id="7"><item sku="A1">apple</item><note>n</note></order>
+@exp ["7","A1"]
+%dw 2.0
+output application/json
+var o = payload.order
+---
+[o.@id, o.item.@sku]
+=== XM18 groupBy on an element
+@inType application/xml
+@in <order><item>apple</item><item>pear</item><note>n</note></order>
+@exp {"apple":{"item":"apple"},"pear":{"item":"pear"},"n":{"note":"n"}}
+payload.order groupBy $
+=== XM19 XML to XML
+@inType application/xml
+@outType application/xml
+@in <order id="7"><item>a</item><empty/></order>
+@exp <?xml version='1.0' encoding='UTF-8'?><order
id="7"><item>a</item><empty/></order>
+%dw 2.0
+output application/xml
+---
+payload
+=== XM20 SOAP body content as XML
+@inType application/xml
+@outType application/xml
+@in <soapenv:Envelope
xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/";
xmlns:ns0="http://example.com/orders";><soapenv:Body><ns0:getOrderResponse><ns0:order
id="7"><ns0:item>apple</ns0:item></ns0:order></ns0:getOrderResponse></soapenv:Body></soapenv:Envelope>
+@exp <?xml version='1.0' encoding='UTF-8'?><ns0:getOrderResponse
xmlns:ns0="http://example.com/orders";><ns0:order
id="7"><ns0:item>apple</ns0:item></ns0:order></ns0:getOrderResponse>
+%dw 2.0
+output application/xml
+---
+payload.Envelope.Body
+=== XM21 element of the input under another key
+@inType application/xml
+@outType application/xml
+@in <soapenv:Envelope
xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/";
xmlns:ns0="http://example.com/orders";><soapenv:Body><ns0:getOrderResponse><ns0:order
id="7"><ns0:item>apple</ns0:item></ns0:order></ns0:getOrderResponse></soapenv:Body></soapenv:Envelope>
+@exp <?xml version='1.0'
encoding='UTF-8'?><resp><status>ok</status><order><ns0:item
xmlns:ns0="http://example.com/orders";>apple</ns0:item></order></resp>
+%dw 2.0
+output application/xml
+---
+{ resp: { status: "ok", order: payload.Envelope.Body.getOrderResponse.order } }
+=== XM22 attributes, null attribute and null element
+@outType application/xml
+@exp <?xml version='1.0' encoding='UTF-8'?><root><price currency="USD"
none="null">10</price><flag a="true">true</flag><empty/></root>
+%dw 2.0
+output application/xml
+---
+{ root: { price @(currency: "USD", none: null): 10, flag @(a: true): true,
empty: null } }
+=== XM23 skipNullOn everywhere in XML
+@outType application/xml
+@exp <?xml version='1.0' encoding='UTF-8'?><root><price
currency="USD">10</price></root>
+%dw 2.0
+output application/xml skipNullOn="everywhere"
+---
+{ root: { price @(currency: "USD", none: null): 10, empty: null } }
+=== XM24 skipNullOn attributes
+@outType application/xml
+@exp <?xml version='1.0' encoding='UTF-8'?><root><price
currency="USD">10</price><empty/></root>
+%dw 2.0
+output application/xml skipNullOn="attributes"
+---
+{ root: { price @(currency: "USD", none: null): 10, empty: null } }
+=== XM25 namespaces of the header in XML
+@outType application/xml
+@exp <?xml version='1.0' encoding='UTF-8'?><o:a xmlns:o="http://x";
id="1"><o:b>1</o:b><c><o:d>2</o:d></c></o:a>
+%dw 2.0
+output application/xml
+ns o http://x
+---
+{ o#a @(id: 1): { o#b: "1", c: { o#d: 2 } } }
+=== XM26 writeDeclaration false
+@outType application/xml
+@exp <root><a>1</a></root>
+%dw 2.0
+output application/xml writeDeclaration=false
+---
+{ root: { a: 1 } }
+=== XM27 arrays and object spreads are elements that repeat
+@outType application/xml
+@exp <?xml version='1.0'
encoding='UTF-8'?><root><head>1</head><item>A1</item><item>B2</item><item>A1</item><n>1</n><n>2</n><tail>2</tail></root>
+%dw 2.0
+output application/xml
+---
+{ root: { head: 1, (payload.items map { item: $.sku }), n: [1, 2], tail: 2 } }
+=== XM28 keys that repeat keep their order
+@outType application/xml
+@exp <?xml version='1.0'
encoding='UTF-8'?><root><a>1</a><b>2</b><a>3</a></root>
+%dw 2.0
+output application/xml
+---
+{ root: { a: 1, b: 2, a: 3 } }
+=== XM29 repeated elements with attributes from XML
+@inType application/xml
+@outType application/xml
+@in <orders><order id="1"><item>a</item></order><order
id="2"><item>b</item></order></orders>
+@exp <?xml version='1.0' encoding='UTF-8'?><ids><id n="0">1</id><id
n="1">2</id></ids>
+%dw 2.0
+output application/xml
+---
+{ ids: { (payload.orders.*order map { id @(n: $$): $.@id }) } }
+=== XM30 copy of an element with an extra element
+@inType application/xml
+@outType application/xml
+@in <order id="7"><item>a</item><item>b</item><note>n</note></order>
+@exp <?xml version='1.0'
encoding='UTF-8'?><copy><item>a</item><item>b</item><note>n</note><extra>x</extra></copy>
+%dw 2.0
+output application/xml
+---
+{ copy: { (payload.order), extra: "x" } }
+=== XM31 element of the input under a key with attributes
+@inType application/xml
+@outType application/xml
+@in <orders><order id="1"><item>a</item></order></orders>
+@exp <?xml version='1.0' encoding='UTF-8'?><x n="1"><item>a</item></x>
+%dw 2.0
+output application/xml
+---
+{ x @(n: "1"): payload.orders.order }
+=== XM32 empty object and array, numbers and escaping in XML
+@outType application/xml
+@exp <?xml version='1.0'
encoding='UTF-8'?><root><e/><ea/><n>1.5</n><t>true</t><s>x & y <
z</s></root>
+%dw 2.0
+output application/xml
+---
+{ root: { e: {}, ea: [], n: 1.50, t: true, s: "x & y < z" } }
+=== XM33 elements with the same local name in two namespaces
+@inType application/xml
+@in <root xmlns:p="urn:p"
xmlns:q="urn:q"><p:item>1</p:item><q:item>2</q:item><p:item>3</p:item></root>
+@exp [["1","2","3"],["1","2","3"],"1",["item","item","item"],["1"]]
+[payload.root.*item, payload..*item, payload.root.item, keysOf(payload.root),
payload..item]
+=== XM34 CDATA next to text
+@inType application/xml
+@in <root><b>x<![CDATA[y]]>z</b><c>t<![CDATA[u]]><d/>v</c></root>
+@exp ["xyz",["__text","d","__text"],"xyz"]
+[payload.root.b, keysOf(payload.root.c), payload.root.*b[0]]
+=== XM35 default namespace
+@inType application/xml
+@in <order xmlns="http://x/o";><item>a</item></order>
+@exp ["a","a",null,null]
+%dw 2.0
+output application/json
+ns o http://x/o
+ns w http://wrong
+---
+[payload.o#order.o#item, payload.order.item, payload.w#order,
payload.order.w#item]
+=== XM36 empty array with an object spread in XML
+@outType application/xml
+@in {"items":[{"sku":"A1"}]}
+@exp <?xml version='1.0'
encoding='UTF-8'?><root><item>A1</item><a/><b>1</b></root>
+%dw 2.0
+output application/xml
+---
+{ root: { (payload.items map { item: $.sku }), a: [], b: 1 } }
+=== XM37 nested lambdas with the same parameter name
+@inType application/xml
+@outType application/xml
+@in <orders><order id="1"><item sku="A">x</item></order></orders>
+@exp <?xml version='1.0'
encoding='UTF-8'?><out><items><sku>A</sku></items><v><item
sku="A">x</item></v></out>
+%dw 2.0
+output application/xml
+---
+{ out: { (payload.orders.*order map ((o) -> { items: { (o.*item map ((o) -> {
sku: o.@sku })) }, v: o })) } }
=== CS01 csv input
@inType application/csv
@in name,age\nann,30\nbob,40
diff --git
a/components/camel-dataweave/src/main/java/org/apache/camel/component/dataweave/DataWeaveAst.java
b/components/camel-dataweave/src/main/java/org/apache/camel/component/dataweave/DataWeaveAst.java
index cc89bb27adc2..ef9ac633460b 100644
---
a/components/camel-dataweave/src/main/java/org/apache/camel/component/dataweave/DataWeaveAst.java
+++
b/components/camel-dataweave/src/main/java/org/apache/camel/component/dataweave/DataWeaveAst.java
@@ -34,10 +34,11 @@ public sealed interface DataWeaveAst {
/**
* The header directives. {@code outputProperties} are the writer
properties of the output directive (such as
- * {@code skipNullOn}), and {@code imports} the imported module paths
(such as {@code dw::core::Strings}).
+ * {@code skipNullOn}), {@code imports} the imported module paths (such as
{@code dw::core::Strings}), and
+ * {@code namespaces} the URIs of the declared XML namespace prefixes
({@code ns prefix uri}).
*/
record Header(String version, String outputType, Map<String, String>
outputProperties, List<InputDecl> inputs,
- List<String> imports)
+ List<String> imports, Map<String, String> namespaces)
implements
DataWeaveAst {
}
@@ -91,11 +92,21 @@ public sealed interface DataWeaveAst {
/**
* An object entry {@code key: value}, where a {@code dynamic} key is an
expression ({@code (expr): value}). A
* {@code null} key is an object spread {@code (expr)}, which adds the
entries of an object (or array of objects).
- * The {@code condition} of a conditional entry {@code (key: value) if
cond} is null for an unconditional entry.
+ * The {@code condition} of a conditional entry {@code (key: value) if
cond} is null for an unconditional entry, and
+ * the {@code attributes} are the XML attributes of the key ({@code key
@(name: value): ...}).
*/
- record ObjectEntry(DataWeaveAst key, DataWeaveAst value, boolean dynamic,
DataWeaveAst condition)
+ record ObjectEntry(DataWeaveAst key, DataWeaveAst value, boolean dynamic,
DataWeaveAst condition,
+ List<ObjectEntry> attributes)
implements
DataWeaveAst {
+
+ public ObjectEntry(DataWeaveAst key, DataWeaveAst value, boolean
dynamic, DataWeaveAst condition) {
+ this(key, value, dynamic, condition, List.of());
+ }
+ }
+
+ /** A name in an XML namespace: {@code prefix#name}, with the prefix
declared in the header. */
+ record QName(String prefix, String name) implements DataWeaveAst {
}
record ArrayLit(List<DataWeaveAst> elements) implements DataWeaveAst {
@@ -120,12 +131,26 @@ public sealed interface DataWeaveAst {
record AttributeAccess(DataWeaveAst object, String attribute) implements
DataWeaveAst {
}
+ /** Attributes selector {@code .@}: all the attributes of an XML element.
*/
+ record AllAttributes(DataWeaveAst object) implements DataWeaveAst {
+ }
+
+ /** Single-value selector of an element in an XML namespace {@code
.prefix#field}. */
+ record QualifiedFieldAccess(DataWeaveAst object, String prefix, String
field) implements DataWeaveAst {
+ }
+
/** Multi-value selector {@code .*field}. */
record MultiValueSelector(DataWeaveAst object, String field) implements
DataWeaveAst {
}
- /** Descendants selector {@code ..field}. */
- record DescendantSelector(DataWeaveAst object, String field) implements
DataWeaveAst {
+ /**
+ * Descendants selector {@code ..field}, or {@code ..*field} (multi) for
all the values of an XML element that
+ * repeats.
+ */
+ record DescendantSelector(DataWeaveAst object, String field, boolean
multi) implements DataWeaveAst {
+ public DescendantSelector(DataWeaveAst object, String field) {
+ this(object, field, false);
+ }
}
/** Index or dynamic key selector {@code [expr]}; the index is a {@link
Range} for {@code [a to b]}. */
diff --git
a/components/camel-dataweave/src/main/java/org/apache/camel/component/dataweave/DataWeaveConverter.java
b/components/camel-dataweave/src/main/java/org/apache/camel/component/dataweave/DataWeaveConverter.java
index 0064db61b0d7..ade2541867b9 100644
---
a/components/camel-dataweave/src/main/java/org/apache/camel/component/dataweave/DataWeaveConverter.java
+++
b/components/camel-dataweave/src/main/java/org/apache/camel/component/dataweave/DataWeaveConverter.java
@@ -28,6 +28,7 @@ import java.util.Map;
import java.util.Set;
import java.util.function.Predicate;
+import org.apache.camel.component.dataweave.DataWeaveAst.AllAttributes;
import org.apache.camel.component.dataweave.DataWeaveAst.ArrayLit;
import org.apache.camel.component.dataweave.DataWeaveAst.AttributeAccess;
import org.apache.camel.component.dataweave.DataWeaveAst.BinaryOp;
@@ -57,6 +58,8 @@ import
org.apache.camel.component.dataweave.DataWeaveAst.NumberLit;
import org.apache.camel.component.dataweave.DataWeaveAst.ObjectEntry;
import org.apache.camel.component.dataweave.DataWeaveAst.ObjectLit;
import org.apache.camel.component.dataweave.DataWeaveAst.Parens;
+import org.apache.camel.component.dataweave.DataWeaveAst.QName;
+import org.apache.camel.component.dataweave.DataWeaveAst.QualifiedFieldAccess;
import org.apache.camel.component.dataweave.DataWeaveAst.Range;
import org.apache.camel.component.dataweave.DataWeaveAst.RegexLit;
import org.apache.camel.component.dataweave.DataWeaveAst.Script;
@@ -165,13 +168,18 @@ public class DataWeaveConverter {
"year", "month", "day", "hour", "minutes", "seconds",
"milliseconds", "nanoseconds", "dayOfWeek",
"dayOfYear", "offsetSeconds", "quarter", "timezone");
- // A variable or function in scope: the arity of a function, or -1 for any
other value, and for a variable
- // with a date or time (an ISO-8601 string) the coercion that gave the
date (for its format)
- private record Binding(String name, int arity, TypeCoercion date) {
+ // A variable or function in scope: the arity of a function, or -1 for any
other value, for a variable with a date
+ // or time (an ISO-8601 string) the coercion that gave the date (for its
format), and whether it is a lambda
+ // parameter kept as an XML element (raw) because the lambda reads its
attributes (its value is then its text)
+ private record Binding(String name, int arity, TypeCoercion date, boolean
raw) {
static final int VALUE = -1;
+ Binding(String name, int arity, TypeCoercion date) {
+ this(name, arity, date, false);
+ }
+
Binding(String name, int arity) {
- this(name, arity, null);
+ this(name, arity, null, false);
}
}
@@ -183,6 +191,10 @@ public class DataWeaveConverter {
private final Deque<Map<String, Binding>> scopes = new ArrayDeque<>();
// the variables $, $$ and $$$ refer to in the body of an implicit lambda,
or null
private List<String> dollars;
+ // whether the script writes XML, the XML namespaces of its header, and
whether null attributes are left out
+ private boolean xmlOutput;
+ private Map<String, String> namespaces = Map.of();
+ private boolean skipNullAttributes;
public DataWeaveConverter() {
}
@@ -231,32 +243,58 @@ public class DataWeaveConverter {
needsDataWeaveLib = false;
scopes.clear();
dollars = null;
+ xmlOutput = false;
+ namespaces = Map.of();
+ skipNullAttributes = false;
}
// -- Script
private String emitScript(Script script) {
Header header = script.header();
- StringBuilder sb = new StringBuilder();
String output = header.outputType() != null ?
mediaType(header.outputType()) : null;
+ xmlOutput = output != null && output.contains("xml");
+ namespaces = header.namespaces();
+ String skipNullOn = xmlOutput ?
header.outputProperties().get("skipNullOn") : null;
+ skipNullAttributes = "attributes".equals(skipNullOn) ||
"everywhere".equals(skipNullOn);
+ StringBuilder todos = new StringBuilder();
+ for (String module : header.imports()) {
+ if (!KNOWN_MODULES.contains(module)) {
+ todos.append(todoComment("import of module", module));
+ }
+ }
+ // DataSonnet writes a null as an empty XML element only when asked
to, DataWeave always does
+ List<String> outputParameters = new ArrayList<>();
+ if (xmlOutput) {
+ outputParameters.add("nullasemptyelement=true");
+ }
+ String body = applyWriterProperties(emit(script.body()), output,
header.outputProperties(), outputParameters);
+ if (xmlOutput) {
+ // an element of the input is written as its content, the elements
in the order of the script, and the
+ // namespaces declared on the root element
+ List<String> declarations = new ArrayList<>();
+ namespaces.forEach((prefix, uri) -> declarations
+ .add((isJsonnetIdentifier(prefix) ? prefix :
string(prefix)) + ": " + string(uri)));
+ body = lib("xmlOutput") + "(" + body + ", "
+ + (declarations.isEmpty() ? "{}" : "{ " + String.join(", ",
declarations) + " }") + ")";
+ } else if (output != null) {
+ // the elements of an XML input are written as DataWeave writes
them, without the XML details
+ body = lib("output") + "(" + body + ", body)";
+ }
+
+ StringBuilder sb = new StringBuilder();
if (output != null || !header.inputs().isEmpty()) {
sb.append("/** DataSonnet\nversion=2.0\n");
if (output != null) {
- sb.append("output ").append(output).append('\n');
+ sb.append("output ").append(output);
+ outputParameters.forEach(p -> sb.append("; ").append(p));
+ sb.append('\n');
}
for (InputDecl input : header.inputs()) {
sb.append("input ").append(input.name()).append('
').append(mediaType(input.mediaType())).append('\n');
}
sb.append("*/\n");
}
- StringBuilder todos = new StringBuilder();
- for (String module : header.imports()) {
- if (!KNOWN_MODULES.contains(module)) {
- todos.append(todoComment("import of module", module));
- }
- }
- String body = emit(script.body());
- body = applyWriterProperties(body, output, header.outputProperties());
if (needsDataWeaveLib) {
sb.append(DW_IMPORT);
}
@@ -274,17 +312,26 @@ public class DataWeaveConverter {
};
}
- private String applyWriterProperties(String body, String output,
Map<String, String> properties) {
+ private String applyWriterProperties(
+ String body, String output, Map<String, String> properties,
List<String> outputParameters) {
String result = body;
for (Map.Entry<String, String> property : properties.entrySet()) {
String name = property.getKey();
- if ("indent".equals(name) || "encoding".equals(name)) {
+ String value = property.getValue();
+ if ("indent".equals(name) || "encoding".equals(name) || (xmlOutput
&& "inlineCloseOn".equals(name))) {
continue; // formatting only
}
- if ("skipNullOn".equals(name) && (output == null ||
output.endsWith("json"))) {
- result = lib("skipNulls") + "(" + result + ", " +
string(property.getValue()) + ")";
+ if (xmlOutput && "skipNullOn".equals(name) &&
"attributes".equals(value)) {
+ continue; // left out by withAttributes
+ }
+ if ("skipNullOn".equals(name) && (output == null ||
output.endsWith("json") || xmlOutput)) {
+ result = lib("skipNulls") + "(" + result + ", " +
string(value) + ")";
+ } else if (xmlOutput && "writeDeclaration".equals(name)) {
+ if ("false".equalsIgnoreCase(value)) {
+ outputParameters.add("omitxmldeclaration=true");
+ }
} else {
- result = todoComment("writer property", name + "=" +
property.getValue()) + result;
+ result = todoComment("writer property", name + "=" + value) +
result;
}
}
return result;
@@ -312,12 +359,15 @@ public class DataWeaveConverter {
return emitIdentifier(id);
} else if (node instanceof Dollar d) {
return emitDollar(d);
- } else if (node instanceof FieldAccess || node instanceof IndexAccess
|| node instanceof MultiValueSelector) {
+ } else if (node instanceof FieldAccess || node instanceof IndexAccess
|| node instanceof MultiValueSelector
+ || node instanceof QualifiedFieldAccess) {
return emitSelector(node, false);
+ } else if (node instanceof AllAttributes aa) {
+ return lib("attrs") + "(" + emitSelector(aa.object(), true) + ")";
} else if (node instanceof AttributeAccess aa) {
return lib("attr") + "(" + emitSelector(aa.object(), true) + ", "
+ string(aa.attribute()) + ")";
} else if (node instanceof DescendantSelector ds) {
- return lib("desc") + "(" + emit(ds.object()) + ", " +
string(ds.field()) + ")";
+ return lib(ds.multi() ? "descAll" : "desc") + "(" +
emit(ds.object()) + ", " + string(ds.field()) + ")";
} else if (node instanceof FilterSelector fs) {
return lib("filter") + "(" + emit(fs.object()) + ", " +
emitImplicitLambda(fs.condition(), ITEM_INDEX)
+ ")";
@@ -387,6 +437,9 @@ public class DataWeaveConverter {
// a date coerced with a format is written in that format
return formatDate(binding.name(), binding.date().properties());
}
+ if (binding != null && binding.raw()) {
+ return lib("text") + "(" + binding.name() + ")";
+ }
if (binding != null) {
return binding.name();
}
@@ -399,7 +452,9 @@ public class DataWeaveConverter {
private String emitDollar(Dollar d) {
if (dollars != null && d.level() <= dollars.size()) {
- return dollars.get(d.level() - 1);
+ String name = dollars.get(d.level() - 1);
+ Binding binding = lookup("$" + name);
+ return binding != null && binding.raw() ? lib("text") + "(" + name
+ ")" : name;
}
return todo("$ outside of a lambda", "$".repeat(d.level()));
}
@@ -408,9 +463,21 @@ public class DataWeaveConverter {
// A selector; raw keeps an XML element with only text as an object (for
its attributes)
private String emitSelector(DataWeaveAst node, boolean raw) {
+ String rawName = rawReference(node);
+ if (rawName != null) {
+ return rawName;
+ }
if (isExchangeAccess(node)) {
return emitExchangeAccess(node);
}
+ if (node instanceof QualifiedFieldAccess q) {
+ String uri = namespaces.get(q.prefix());
+ if (uri == null) {
+ return todo("undeclared namespace prefix", q.prefix() + "#" +
q.field());
+ }
+ return lib(raw ? "selNsRaw" : "selNs") + "(" +
emitSelector(q.object(), true) + ", " + string(uri) + ", "
+ + string(q.field()) + ")";
+ }
if (node instanceof FieldAccess fa && DATE_PARTS.contains(fa.field())
&& isDateValue(fa.object())) {
return "cml.datePart(" + emitDate(fa.object()) + ", " +
string(fa.field()) + ")";
}
@@ -426,9 +493,7 @@ public class DataWeaveConverter {
fields.add(0, string(f.field()));
base = f.object();
}
- String object = base instanceof FieldAccess || base instanceof
IndexAccess
- || base instanceof MultiValueSelector
- ? emitSelector(base, true) : emit(base);
+ String object = emitSelector(base, true);
if (fields.size() == 1) {
return lib(raw ? "selRaw" : "sel") + "(" + object + ", " +
fields.get(0) + ")";
}
@@ -444,6 +509,17 @@ public class DataWeaveConverter {
return emit(node);
}
+ // The Jsonnet name of a lambda parameter that is an XML element kept as
an object, or null
+ private String rawReference(DataWeaveAst node) {
+ Binding binding = null;
+ if (node instanceof Identifier id) {
+ binding = lookup(id.name());
+ } else if (node instanceof Dollar d && dollars != null && d.level() <=
dollars.size()) {
+ binding = lookup("$" + dollars.get(d.level() - 1));
+ }
+ return binding != null && binding.raw() ? binding.name() : null;
+ }
+
// A date or time: a date literal, now(), a coercion to a date or time
type, or a variable with one
private boolean isDateValue(DataWeaveAst node) {
DataWeaveAst n = unwrap(node);
@@ -554,6 +630,9 @@ public class DataWeaveConverter {
// -- Objects
private String emitObject(ObjectLit obj) {
+ if (xmlOutput && (obj.entries().stream().anyMatch(e -> e.key() ==
null) || hasRepeatedKey(obj))) {
+ return emitXmlObject(obj);
+ }
// fields are emitted as object literals, and an object spread (expr)
is merged in between
List<String> parts = new ArrayList<>();
List<String> fields = new ArrayList<>();
@@ -578,6 +657,53 @@ public class DataWeaveConverter {
return parts.size() == 1 ? parts.get(0) : "(" + String.join(" + ",
parts) + ")";
}
+ // An object written as XML: a key that repeats (also by object spreads)
is an element that repeats, so its parts
+ // (fields, and object spreads) are merged by dw.xmlObject
+ private String emitXmlObject(ObjectLit obj) {
+ List<String> parts = new ArrayList<>();
+ List<String> fields = new ArrayList<>();
+ Set<String> names = new HashSet<>();
+ for (ObjectEntry entry : obj.entries()) {
+ String name = entry.key() != null && !entry.dynamic() ?
keyName(entry.key()) : null;
+ if (entry.key() == null || (name != null && names.contains(name)))
{
+ if (!fields.isEmpty()) {
+ parts.add(objectLiteral(fields));
+ fields = new ArrayList<>();
+ names.clear();
+ }
+ }
+ if (entry.key() == null) {
+ String spread = emit(entry.value());
+ parts.add(entry.condition() != null
+ ? "(if " + emit(entry.condition()) + " then " + spread
+ " else null)" : spread);
+ } else {
+ if (name != null) {
+ names.add(name);
+ }
+ fields.add(emitField(entry));
+ }
+ }
+ if (!fields.isEmpty()) {
+ parts.add(objectLiteral(fields));
+ }
+ return lib("xmlObject") + "([" + String.join(", ", parts) + "])";
+ }
+
+ private static boolean hasRepeatedKey(ObjectLit obj) {
+ Set<String> names = new HashSet<>();
+ for (ObjectEntry entry : obj.entries()) {
+ if (entry.key() != null && !entry.dynamic() &&
!names.add(keyName(entry.key()))) {
+ return true;
+ }
+ }
+ return false;
+ }
+
+ // the name of a key as written in XML
+ private static String keyName(DataWeaveAst key) {
+ return key instanceof QName q ? q.prefix() + ":" + q.name() :
((StringLit) key).value();
+ }
+
private static String objectLiteral(List<String> fields) {
if (fields.isEmpty()) {
return "{}";
@@ -587,22 +713,42 @@ public class DataWeaveConverter {
private String emitField(ObjectEntry entry) {
String key;
+ String name = null;
if (entry.dynamic()) {
DataWeaveAst k = entry.key();
key = k instanceof StringLit || k instanceof Interpolation ?
emit(k) : lib("str") + "(" + emit(k) + ")";
} else {
- key = string(((StringLit) entry.key()).value());
+ name = staticName(entry.key());
+ key = string(name);
}
String value = emit(entry.value());
+ if (!entry.attributes().isEmpty() && xmlOutput) {
+ // attributes are only written in XML (DataWeave leaves them out
in the other formats)
+ List<String> attributes = new ArrayList<>();
+ for (ObjectEntry attribute : entry.attributes()) {
+ attributes.add(emitField(attribute));
+ }
+ value = lib("withAttributes") + "(" + value + ", { " +
String.join(", ", attributes) + " }"
+ + (skipNullAttributes ? ", true" : "") + ")";
+ }
+ String todo = entry.key() instanceof QName q && xmlOutput &&
!namespaces.containsKey(q.prefix())
+ ? todoComment("undeclared namespace prefix", q.prefix() + "#"
+ q.name()) : "";
if (entry.condition() != null) {
// a field with a null name is left out
- return "[if " + emit(entry.condition()) + " then " + key + " else
null]: " + value;
+ return todo + "[if " + emit(entry.condition()) + " then " + key +
" else null]: " + value;
}
if (entry.dynamic()) {
return "[" + key + "]: " + value;
}
- String name = ((StringLit) entry.key()).value();
- return (isJsonnetIdentifier(name) ? name : key) + ": " + value;
+ return todo + (isJsonnetIdentifier(name) ? name : key) + ": " + value;
+ }
+
+ // The name of a static key: a name, or prefix#name (written with the
prefix in XML only, as DataWeave does)
+ private String staticName(DataWeaveAst key) {
+ if (key instanceof QName q) {
+ return xmlOutput ? q.prefix() + ":" + q.name() : q.name();
+ }
+ return ((StringLit) key).value();
}
private static boolean isJsonnetIdentifier(String name) {
@@ -854,10 +1000,11 @@ public class DataWeaveConverter {
return wrongArguments(name, args);
}
List<String> emitted = new ArrayList<>();
+ boolean rawItems = functionArg != null &&
readsItemAttributes(args.get(functionArg.position()));
for (int i = 0; i < args.size(); i++) {
if (functionArg != null && i == functionArg.position()) {
- emitted.add(emitFunctionArg(args.get(i),
functionArg.params()));
- } else if (functionArg != null && i == 0 &&
readsItemAttributes(args.get(functionArg.position()))) {
+ emitted.add(emitFunctionArg(args.get(i), functionArg.params(),
rawItems));
+ } else if (rawItems && i == 0) {
// keep XML elements with only text as objects, as the
function reads their attributes
emitted.add(emitSelector(args.get(i), true));
} else {
@@ -896,14 +1043,19 @@ public class DataWeaveConverter {
// A function argument of a higher-order function, which DataWeave calls
with params.size() arguments
private String emitFunctionArg(DataWeaveAst arg, List<String> params) {
+ return emitFunctionArg(arg, params, false);
+ }
+
+ // rawItem: the first parameter is an XML element kept as an object,
because the function reads its attributes
+ private String emitFunctionArg(DataWeaveAst arg, List<String> params,
boolean rawItem) {
if (arg instanceof Lambda lam) {
if (lam.params().size() > params.size()) {
return todo("lambda with " + lam.params().size() + "
parameters", "expected at most " + params.size());
}
- return emitLambda(lam, params.size());
+ return emitLambda(lam, params.size(), rawItem);
}
if (containsDollar(arg)) {
- return emitImplicitLambda(arg, params);
+ return emitImplicitLambda(arg, params, rawItem);
}
if (arg instanceof Identifier id && lookup(id.name()) != null &&
lookup(id.name()).arity() >= 0) {
// a function by name: pass the arguments it accepts
@@ -924,10 +1076,15 @@ public class DataWeaveConverter {
// An expression with $, $$, $$$ (such as payload map $.name): a function
of the parameters
private String emitImplicitLambda(DataWeaveAst body, List<String> params) {
+ return emitImplicitLambda(body, params, false);
+ }
+
+ private String emitImplicitLambda(DataWeaveAst body, List<String> params,
boolean rawItem) {
List<String> names = freshNames(params);
Map<String, Binding> scope = new HashMap<>();
- for (String name : names) {
- scope.put("$" + name, new Binding(name, Binding.VALUE)); //
reserve the names
+ for (int i = 0; i < names.size(); i++) {
+ // reserve the names
+ scope.put("$" + names.get(i), new Binding(names.get(i),
Binding.VALUE, null, rawItem && i == 0));
}
List<String> saved = dollars;
dollars = names;
@@ -942,10 +1099,15 @@ public class DataWeaveConverter {
// A lambda with at least the given number of parameters (DataWeave passes
arguments a lambda does not declare)
private String emitLambda(Lambda lam, int arity) {
+ return emitLambda(lam, arity, false);
+ }
+
+ private String emitLambda(Lambda lam, int arity, boolean rawFirst) {
Map<String, Binding> scope = new HashMap<>();
List<String> params = new ArrayList<>();
Set<String> used = new HashSet<>();
- for (LambdaParam param : lam.params()) {
+ for (int i = 0; i < lam.params().size(); i++) {
+ LambdaParam param = lam.params().get(i);
String name = jsonnetName(param.name());
used.add(name);
String p = name;
@@ -953,7 +1115,7 @@ public class DataWeaveConverter {
p += " = " + emit(param.defaultValue());
}
params.add(p);
- scope.put(param.name(), new Binding(name, Binding.VALUE));
+ scope.put(param.name(), new Binding(name, Binding.VALUE, null,
rawFirst && i == 0));
}
for (int i = lam.params().size(); i < arity; i++) {
String name = "_" + i;
@@ -987,17 +1149,23 @@ public class DataWeaveConverter {
return names;
}
- // Whether a function argument reads an attribute of its (first)
parameter: (e) -> e.@id, or $.@id
+ // Whether a function argument reads the attributes of its (first)
parameter: (e) -> e.@id, $.@id or $.@
private static boolean readsItemAttributes(DataWeaveAst fn) {
+ Predicate<DataWeaveAst> item;
+ DataWeaveAst body;
if (fn instanceof Lambda lam) {
if (lam.params().isEmpty()) {
return false;
}
String name = lam.params().get(0).name();
- return anyNode(lam.body(), n -> n instanceof AttributeAccess aa &&
aa.object() instanceof Identifier id
- && id.name().equals(name));
+ item = n -> n instanceof Identifier id && id.name().equals(name);
+ body = lam.body();
+ } else {
+ item = n -> n instanceof Dollar d && d.level() == 1;
+ body = fn;
}
- return anyNode(fn, n -> n instanceof AttributeAccess aa && aa.object()
instanceof Dollar d && d.level() == 1);
+ return anyNode(body, n -> (n instanceof AttributeAccess aa &&
item.test(aa.object()))
+ || (n instanceof AllAttributes all &&
item.test(all.object())));
}
private static boolean anyNode(DataWeaveAst node, Predicate<DataWeaveAst>
predicate) {
diff --git
a/components/camel-dataweave/src/main/java/org/apache/camel/component/dataweave/DataWeaveLexer.java
b/components/camel-dataweave/src/main/java/org/apache/camel/component/dataweave/DataWeaveLexer.java
index 72b40e9d76d9..a871895b8095 100644
---
a/components/camel-dataweave/src/main/java/org/apache/camel/component/dataweave/DataWeaveLexer.java
+++
b/components/camel-dataweave/src/main/java/org/apache/camel/component/dataweave/DataWeaveLexer.java
@@ -19,6 +19,8 @@ package org.apache.camel.component.dataweave;
import java.util.ArrayList;
import java.util.List;
import java.util.Set;
+import java.util.regex.Matcher;
+import java.util.regex.Pattern;
/**
* Tokenizer for DataWeave 2.0 scripts.
@@ -83,6 +85,10 @@ public class DataWeaveLexer {
}
}
+ private static final Pattern HEADER_SEPARATOR_LINE =
Pattern.compile("(?m)^[ \\t]*---[ \\t]*\\r?$");
+ private static final Pattern NAMESPACE_DIRECTIVE
+ = Pattern.compile("ns[ \\t]+([A-Za-z_][A-Za-z0-9_]*)[ \\t]+(\\S+)[
\\t]*(//.*)?\\r?");
+
// A '/' after one of these identifiers starts a regular expression
literal rather than a division
private static final Set<String> REGEX_PREFIX_IDENTIFIERS = Set.of(
"replace", "contains", "matches", "splitBy", "scan", "find",
"match", "case", "startsWith", "endsWith");
@@ -92,6 +98,8 @@ public class DataWeaveLexer {
private int pos;
private int line;
private int col;
+ // whether the --- separator of the header was read
+ private boolean body;
public DataWeaveLexer(String input) {
this.input = input;
@@ -106,12 +114,40 @@ public class DataWeaveLexer {
if (pos >= input.length()) {
break;
}
- tokens.add(readToken());
+ if (!readNamespaceDirective()) {
+ Token token = readToken();
+ body |= token.type() == TokenType.HEADER_SEPARATOR;
+ tokens.add(token);
+ }
}
tokens.add(new Token(TokenType.EOF, "", line, col));
return tokens;
}
+ // A namespace declaration of the header, "ns prefix uri" on a line of its
own before the --- separator: the URI is
+ // read as a string (it has characters such as // that start a comment
elsewhere)
+ private boolean readNamespaceDirective() {
+ if (body || !input.startsWith("ns", pos) || (!tokens.isEmpty() &&
tokens.get(tokens.size() - 1).line() == line)
+ || !HEADER_SEPARATOR_LINE.matcher(input).region(pos,
input.length()).find()) {
+ return false;
+ }
+ int end = input.indexOf('\n', pos);
+ String rest = input.substring(pos, end < 0 ? input.length() : end);
+ Matcher m = NAMESPACE_DIRECTIVE.matcher(rest);
+ if (!m.matches()) {
+ return false;
+ }
+ int startLine = line;
+ int startCol = col;
+ tokens.add(token(TokenType.IDENTIFIER, "ns", startLine, startCol));
+ tokens.add(token(TokenType.IDENTIFIER, m.group(1), startLine, startCol
+ m.start(1)));
+ tokens.add(token(TokenType.STRING, m.group(2), startLine, startCol +
m.start(2)));
+ for (int i = 0; i < rest.length(); i++) {
+ advance();
+ }
+ return true;
+ }
+
private void skipWhitespaceAndComments() {
while (pos < input.length()) {
char c = input.charAt(pos);
diff --git
a/components/camel-dataweave/src/main/java/org/apache/camel/component/dataweave/DataWeaveParser.java
b/components/camel-dataweave/src/main/java/org/apache/camel/component/dataweave/DataWeaveParser.java
index 74c148495047..cee119a1e277 100644
---
a/components/camel-dataweave/src/main/java/org/apache/camel/component/dataweave/DataWeaveParser.java
+++
b/components/camel-dataweave/src/main/java/org/apache/camel/component/dataweave/DataWeaveParser.java
@@ -22,6 +22,7 @@ import java.util.List;
import java.util.Map;
import java.util.Set;
+import org.apache.camel.component.dataweave.DataWeaveAst.AllAttributes;
import org.apache.camel.component.dataweave.DataWeaveAst.ArrayLit;
import org.apache.camel.component.dataweave.DataWeaveAst.AttributeAccess;
import org.apache.camel.component.dataweave.DataWeaveAst.BinaryOp;
@@ -51,6 +52,8 @@ import
org.apache.camel.component.dataweave.DataWeaveAst.NumberLit;
import org.apache.camel.component.dataweave.DataWeaveAst.ObjectEntry;
import org.apache.camel.component.dataweave.DataWeaveAst.ObjectLit;
import org.apache.camel.component.dataweave.DataWeaveAst.Parens;
+import org.apache.camel.component.dataweave.DataWeaveAst.QName;
+import org.apache.camel.component.dataweave.DataWeaveAst.QualifiedFieldAccess;
import org.apache.camel.component.dataweave.DataWeaveAst.Range;
import org.apache.camel.component.dataweave.DataWeaveAst.RegexLit;
import org.apache.camel.component.dataweave.DataWeaveAst.Script;
@@ -97,7 +100,7 @@ public class DataWeaveParser {
* Parses a complete script: an optional header (directives and
declarations ending with {@code ---}) and the body.
*/
public DataWeaveAst parse() {
- Header header = new Header("2.0", null, Map.of(), List.of(),
List.of());
+ Header header = new Header("2.0", null, Map.of(), List.of(),
List.of(), Map.of());
List<DataWeaveAst> declarations = new ArrayList<>();
if (hasHeaderSeparator()) {
header = parseHeader(declarations);
@@ -146,6 +149,7 @@ public class DataWeaveParser {
Map<String, String> outputProperties = new LinkedHashMap<>();
List<InputDecl> inputs = new ArrayList<>();
List<String> imports = new ArrayList<>();
+ Map<String, String> namespaces = new LinkedHashMap<>();
while (!check(TokenType.HEADER_SEPARATOR)) {
Token start = current();
@@ -169,12 +173,10 @@ public class DataWeaveParser {
} else if (checkIdentifier("import")) {
imports.add(parseImport());
} else if (checkIdentifier("ns")) {
- StringBuilder text = new StringBuilder();
- while (current().line() == start.line() &&
!check(TokenType.EOF)) {
- text.append(current().value()).append(' ');
- advance();
- }
- declarations.add(new Unsupported(text.toString().trim(), "XML
namespace declaration"));
+ // ns prefix uri (the lexer reads the URI as a string)
+ advance();
+ String prefix = expectName();
+ namespaces.put(prefix, expect(TokenType.STRING).value());
} else if (checkIdentifier("type")) {
skipTypeDeclaration();
} else if (checkIdentifier("var") || checkIdentifier("fun")) {
@@ -184,7 +186,7 @@ public class DataWeaveParser {
}
}
advance(); // ---
- return new Header(version, outputType, outputProperties, inputs,
imports);
+ return new Header(version, outputType, outputProperties, inputs,
imports, namespaces);
}
private String parseMediaType() {
@@ -553,9 +555,17 @@ public class DataWeaveParser {
if (check(TokenType.DOT) && isTokenAhead(1, TokenType.DOT)) {
advance(); // .
advance(); // .
- if (check(TokenType.STAR) || check(TokenType.AT)) {
- expr = new Unsupported(".." + current().value(),
"descendants selector ..* / ..@");
- advance();
+ if (check(TokenType.STAR)) {
+ advance(); // *
+ expr = new DescendantSelector(expr, expectSelectorName(),
true);
+ } else if (check(TokenType.AT)) {
+ advance(); // @
+ String name = "";
+ if (check(TokenType.IDENTIFIER)) {
+ name = current().value();
+ advance();
+ }
+ expr = new Unsupported("..@" + name, "descendants selector
..@");
} else {
expr = new DescendantSelector(expr, expectSelectorName());
}
@@ -597,7 +607,7 @@ public class DataWeaveParser {
if (check(TokenType.IDENTIFIER) || check(TokenType.STRING)) {
return new AttributeAccess(expr, expectSelectorName());
}
- return new Unsupported(".@", "all attributes selector .@");
+ return new AllAttributes(expr);
}
if (check(TokenType.STAR)) {
advance(); // *
@@ -619,7 +629,7 @@ public class DataWeaveParser {
if (check(TokenType.IDENTIFIER) && isTokenAhead(1, TokenType.HASH)) {
String prefix = expectName();
advance(); // #
- return new Unsupported("." + prefix + "#" + expectSelectorName(),
"XML namespace selector");
+ return new QualifiedFieldAccess(expr, prefix,
expectSelectorName());
}
return new FieldAccess(expr, expectSelectorName());
}
@@ -791,7 +801,7 @@ public class DataWeaveParser {
advance(); // )
DataWeaveAst condition = parseEntryCondition();
for (ObjectEntry entry : group) {
- entries.add(new ObjectEntry(entry.key(), entry.value(),
entry.dynamic(), condition));
+ entries.add(new ObjectEntry(entry.key(), entry.value(),
entry.dynamic(), condition, entry.attributes()));
}
} else if (check(TokenType.LPAREN)) {
// (expr): value is a dynamic key, and (expr) alone an object
spread
@@ -799,9 +809,9 @@ public class DataWeaveParser {
DataWeaveAst expr = parseExpression();
expect(TokenType.RPAREN);
if (check(TokenType.AT)) {
- entries.add(unsupportedEntry(skipAttributes(), "XML
attributes"));
+ List<ObjectEntry> attributes = parseAttributes();
expect(TokenType.COLON);
- parseExpression();
+ entries.add(new ObjectEntry(expr, parseExpression(), true,
null, attributes));
} else if (check(TokenType.COLON)) {
advance();
entries.add(new ObjectEntry(expr, parseExpression(), true,
null));
@@ -856,45 +866,33 @@ public class DataWeaveParser {
} else {
key = new StringLit(expectName());
}
- if (check(TokenType.HASH)) {
+ if (check(TokenType.HASH) && !dynamic && key instanceof StringLit
prefix) {
+ // prefix#name: a name in an XML namespace
advance(); // #
- String name = expectName();
- expect(TokenType.COLON);
- parseExpression();
- return unsupportedEntry(token.value() + "#" + name, "XML namespace
prefix");
- }
- if (check(TokenType.AT)) {
- String attributes = skipAttributes();
- expect(TokenType.COLON);
- parseExpression();
- return unsupportedEntry(token.value() + " " + attributes, "XML
attributes");
+ key = new QName(prefix.value(), expectName());
}
+ List<ObjectEntry> attributes = check(TokenType.AT) ? parseAttributes()
: List.of();
expect(TokenType.COLON);
- return new ObjectEntry(key, parseExpression(), dynamic, null);
- }
-
- private static ObjectEntry unsupportedEntry(String text, String reason) {
- return new ObjectEntry(null, new Unsupported(text, reason), false,
null);
+ return new ObjectEntry(key, parseExpression(), dynamic, null,
attributes);
}
- private String skipAttributes() {
+ // The XML attributes of a key: @(name: value, ...)
+ private List<ObjectEntry> parseAttributes() {
advance(); // @
- StringBuilder text = new StringBuilder("@");
- if (check(TokenType.LPAREN)) {
- int depth = 0;
- do {
- if (check(TokenType.LPAREN)) {
- depth++;
- } else if (check(TokenType.RPAREN)) {
- depth--;
- } else if (check(TokenType.EOF)) {
- throw error("unterminated attributes");
- }
- text.append(current().value());
- advance();
- } while (depth > 0);
+ expect(TokenType.LPAREN);
+ List<ObjectEntry> attributes = new ArrayList<>();
+ while (!check(TokenType.RPAREN)) {
+ ObjectEntry attribute = parseObjectEntry();
+ if (!attribute.attributes().isEmpty()) {
+ throw error("an attribute has no attributes");
+ }
+ attributes.add(attribute);
+ if (!check(TokenType.RPAREN)) {
+ expect(TokenType.COMMA);
+ }
}
- return text.toString();
+ advance(); // )
+ return attributes;
}
private DataWeaveAst parseArray() {
diff --git
a/components/camel-dataweave/src/test/java/org/apache/camel/component/dataweave/DataWeaveConverterTest.java
b/components/camel-dataweave/src/test/java/org/apache/camel/component/dataweave/DataWeaveConverterTest.java
index 086b3004e9c9..0dfdf6357a1b 100644
---
a/components/camel-dataweave/src/test/java/org/apache/camel/component/dataweave/DataWeaveConverterTest.java
+++
b/components/camel-dataweave/src/test/java/org/apache/camel/component/dataweave/DataWeaveConverterTest.java
@@ -66,9 +66,11 @@ class DataWeaveConverterTest {
---
{ a: 1 }
""");
- assertEquals("/** DataSonnet\nversion=2.0\noutput
application/json\ninput payload application/xml\n*/\n{\n a: 1\n}",
+ // an XML input is written as DataWeave writes it (dw.output)
+ assertEquals("/** DataSonnet\nversion=2.0\noutput
application/json\ninput payload application/xml\n*/\n" + DW_IMPORT
+ + "dw.output({\n a: 1\n}, body)",
result);
- assertFalse(converter.needsDataWeaveLib());
+ assertTrue(converter.needsDataWeaveLib());
assertEquals(0, converter.getTodoCount());
}
@@ -81,7 +83,7 @@ class DataWeaveConverterTest {
@Test
void testSkipNullOn() {
String result = converter.convert("%dw 2.0\noutput application/json
skipNullOn=\"everywhere\", indent=false\n---\n1");
- assertTrue(result.endsWith("dw.skipNulls(1, \"everywhere\")"), result);
+ assertTrue(result.endsWith("dw.output(dw.skipNulls(1, \"everywhere\"),
body)"), result);
assertEquals(0, converter.getTodoCount());
}
@@ -283,6 +285,109 @@ class DataWeaveConverterTest {
assertEquals("{}", expr("{}"));
}
+ // -- XML
+
+ @Test
+ void testXmlOutput() {
+ String result = converter.convert("""
+ %dw 2.0
+ output application/xml writeDeclaration=false, indent=false
+ ns ns0 http://example.com/order
+ ---
+ { ns0#order @(id: payload.id): { item: payload.sku } }
+ """);
+ assertEquals("""
+ /** DataSonnet
+ version=2.0
+ output application/xml; nullasemptyelement=true;
omitxmldeclaration=true
+ */
+ """ + DW_IMPORT + """
+ dw.xmlOutput({
+ "ns0:order": dw.withAttributes({
+ item: dw.sel(body, "sku")
+ }, { id: dw.sel(body, "id") })
+ }, { ns0: "http://example.com/order"; })""", result);
+ assertEquals(0, converter.getTodoCount());
+ }
+
+ @Test
+ void testXmlOutputSkipsNullAttributes() {
+ String result
+ = converter.convert("%dw 2.0\noutput application/xml
skipNullOn=\"attributes\"\n---\n{ a @(x: null): 1 }");
+ assertTrue(result.endsWith("dw.xmlOutput({\n a: dw.withAttributes(1,
{ x: null }, true)\n}, {})"), result);
+ result = converter.convert("%dw 2.0\noutput application/xml
skipNullOn=\"everywhere\"\n---\n{ a @(x: null): 1 }");
+ assertTrue(result.endsWith(
+ "dw.xmlOutput(dw.skipNulls({\n a: dw.withAttributes(1, { x:
null }, true)\n}, \"everywhere\"), {})"),
+ result);
+ assertEquals(0, converter.getTodoCount());
+ }
+
+ @Test
+ void testXmlObjectWithRepeatedKeys() {
+ // in XML a key that repeats is an element that repeats: the fields
and object spreads are merged by dw.xmlObject
+ String result = converter.convert("%dw 2.0\noutput
application/xml\n---\n{ root: { a: 1, (payload.items), a: 2 } }");
+ assertTrue(result.endsWith("dw.xmlOutput({\n root: dw.xmlObject([{\n
a: 1\n }, dw.sel(body, \"items\"), {\n"
+ + " a: 2\n }])\n}, {})"),
+ result);
+ assertEquals(0, converter.getTodoCount());
+ }
+
+ @Test
+ void testXmlKeysInOtherOutputs() {
+ // attributes and namespace prefixes are only written in XML
+ assertTrue(converter.convert("%dw 2.0\noutput application/json\n---\n{
a @(x: 1): 2, o#b: 3 }")
+ .endsWith("dw.output({\n a: 2,\n b: 3\n}, body)"));
+ assertEquals(0, converter.getTodoCount());
+ }
+
+ @Test
+ void testNamespaceDirectiveOnlyInHeader() {
+ // a line "ns a b" in the body is an expression (ns is a variable)
+ String result = converter.convert("%dw 2.0\noutput
application/json\nvar ns = [1]\n---\nns map $");
+ assertTrue(result.endsWith("dw.output(local ns = [1];\ndw.map(ns,
function(item, index) item), body)"), result);
+ assertEquals(0, converter.getTodoCount());
+ }
+
+ @Test
+ void testUndeclaredNamespacePrefixIsTodo() {
+ String result = converter.convert("%dw 2.0\noutput
application/xml\n---\n{ x#a: 1 }");
+ assertTrue(result.contains("/* TODO: manual conversion needed --
undeclared namespace prefix: x#a */"), result);
+ assertEquals(1, converter.getTodoCount());
+ converter.convert("%dw 2.0\noutput
application/json\n---\npayload.x#a");
+ assertEquals(1, converter.getTodoCount());
+ }
+
+ @Test
+ void testXmlSelectors() {
+ String result = converter.convert("""
+ %dw 2.0
+ output application/json
+ ns o http://example.com/orders
+ ---
+ [payload.o#order.@, payload..*item, payload.order.o#item]
+ """);
+ assertTrue(result.endsWith("dw.output([dw.attrs(dw.selNsRaw(body,
\"http://example.com/orders\";, \"order\")), "
+ + "dw.descAll(body, \"item\"),
dw.selNs(dw.selRaw(body, \"order\"), "
+ + "\"http://example.com/orders\";,
\"item\")], body)"),
+ result);
+ assertEquals(0, converter.getTodoCount());
+ }
+
+ @Test
+ void testAttributesOfLambdaItems() {
+ // the items are kept as XML elements for their attributes, and used
as their text otherwise
+ assertEquals("dw.map(dw.multiRaw(body, \"order\"), function(item,
index) {\n id: dw.attr(item, \"id\"),\n"
+ + " text: dw.text(item)\n})",
+ expr("payload.*order map { id: $.@id, text: $ }"));
+ assertEquals("dw.map(dw.multiRaw(body, \"order\"), function(o, _1) {\n
all: dw.attrs(o),\n v: dw.text(o)\n})",
+ expr("payload.*order map ((o) -> { all: o.@, v: o })"));
+ // an inner lambda with a parameter of the same name does not change
the outer one
+ assertEquals("dw.map(dw.multiRaw(body, \"order\"), function(o, _1) {\n"
+ + " items: (dw.map(dw.multiRaw(o, \"item\"), function(o,
_1) dw.attr(o, \"sku\"))),\n v: dw.text(o)\n})",
+ expr("payload.*order map ((o) -> { items: (o.*item map ((o) ->
o.@sku)), v: o })"));
+ assertEquals(0, converter.getTodoCount());
+ }
+
// -- Match
@Test
@@ -363,7 +468,7 @@ class DataWeaveConverterTest {
void testUnsupportedConstructs() {
assertTrue(converter.convertExpression("payload update { case .a -> 1
}").contains("update operator"));
assertEquals(1, converter.getTodoCount());
- converter.convertExpression("{ a @(id: 1): 2 }");
+ converter.convertExpression("payload..@id");
assertEquals(1, converter.getTodoCount());
}
diff --git
a/components/camel-dataweave/src/test/java/org/apache/camel/component/dataweave/DataWeaveParserTest.java
b/components/camel-dataweave/src/test/java/org/apache/camel/component/dataweave/DataWeaveParserTest.java
index 981bdee5faae..aa522dfeb61a 100644
---
a/components/camel-dataweave/src/test/java/org/apache/camel/component/dataweave/DataWeaveParserTest.java
+++
b/components/camel-dataweave/src/test/java/org/apache/camel/component/dataweave/DataWeaveParserTest.java
@@ -19,6 +19,7 @@ package org.apache.camel.component.dataweave;
import java.util.List;
import java.util.Map;
+import org.apache.camel.component.dataweave.DataWeaveAst.AllAttributes;
import org.apache.camel.component.dataweave.DataWeaveAst.ArrayLit;
import org.apache.camel.component.dataweave.DataWeaveAst.AttributeAccess;
import org.apache.camel.component.dataweave.DataWeaveAst.BinaryOp;
@@ -42,6 +43,8 @@ import
org.apache.camel.component.dataweave.DataWeaveAst.NumberLit;
import org.apache.camel.component.dataweave.DataWeaveAst.ObjectEntry;
import org.apache.camel.component.dataweave.DataWeaveAst.ObjectLit;
import org.apache.camel.component.dataweave.DataWeaveAst.Parens;
+import org.apache.camel.component.dataweave.DataWeaveAst.QName;
+import org.apache.camel.component.dataweave.DataWeaveAst.QualifiedFieldAccess;
import org.apache.camel.component.dataweave.DataWeaveAst.Range;
import org.apache.camel.component.dataweave.DataWeaveAst.RegexLit;
import org.apache.camel.component.dataweave.DataWeaveAst.Script;
@@ -120,10 +123,11 @@ class DataWeaveParserTest {
}
@Test
- void shouldRepresentNamespaceDeclarationAsUnsupported() {
- Script script = parseScript("%dw 2.0\nns ns0
http://example.com\n---\npayload";);
- Block block = assertInstanceOf(Block.class, script.body());
- assertInstanceOf(Unsupported.class, block.declarations().get(0));
+ void shouldParseNamespaceDeclaration() {
+ Script script
+ = parseScript("%dw 2.0\nns ns0 http://example.com/a?b=1 // the
orders\noutput application/xml\n---\npayload");
+ assertEquals(Map.of("ns0", "http://example.com/a?b=1";),
script.header().namespaces());
+ assertEquals("payload", name(script.body()));
}
@Test
@@ -287,6 +291,13 @@ class DataWeaveParserTest {
assertEquals("id", assertInstanceOf(AttributeAccess.class,
parseExpr("payload.order.@id")).attribute());
assertEquals("item", assertInstanceOf(MultiValueSelector.class,
parseExpr("payload.*item")).field());
assertEquals("sku", assertInstanceOf(DescendantSelector.class,
parseExpr("payload..sku")).field());
+ assertTrue(assertInstanceOf(DescendantSelector.class,
parseExpr("payload..*sku")).multi());
+ assertInstanceOf(Unsupported.class, parseExpr("payload..@id"));
+ assertEquals("order", assertInstanceOf(FieldAccess.class,
+ assertInstanceOf(AllAttributes.class,
parseExpr("payload.order.@")).object()).field());
+ QualifiedFieldAccess qualified =
assertInstanceOf(QualifiedFieldAccess.class, parseExpr("payload.o#order"));
+ assertEquals("o", qualified.prefix());
+ assertEquals("order", qualified.field());
assertInstanceOf(ExistenceCheck.class, parseExpr("payload.a?"));
assertInstanceOf(FilterSelector.class,
parseExpr("payload.items[?($.qty > 1)]"));
Range range = assertInstanceOf(Range.class,
assertInstanceOf(IndexAccess.class, parseExpr("xs[-1 to 0]")).index());
@@ -309,6 +320,21 @@ class DataWeaveParserTest {
assertEquals("default", assertInstanceOf(StringLit.class,
entries.get(4).key()).value());
}
+ @Test
+ void shouldParseXmlKeys() {
+ ObjectLit obj = assertInstanceOf(ObjectLit.class, parseExpr("{ o#a
@(id: 1, o#b: x): 2, (k) @(c: 3): 4, d: 5 }"));
+ List<ObjectEntry> entries = obj.entries();
+ QName qName = assertInstanceOf(QName.class, entries.get(0).key());
+ assertEquals("o", qName.prefix());
+ assertEquals("a", qName.name());
+ assertEquals(2, entries.get(0).attributes().size());
+ assertEquals("id", assertInstanceOf(StringLit.class,
entries.get(0).attributes().get(0).key()).value());
+ assertInstanceOf(QName.class,
entries.get(0).attributes().get(1).key());
+ assertTrue(entries.get(1).dynamic());
+ assertEquals(1, entries.get(1).attributes().size());
+ assertTrue(entries.get(2).attributes().isEmpty());
+ }
+
@Test
void shouldParseArrayLiteral() {
ArrayLit arr = assertInstanceOf(ArrayLit.class, parseExpr("[1, 2,
3]"));
diff --git
a/docs/user-manual/modules/ROOT/pages/camel-4x-upgrade-guide-4_23.adoc
b/docs/user-manual/modules/ROOT/pages/camel-4x-upgrade-guide-4_23.adoc
index 5fc3dada618f..d55a3b583902 100644
--- a/docs/user-manual/modules/ROOT/pages/camel-4x-upgrade-guide-4_23.adoc
+++ b/docs/user-manual/modules/ROOT/pages/camel-4x-upgrade-guide-4_23.adoc
@@ -5037,10 +5037,12 @@ DataSonnet directly, or moved to a `.ds` file.
The conversion also follows the DataWeave semantics more closely, which
changes the result of some
scripts that were converted before. For example, a selector on a missing field
now gives `null` (as
in DataWeave) instead of failing, `default` applies to missing fields, and a
field selector on an array
-selects the field of every element. Unknown functions and variables are no
longer passed through
-unchanged, but fail the conversion. The converted scripts use the new
`camel-dataweave.libsonnet` library of
-`camel-datasonnet`, so DataSonnet scripts generated with `camel transform
dataweave` need
-`camel-datasonnet` 4.23 or newer.
+selects the field of every element. With an XML input, a selector matches an
element by its local name
+(whatever its namespace prefix), an empty element is `null`, and an element
written to another format
+than XML no longer includes the attributes, namespace declarations and
positions of the DataSonnet XML
+model. Unknown functions and variables are no longer passed through unchanged,
but fail the conversion.
+The converted scripts use the new `camel-dataweave.libsonnet` library of
`camel-datasonnet`, so
+DataSonnet scripts generated with `camel transform dataweave` need
`camel-datasonnet` 4.23 or newer.
=== camel-datasonnet - `c.sortBy` sorts arrays of numbers and strings by the
key function
