[
https://issues.apache.org/jira/browse/NIFI-13468?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
endzeit updated NIFI-13468:
---------------------------
Description:
NiFi version 1.25.0 / NIFI-12538 introduced a new standalone function to the
RecordPath DSL named {{{}mapOf{}}}.
As the name suggests, it allows DSL users to create more complex data
structures, namely maps, using the DSL.
However, due to restrictions in the Record data type definitions, it only
supports the creation of map structures whose values are all of the same type.
The current implementation even explicitly limits the values to be of type
{{{}String{}}}, effectively limiting the result type of {{mapOf}} to
{{Map<String, String>}} but this limit could be lifted in the future to support
{{Map<String, T>}} instead.
Due to the underlying restriction enforced by the
{{{}org.apache.nifi.serialization.record.type.MapDataType{}}}, adding support
for differently typed values seems not feasible. Thus this issue proposes the
addition of a {{recordOf}} standalone function to the RecordPath DSL, that
allows DSL users to create data structures of type
{{org.apache.nifi.serialization.record.type.RecordDataType}} as well.
{{recordOf}} has the same requirements about its arguments, namely
- there has be an even number of arguments provided
- every odd argument must resemble a String used as field name
- every even argument must resemble a record path whose resulting value is
used as field value
----
_An example_
As {{mapOf}} and {{recordOf}} might look similar from and outside, so lets
present a behavioral difference with on an example. I'll make use of the
existing {{encodeJson}} RecordPath DSL function to turn the result into JSON as
it makes the differences quite obvious.
Assume we have the following record with different types of fields.
{code:json}
{
"aLong": 9876543210,
"aDouble": 2.5,
"aString": "texty",
"anArray": [
"a",
"b",
"c"
],
"aMap": {
"anotherKey": "anotherValue",
"aKey": "aValue"
},
"aRecord": {
"aField": 2,
"anotherField": "aRecordValue"
}
}
{code}
The result of {{recordOf}} preserves those types.
{code:json}
{
"mappedLong": 9876543210,
"mappedDouble": 2.5,
"mappedString": "texty",
"mappedArray": [
"a",
"b",
"c"
],
"mappedMap": {
"anotherKey": "anotherValue",
"aKey": "aValue"
},
"mappedRecord": {
"aField": 2,
"anotherField": "aRecordValue"
}
}
{code}
With {{mapOf}}, all types are coerced to String instead.
{code:json}
{
"mappedLong": "9876543210",
"mappedDouble": "2.5",
"mappedString": "texty",
"mappedArray": "[a, b, c]",
"mappedMap": "{anotherKey=anotherValue, aKey=aValue}",
"mappedRecord": "MapRecord[{anotherField=aRecordValue, aField=2}]"
}
{code}
----
_Why not augment {{mapOf}} instead?_
I thought about this. Actually this was my initial idea. However, {{mapOf}} has
been introduced almost 6 months ago. Adding support for different types of
values is a backwards incompatible change. It may break existing code that
expects values being coerced into {{String}} as its done at the moment.
was:TODO
> Add RecordPath function recordOf
> --------------------------------
>
> Key: NIFI-13468
> URL: https://issues.apache.org/jira/browse/NIFI-13468
> Project: Apache NiFi
> Issue Type: Improvement
> Reporter: endzeit
> Assignee: endzeit
> Priority: Major
>
> NiFi version 1.25.0 / NIFI-12538 introduced a new standalone function to the
> RecordPath DSL named {{{}mapOf{}}}.
> As the name suggests, it allows DSL users to create more complex data
> structures, namely maps, using the DSL.
> However, due to restrictions in the Record data type definitions, it only
> supports the creation of map structures whose values are all of the same type.
> The current implementation even explicitly limits the values to be of type
> {{{}String{}}}, effectively limiting the result type of {{mapOf}} to
> {{Map<String, String>}} but this limit could be lifted in the future to
> support {{Map<String, T>}} instead.
> Due to the underlying restriction enforced by the
> {{{}org.apache.nifi.serialization.record.type.MapDataType{}}}, adding support
> for differently typed values seems not feasible. Thus this issue proposes the
> addition of a {{recordOf}} standalone function to the RecordPath DSL, that
> allows DSL users to create data structures of type
> {{org.apache.nifi.serialization.record.type.RecordDataType}} as well.
> {{recordOf}} has the same requirements about its arguments, namely
> - there has be an even number of arguments provided
> - every odd argument must resemble a String used as field name
> - every even argument must resemble a record path whose resulting value is
> used as field value
>
> ----
> _An example_
> As {{mapOf}} and {{recordOf}} might look similar from and outside, so lets
> present a behavioral difference with on an example. I'll make use of the
> existing {{encodeJson}} RecordPath DSL function to turn the result into JSON
> as it makes the differences quite obvious.
> Assume we have the following record with different types of fields.
> {code:json}
> {
> "aLong": 9876543210,
> "aDouble": 2.5,
> "aString": "texty",
> "anArray": [
> "a",
> "b",
> "c"
> ],
> "aMap": {
> "anotherKey": "anotherValue",
> "aKey": "aValue"
> },
> "aRecord": {
> "aField": 2,
> "anotherField": "aRecordValue"
> }
> }
> {code}
> The result of {{recordOf}} preserves those types.
> {code:json}
> {
> "mappedLong": 9876543210,
> "mappedDouble": 2.5,
> "mappedString": "texty",
> "mappedArray": [
> "a",
> "b",
> "c"
> ],
> "mappedMap": {
> "anotherKey": "anotherValue",
> "aKey": "aValue"
> },
> "mappedRecord": {
> "aField": 2,
> "anotherField": "aRecordValue"
> }
> }
> {code}
> With {{mapOf}}, all types are coerced to String instead.
> {code:json}
> {
> "mappedLong": "9876543210",
> "mappedDouble": "2.5",
> "mappedString": "texty",
> "mappedArray": "[a, b, c]",
> "mappedMap": "{anotherKey=anotherValue, aKey=aValue}",
> "mappedRecord": "MapRecord[{anotherField=aRecordValue, aField=2}]"
> }
> {code}
>
> ----
> _Why not augment {{mapOf}} instead?_
> I thought about this. Actually this was my initial idea. However, {{mapOf}}
> has been introduced almost 6 months ago. Adding support for different types
> of values is a backwards incompatible change. It may break existing code that
> expects values being coerced into {{String}} as its done at the moment.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
