[
https://issues.apache.org/jira/browse/NIFI-13468?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
endzeit updated NIFI-13468:
---------------------------
Description:
NiFi version 1.25.0 / NIFI-12538 introduced a new standalone function to the
RecordPath DSL named {{{}mapOf{}}}.
As the name suggests, it allows DSL users to create more complex data
structures, namely maps, using the DSL.
However, due to restrictions in the Record data type definitions, it only
supports the creation of map structures whose values are all of the same type.
The current implementation even explicitly limits the values to be of type
{{{}String{}}}, effectively limiting the result type of {{mapOf}} to
{{Map<String, String>}} but this limit could be lifted in the future to support
{{Map<String, T>}} instead.
Due to the underlying restriction enforced by the
{{{}org.apache.nifi.serialization.record.type.MapDataType{}}}, adding support
for differently typed values seems not feasible. Thus this issue proposes the
addition of a {{recordOf}} standalone function to the RecordPath DSL, that
allows DSL users to create data structures of type
{{org.apache.nifi.serialization.record.type.RecordDataType}} as well.
{{recordOf}} has the same requirements about its arguments, namely
- there has be an even number of arguments provided
- every odd argument must resemble a String used as field name
- every even argument is used as field value
----
_An example_
As {{mapOf}} and {{recordOf}} might look similar from the outside, lets present
a behavioral difference using an example. I'll make use of the existing
{{encodeJson}} RecordPath DSL function to turn the result into JSON as it makes
the differences quite obvious.
Assume we have the following record with different types of fields.
{code:json}
{
"aLong": 9876543210,
"aDouble": 2.5,
"aString": "texty",
"anArray": [
"a",
"b",
"c"
],
"aMap": {
"anotherKey": "anotherValue",
"aKey": "aValue"
},
"aRecord": {
"aField": 2,
"anotherField": "aRecordValue"
}
}
{code}
The result of {{recordOf}} preserves those types.
{noformat}
escapeJson(recordOf('mappedLong', /aLong, 'mappedDouble', /aDouble,
'mappedString', /aString, 'mappedArray', /anArray, 'mappedMap', /aMap,
'mappedRecord', /aRecord)){noformat}
{code:json}
{
"mappedLong": 9876543210,
"mappedDouble": 2.5,
"mappedString": "texty",
"mappedArray": [
"a",
"b",
"c"
],
"mappedMap": {
"anotherKey": "anotherValue",
"aKey": "aValue"
},
"mappedRecord": {
"aField": 2,
"anotherField": "aRecordValue"
}
}
{code}
With {{{}mapOf{}}}, all types are coerced to String instead.
{noformat}
escapeJson(mapOf('mappedLong', /aLong, 'mappedDouble', /aDouble,
'mappedString', /aString, 'mappedArray', /anArray, 'mappedMap', /aMap,
'mappedRecord', /aRecord)){noformat}
{code:json}
{
"mappedLong": "9876543210",
"mappedDouble": "2.5",
"mappedString": "texty",
"mappedArray": "[a, b, c]",
"mappedMap": "{anotherKey=anotherValue, aKey=aValue}",
"mappedRecord": "MapRecord[{anotherField=aRecordValue, aField=2}]"
}
{code}
----
_Why not augment {{mapOf}} instead?_
I thought about this. Actually this was my initial idea.
However, {{mapOf}} has been introduced almost 6 months ago. Adding support for
different types of values is a backwards incompatible change. It may break
existing code that expects values being coerced into {{String}} as its done at
the moment.
Additionally, it may seems strange that a function called "mapOf" actually
creates a "Record" instead of a "Map".
was:
NiFi version 1.25.0 / NIFI-12538 introduced a new standalone function to the
RecordPath DSL named {{{}mapOf{}}}.
As the name suggests, it allows DSL users to create more complex data
structures, namely maps, using the DSL.
However, due to restrictions in the Record data type definitions, it only
supports the creation of map structures whose values are all of the same type.
The current implementation even explicitly limits the values to be of type
{{{}String{}}}, effectively limiting the result type of {{mapOf}} to
{{Map<String, String>}} but this limit could be lifted in the future to support
{{Map<String, T>}} instead.
Due to the underlying restriction enforced by the
{{{}org.apache.nifi.serialization.record.type.MapDataType{}}}, adding support
for differently typed values seems not feasible. Thus this issue proposes the
addition of a {{recordOf}} standalone function to the RecordPath DSL, that
allows DSL users to create data structures of type
{{org.apache.nifi.serialization.record.type.RecordDataType}} as well.
{{recordOf}} has the same requirements about its arguments, namely
- there has be an even number of arguments provided
- every odd argument must resemble a String used as field name
- every even argument is used as field value
----
_An example_
As {{mapOf}} and {{recordOf}} might look similar from and outside, so lets
present a behavioral difference with on an example. I'll make use of the
existing {{encodeJson}} RecordPath DSL function to turn the result into JSON as
it makes the differences quite obvious.
Assume we have the following record with different types of fields.
{code:json}
{
"aLong": 9876543210,
"aDouble": 2.5,
"aString": "texty",
"anArray": [
"a",
"b",
"c"
],
"aMap": {
"anotherKey": "anotherValue",
"aKey": "aValue"
},
"aRecord": {
"aField": 2,
"anotherField": "aRecordValue"
}
}
{code}
The result of {{recordOf}} preserves those types.
{noformat}
escapeJson(recordOf('mappedLong', /aLong, 'mappedDouble', /aDouble,
'mappedString', /aString, 'mappedArray', /anArray, 'mappedMap', /aMap,
'mappedRecord', /aRecord)){noformat}
{code:json}
{
"mappedLong": 9876543210,
"mappedDouble": 2.5,
"mappedString": "texty",
"mappedArray": [
"a",
"b",
"c"
],
"mappedMap": {
"anotherKey": "anotherValue",
"aKey": "aValue"
},
"mappedRecord": {
"aField": 2,
"anotherField": "aRecordValue"
}
}
{code}
With {{{}mapOf{}}}, all types are coerced to String instead.
{noformat}
escapeJson(mapOf('mappedLong', /aLong, 'mappedDouble', /aDouble,
'mappedString', /aString, 'mappedArray', /anArray, 'mappedMap', /aMap,
'mappedRecord', /aRecord)){noformat}
{code:json}
{
"mappedLong": "9876543210",
"mappedDouble": "2.5",
"mappedString": "texty",
"mappedArray": "[a, b, c]",
"mappedMap": "{anotherKey=anotherValue, aKey=aValue}",
"mappedRecord": "MapRecord[{anotherField=aRecordValue, aField=2}]"
}
{code}
----
_Why not augment {{mapOf}} instead?_
I thought about this. Actually this was my initial idea.
However, {{mapOf}} has been introduced almost 6 months ago. Adding support for
different types of values is a backwards incompatible change. It may break
existing code that expects values being coerced into {{String}} as its done at
the moment.
Additionally, it may seems strange that a function called "mapOf" actually
creates a "Record" instead of a "Map".
> Add RecordPath function recordOf
> --------------------------------
>
> Key: NIFI-13468
> URL: https://issues.apache.org/jira/browse/NIFI-13468
> Project: Apache NiFi
> Issue Type: Improvement
> Reporter: endzeit
> Assignee: endzeit
> Priority: Major
> Time Spent: 4h 20m
> Remaining Estimate: 0h
>
> NiFi version 1.25.0 / NIFI-12538 introduced a new standalone function to the
> RecordPath DSL named {{{}mapOf{}}}.
> As the name suggests, it allows DSL users to create more complex data
> structures, namely maps, using the DSL.
> However, due to restrictions in the Record data type definitions, it only
> supports the creation of map structures whose values are all of the same type.
> The current implementation even explicitly limits the values to be of type
> {{{}String{}}}, effectively limiting the result type of {{mapOf}} to
> {{Map<String, String>}} but this limit could be lifted in the future to
> support {{Map<String, T>}} instead.
> Due to the underlying restriction enforced by the
> {{{}org.apache.nifi.serialization.record.type.MapDataType{}}}, adding support
> for differently typed values seems not feasible. Thus this issue proposes the
> addition of a {{recordOf}} standalone function to the RecordPath DSL, that
> allows DSL users to create data structures of type
> {{org.apache.nifi.serialization.record.type.RecordDataType}} as well.
> {{recordOf}} has the same requirements about its arguments, namely
> - there has be an even number of arguments provided
> - every odd argument must resemble a String used as field name
> - every even argument is used as field value
>
> ----
> _An example_
> As {{mapOf}} and {{recordOf}} might look similar from the outside, lets
> present a behavioral difference using an example. I'll make use of the
> existing {{encodeJson}} RecordPath DSL function to turn the result into JSON
> as it makes the differences quite obvious.
> Assume we have the following record with different types of fields.
> {code:json}
> {
> "aLong": 9876543210,
> "aDouble": 2.5,
> "aString": "texty",
> "anArray": [
> "a",
> "b",
> "c"
> ],
> "aMap": {
> "anotherKey": "anotherValue",
> "aKey": "aValue"
> },
> "aRecord": {
> "aField": 2,
> "anotherField": "aRecordValue"
> }
> }
> {code}
> The result of {{recordOf}} preserves those types.
> {noformat}
> escapeJson(recordOf('mappedLong', /aLong, 'mappedDouble', /aDouble,
> 'mappedString', /aString, 'mappedArray', /anArray, 'mappedMap', /aMap,
> 'mappedRecord', /aRecord)){noformat}
> {code:json}
> {
> "mappedLong": 9876543210,
> "mappedDouble": 2.5,
> "mappedString": "texty",
> "mappedArray": [
> "a",
> "b",
> "c"
> ],
> "mappedMap": {
> "anotherKey": "anotherValue",
> "aKey": "aValue"
> },
> "mappedRecord": {
> "aField": 2,
> "anotherField": "aRecordValue"
> }
> }
> {code}
> With {{{}mapOf{}}}, all types are coerced to String instead.
> {noformat}
> escapeJson(mapOf('mappedLong', /aLong, 'mappedDouble', /aDouble,
> 'mappedString', /aString, 'mappedArray', /anArray, 'mappedMap', /aMap,
> 'mappedRecord', /aRecord)){noformat}
> {code:json}
> {
> "mappedLong": "9876543210",
> "mappedDouble": "2.5",
> "mappedString": "texty",
> "mappedArray": "[a, b, c]",
> "mappedMap": "{anotherKey=anotherValue, aKey=aValue}",
> "mappedRecord": "MapRecord[{anotherField=aRecordValue, aField=2}]"
> }
> {code}
>
>
> ----
> _Why not augment {{mapOf}} instead?_
> I thought about this. Actually this was my initial idea.
> However, {{mapOf}} has been introduced almost 6 months ago. Adding support
> for different types of values is a backwards incompatible change. It may
> break existing code that expects values being coerced into {{String}} as its
> done at the moment.
> Additionally, it may seems strange that a function called "mapOf" actually
> creates a "Record" instead of a "Map".
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
