Stephen Jeffrey Hindmarch created NIFI-12703:
------------------------------------------------
Summary: LookupRecord against Elasticsearch creates MapRecord
string on 2nd lookup
Key: NIFI-12703
URL: https://issues.apache.org/jira/browse/NIFI-12703
Project: Apache NiFi
Issue Type: Bug
Components: Extensions
Affects Versions: 1.24.0
Environment: Tried in both Docker and RedHat 8. Uses Elasticsearch 8.
Reporter: Stephen Jeffrey Hindmarch
What Happens: When using 2 different record lookups in series against a single
Elasticsearch lookup service, the second lookup will add the returned data as a
MapRecord string instead of a JSON sub-record.
What I Expect: The lookup results should be written into the record as a JSON
sub-record.
How To Reproduce:
In this scenario I am trying to enrich some records from data stored in an
Elasticsearch database. This uses an ElasticsearchLookupService, and JSON
readers and writers with default settings. No schemas are used.
For example the data set might be
{noformat}
[
{"key":"two","value":2},
{"key":"four","value":4}
]{noformat}
My use case is that my data has two possible key fields to use to enrich
against one data set. I want to try enriching against one field, and if that
fails try the second set.
For example the input data might be
{noformat}
[
{"name":"bob","role":"user","hands":"two","Enrichment":{}},
{"name":"bill","role":"dog","legs":"four","Enrichment":{}}
]{noformat}
I run a lookup with a LookupRecord processor using one of the fields. For data
which matches this data is updated correctly in the record as a JSON sub-record.
For example, if the first processor uses the "hands" field for the lookup, to
result is as follows
{noformat}
[
{"name":"bob","role":"user","hands":"two",
"Enrichment":{"limbs":{"key":"two","value":2}},
"legs":null}
]{noformat}
The data that did not match is now passed through a second LookupRecord
processor, this time using the alternative key. This time the returned data is
incorrectly written into the flow as a MapRecord string. For example, if the
second processor uses the "legs" field for the lookup, the result is as follows.
{noformat}
[
{"name":"bill","role":"dog","hands":null,
"Enrichment":{"limbs":"MapRecord[{key=four, value=4}]"},
"legs":"four"}
]{noformat}
It does not matter which order the enrichment lookups are tried, it is always
the second processor which produces the MapRecord string.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
