Chris Sampson created NIFI-14244:
------------------------------------
Summary: PutElasticsearchRecord outputs modified Records to the
errors relationship
Key: NIFI-14244
URL: https://issues.apache.org/jira/browse/NIFI-14244
Project: Apache NiFi
Issue Type: Bug
Affects Versions: 2.2.0
Reporter: Chris Sampson
The {{PutElasticsearchRecord}} processor allows for batches of records to be
sent as documents to Elasticsearch. Optionally, the document could consist of
an update/upsert script (written in painless) for executing by Elasticsearch.
The field containing this script can be identified using a Record Path, similar
to an RP for the doc _id (and some other options). Unlike with the _id, there
is no processor option to retain the "script" field in the Record.
While this makes sense from an Elasticsearch perspective (and for the output of
the processor, which provides the records that were sent to Elasticsearch), as
it's not likely one would want the painless script to be indexed, it causes a
problem for record output of documents that error in Elasticsearch. The
processor identifies the records relating to the erroring documents and seems
then to the error relationship but those records will not contain the "script"
field (nor _id or other fields that haven't been retained in the doc sent to
Elasticsearch). This means they cannot be retried by sending them back through
the processor.
The {{PutElasticsearchRecord}} processor should be updated to provide the
original Record content related to any documents that caused an error within
Elasticsearch.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
