[
https://issues.apache.org/jira/browse/NIFI-10067?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17618887#comment-17618887
]
Chris Sampson commented on NIFI-10067:
--------------------------------------
My suggestion for this would probably be to:
* Update {{PutElasticsearchRecord}} to allow users to specify Record Paths to
fields within the FlowFile Record(s) containing {{_bulk}} IndexOperation
fields, e.g. {{script}}, {{retry_on_conflict}}, {{routing}}, {{version}}, etc.;
these are then removed from the Record (much like the {{id}} or {{index}} can
be) then used to create the {{IndexOperation}} and subsequent request to
Elasticsearch
** the list of {{_bulk}} IndexOperation fields will be different for different
versions of Elasticsearch and most likely change in future, so a dynamic way of
configuring this would be preferable to a hardcoded list of fields
*** e.g. look for Dynamic Properties configured on the Processor that start
with a {{BULK:}} prefix, use the remainder of the property name (sans prefix)
as the {{IndexOperation}} field name within the {{_bulk}} request and the
property's value as the Record Path to lookup the (optional) field values from
the Record(s) within the FlowFile
*** append any non-blank {{IndexOperation}} fields to the {{_bulk}} request for
the corresponding document
* Update {{PutElasticsearchJson}} to allow users to specify {{_bulk}}
IndexOperations via Dynamic Properties with the {{IndexOperation}} field being
the remainder of the property name and the value being the (Expression Language
evaluated) value
** e.g. the same {{BULK:}} prefix approach as above
** fields should be optional and only included in the {{_bulk}} request if the
value is non-blank after evaluation
Be interested in other opinions as to the approach (e.g. [~jgresock] /
[~mike.thomsen]).
> Support script in Elasticsearch update operation
> ------------------------------------------------
>
> Key: NIFI-10067
> URL: https://issues.apache.org/jira/browse/NIFI-10067
> Project: Apache NiFi
> Issue Type: Improvement
> Reporter: Stijn Caerts
> Priority: Major
> Labels: elasticsearch
>
> Right now, all update operations on Elasticsearch with the
> PutElasticsearchRecord/Json processors use [the partial document update
> approach|https://www.elastic.co/guide/en/elasticsearch/reference/current/docs-update.html#_update_part_of_a_document].
> The content of the flow file is added as the {{doc}} in the update request
> by the
> [ElasticSearchClientServiceImpl|https://github.com/apache/nifi/blob/d79b3908360265eb0d50a5ce298a00c3054cdf82/nifi-nar-bundles/nifi-elasticsearch-bundle/nifi-elasticsearch-client-service/src/main/java/org/apache/nifi/elasticsearch/ElasticSearchClientServiceImpl.java#L281].
> Can there also be provided an option to [add a script to the update
> request|https://www.elastic.co/guide/en/elasticsearch/reference/current/docs-update.html#update-api-desc]?
> Using script updates [is supported in the Elasticsearch {{_bulk}}
> API|https://www.elastic.co/guide/en/elasticsearch/reference/current/docs-bulk.html#bulk-update],
> so it should be possible to add this feature.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
