[
https://issues.apache.org/jira/browse/NIFI-8002?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chris Sampson updated NIFI-8002:
--------------------------------
Attachment: (was: ES_Paginated_Queries.xml)
> Create new Elasticsearch Scroll/Paginated Query processor using REST API
> ------------------------------------------------------------------------
>
> Key: NIFI-8002
> URL: https://issues.apache.org/jira/browse/NIFI-8002
> Project: Apache NiFi
> Issue Type: Improvement
> Affects Versions: 1.12.1
> Reporter: Chris Sampson
> Priority: Minor
> Attachments: ES_Paginated_Queries.json, ES_Paginated_Queries.xml
>
> Original Estimate: 5h
> Time Spent: 57h 10m
> Remaining Estimate: 0h
>
> The existing ScrollElasticsearch processor does not fully work with newer
> versions of Elasticsearch (e.g. 7.x).
> A new version of this processor should be created using the REST API
> approach, allowing for a query to be passed as input FlowFile content.
> Appropriate parameters should be set as processor properties (e.g. index,
> scroll size, scroll time) and allow for additional query string parameters to
> be set as dynamic properties (see NIFI-8001).
> The processor should support both
> [_scroll|https://www.elastic.co/guide/en/elasticsearch/reference/current/paginate-search-results.html#scroll-search-results]
> and
> [_pit/search_after|https://www.elastic.co/guide/en/elasticsearch/reference/current/paginate-search-results.html#search-after].
> While scroll may be backward compatible, _pit was introduced in ES 7 (an
> earlier version of search_before exists for <7, but suggest not trying to
> support that as part of this ticket).
> It may be possible to simply extend the existing REST API's Query processor
> to page through results with the selected method, rather than create a new
> processor.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
