[
https://issues.apache.org/jira/browse/NIFI-10192?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Joe Witt updated NIFI-10192:
----------------------------
Fix Version/s: 1.18.0
(was: 1.19.0)
> LookupRecord attempts first lookup multiple times
> -------------------------------------------------
>
> Key: NIFI-10192
> URL: https://issues.apache.org/jira/browse/NIFI-10192
> Project: Apache NiFi
> Issue Type: Bug
> Components: Core Framework
> Affects Versions: 1.16.1, 1.16.2, 1.16.3
> Reporter: Chris Norton
> Assignee: Ryan Miller
> Priority: Minor
> Fix For: 1.18.0
>
> Time Spent: 2h
> Remaining Estimate: 0h
>
> A change was implemented in
> [NIFI-9903|https://issues.apache.org/jira/browse/NIFI-9903] which results in
> *lookupService.lookup()* being called twice per record ([at least until the
> first
> match|https://github.com/apache/nifi/blob/rel/nifi-1.16.1/nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/LookupRecord.java#L350]).
> For lookup services which are idempotent (CSVRecordLookupService,
> DistributedMapCacheLookupService, PropertiesFileLookupService) making lookups
> twice won’t affect the result or have undesired side effects. However, the
> RestLookupService can make arbitrary HTTP requests for the standard HTTP
> methods (GET, POST, PUT, DELETE) and there is no guarantee that these
> requests will be idempotent. POST requests in particular are not expected to
> be idempotent and may cause undesirable behaviour if invoked multiple times
> (as in our case).
> As the name suggests, LookupRecord could be expected to be used only to
> perform lookups which are idempotent and do not have side effects. [Matt
> Burgess wrote an
> article|http://funnifi.blogspot.com/2018/08/database-sequence-lookup-with.html]
> where it seems the expected behaviour was that *lookupService.lookup()*
> would only be called once. The change in behaviour and being called twice
> would now cause IDs to be skipped.
> It was suggested by Mark Payne in a Slack discussion that lookup results
> could be cached up until the first match, which may alleviate the issues we
> are seeing.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
