[
https://issues.apache.org/jira/browse/NIFI-10192?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17611194#comment-17611194
]
ASF subversion and git services commented on NIFI-10192:
--------------------------------------------------------
Commit 4b691b133b231cdec04290afaf1fe433b6047717 in nifi's branch
refs/heads/main from Aerilym
[ https://gitbox.apache.org/repos/asf?p=nifi.git;h=4b691b133b ]
NIFI-10192 Caffeine cache schema request for reuse
This closes #6364
Signed-off-by: Mike Thomsen <[email protected]>
> LookupRecord attempts first lookup multiple times
> -------------------------------------------------
>
> Key: NIFI-10192
> URL: https://issues.apache.org/jira/browse/NIFI-10192
> Project: Apache NiFi
> Issue Type: Bug
> Components: Core Framework
> Affects Versions: 1.16.1, 1.16.2, 1.16.3
> Reporter: Chris Norton
> Assignee: Ryan Miller
> Priority: Minor
> Time Spent: 1h 20m
> Remaining Estimate: 0h
>
> A change was implemented in
> [NIFI-9903|https://issues.apache.org/jira/browse/NIFI-9903] which results in
> *lookupService.lookup()* being called twice per record ([at least until the
> first
> match|https://github.com/apache/nifi/blob/rel/nifi-1.16.1/nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/LookupRecord.java#L350]).
> For lookup services which are idempotent (CSVRecordLookupService,
> DistributedMapCacheLookupService, PropertiesFileLookupService) making lookups
> twice won’t affect the result or have undesired side effects. However, the
> RestLookupService can make arbitrary HTTP requests for the standard HTTP
> methods (GET, POST, PUT, DELETE) and there is no guarantee that these
> requests will be idempotent. POST requests in particular are not expected to
> be idempotent and may cause undesirable behaviour if invoked multiple times
> (as in our case).
> As the name suggests, LookupRecord could be expected to be used only to
> perform lookups which are idempotent and do not have side effects. [Matt
> Burgess wrote an
> article|http://funnifi.blogspot.com/2018/08/database-sequence-lookup-with.html]
> where it seems the expected behaviour was that *lookupService.lookup()*
> would only be called once. The change in behaviour and being called twice
> would now cause IDs to be skipped.
> It was suggested by Mark Payne in a Slack discussion that lookup results
> could be cached up until the first match, which may alleviate the issues we
> are seeing.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
