[ 
https://issues.apache.org/jira/browse/NIFI-16474?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18146435#comment-18146435
 ] 

ASF subversion and git services commented on NIFI-16474:
--------------------------------------------------------

Commit f7e35ee2e356fc79f6d34f95c7ac619ba7b3b582 in nifi's branch 
refs/heads/main from Raul Roberto da Silva
[ https://gitbox.apache.org/repos/asf?p=nifi.git;h=f7e35ee2e35 ]

NIFI-16474 Fetch nested versioned flows once per occurrence when loading flow 
contents (#11913)

* NIFI-16474 Fetch each nested versioned flow once when loading flow contents

Loading a flow with its nested versioned flows called the recursive
population twice for every top-level child, and each nested flow was
fetched with its own descendants and then fetched again by the loop over
its children. A flow nested three levels deep was fetched 18 times, a
factor of three per level, each fetch costing Git provider API requests.

Each nested flow is now fetched without its descendants, which the loop
over its children fetches exactly once.

> Nested versioned flows are fetched from the Flow Registry an exponential 
> number of times when loading a flow's contents
> -----------------------------------------------------------------------------------------------------------------------
>
>                 Key: NIFI-16474
>                 URL: https://issues.apache.org/jira/browse/NIFI-16474
>             Project: Apache NiFi
>          Issue Type: Bug
>    Affects Versions: 2.12.0
>            Reporter: Marc Pellmann
>            Priority: Major
>          Time Spent: 40m
>  Remaining Estimate: 0h
>
> When NiFi loads a versioned flow together with its nested versioned flows, 
> each nested flow is fetched from the Flow Registry Client many times instead 
> of once. This happens through 
> StandardFlowRegistryClientNode.getFlowContents(..., fetchRemoteFlows = true). 
> The number of fetches grows exponentially with nesting depth.
>  
> This affects every operation that resolves the full snapshot, for example:
>  * GET /process-groups/\{id}/local-modifications 
> (StandardNiFiServiceFacade.getLocalModifications)
>  * putting an existing process group under version control (PUT 
> /process-groups/\{id} with versionControlInformation)
>  * importing a versioned flow into the canvas
>  
> The Git-based registry clients (GitHub, GitLab, Bitbucket, Azure DevOps) talk 
> to a hosted REST API. With them, a single request takes minutes and issues 
> hundreds of API calls.
>  
> *Cause 1: duplicated call in getFlowContents*
> {code:java}
> for (final VersionedProcessGroup child : contents.getProcessGroups()) {
>     final Map<String, VersionedParameterContext> childParameterContexts = 
> populateVersionedContentsRecursively(context, child, snapshotContainer);
>     
>     for (final Map.Entry<String, VersionedParameterContext> 
> childParameterContext : childParameterContexts.entrySet()) {
>         
> flowSnapshot.getParameterContexts().putIfAbsent(childParameterContext.getKey(),
>  childParameterContext.getValue());
>     }
>     populateVersionedContentsRecursively(context, child, snapshotContainer);  
> // <-- second, identical call
> } {code}
> populateVersionedContentsRecursively is called twice for every child. The 
> result of the second call is discarded. The line was introduced with 
> NIFI-11464.
>  
> *Cause 2: nested flows are resolved twice per level*
> For a group that has VersionedFlowCoordinates, 
> populateVersionedContentsRecursively calls fetchFlowContents(context, 
> coordinates, true). That goes through clientNode.getFlowContents(..., 
> fetchRemoteFlows = true), which already resolves every nested flow below that 
> group. The group's contents are then copied over, and the method recurses 
> into group.getProcessGroups() again. Those children still carry their 
> coordinates, so every nested flow below is fetched again.
> Combining both causes, reading the code suggests that a flow at nesting depth 
> d is fetched on the order of 2·3^(d−1) times. If the same flow occurs several 
> times in the tree, that number is multiplied again. Nested contents are also 
> added to the FlowSnapshotContainer repeatedly.
>  
> *Cause 3: every fetch also lists the file's commit history (Git-based 
> clients)*
> AbstractGitFlowRegistryClient.getFlowContents calls getLatestVersion on every 
> fetch, only to set RegisteredFlowSnapshot.isLatest. That lists the commit 
> history of the snapshot file (GitRepositoryClient.getCommits), which may span 
> several pages. Each fetch therefore costs at least two REST calls: content 
> plus history. 
>  
> *Observed*
> A flow with three levels of nesting, where one nested flow is used three 
> times, on Bitbucket Cloud. A single GET 
> /process-groups/\{id}/local-modifications ran for more than 4.5 minutes. 
> During that time it fetched individual nested flows dozens of times (e.g. 
> Nested.json 44 times, still running). One request thread produced more than 
> 1,200 Bitbucket client log entries. Earlier, putting the same group under 
> version control fetched the deepest nested flows 108 times each and took ~35 
> s.
>  
> *Impact*
>  * getLocalModifications holds the flow read lock 
> (NiFiServiceFacadeLock.proceedWithReadLock) for the whole time. Every write 
> request waits behind it, e.g. deleting the very process group. 
>  * Synchronous REST calls such as starting version control and importing a 
> flow can outlast client read timeouts. The operation still completes on the 
> server, and a client retry then fails with 409 ("process group is already 
> under version control") or creates a duplicate group.
>  * The hosted Git APIs see hundreds of calls per user action, which risks 
> rate limiting.
>  
> *Steps to reproduce*
>  #  Configure a Git-based Flow Registry Client, e.g. the Bitbucket or GitHub 
> client.
>  #  Version flows B and C. Version flow A so that it contains B, and B 
> contains C. Optionally reference B more than once inside A.
>  #  Import A into the canvas, or call GET 
> /nifi-api/process-groups/\{id}/local-modifications on it.
>  #  Enable DEBUG logging for the registry client. Each nested snapshot file 
> is requested many times, and the count grows with depth.



--
This message was sent by Atlassian Jira
(v10.3.26#10030026)

Reply via email to