[
https://issues.apache.org/jira/browse/NIFI-16474?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18146435#comment-18146435
]
ASF subversion and git services commented on NIFI-16474:
--------------------------------------------------------
Commit f7e35ee2e356fc79f6d34f95c7ac619ba7b3b582 in nifi's branch
refs/heads/main from Raul Roberto da Silva
[ https://gitbox.apache.org/repos/asf?p=nifi.git;h=f7e35ee2e35 ]
NIFI-16474 Fetch nested versioned flows once per occurrence when loading flow
contents (#11913)
* NIFI-16474 Fetch each nested versioned flow once when loading flow contents
Loading a flow with its nested versioned flows called the recursive
population twice for every top-level child, and each nested flow was
fetched with its own descendants and then fetched again by the loop over
its children. A flow nested three levels deep was fetched 18 times, a
factor of three per level, each fetch costing Git provider API requests.
Each nested flow is now fetched without its descendants, which the loop
over its children fetches exactly once.
> Nested versioned flows are fetched from the Flow Registry an exponential
> number of times when loading a flow's contents
> -----------------------------------------------------------------------------------------------------------------------
>
> Key: NIFI-16474
> URL: https://issues.apache.org/jira/browse/NIFI-16474
> Project: Apache NiFi
> Issue Type: Bug
> Affects Versions: 2.12.0
> Reporter: Marc Pellmann
> Priority: Major
> Time Spent: 40m
> Remaining Estimate: 0h
>
> When NiFi loads a versioned flow together with its nested versioned flows,
> each nested flow is fetched from the Flow Registry Client many times instead
> of once. This happens through
> StandardFlowRegistryClientNode.getFlowContents(..., fetchRemoteFlows = true).
> The number of fetches grows exponentially with nesting depth.
>
> This affects every operation that resolves the full snapshot, for example:
> * GET /process-groups/\{id}/local-modifications
> (StandardNiFiServiceFacade.getLocalModifications)
> * putting an existing process group under version control (PUT
> /process-groups/\{id} with versionControlInformation)
> * importing a versioned flow into the canvas
>
> The Git-based registry clients (GitHub, GitLab, Bitbucket, Azure DevOps) talk
> to a hosted REST API. With them, a single request takes minutes and issues
> hundreds of API calls.
>
> *Cause 1: duplicated call in getFlowContents*
> {code:java}
> for (final VersionedProcessGroup child : contents.getProcessGroups()) {
> final Map<String, VersionedParameterContext> childParameterContexts =
> populateVersionedContentsRecursively(context, child, snapshotContainer);
>
> for (final Map.Entry<String, VersionedParameterContext>
> childParameterContext : childParameterContexts.entrySet()) {
>
> flowSnapshot.getParameterContexts().putIfAbsent(childParameterContext.getKey(),
> childParameterContext.getValue());
> }
> populateVersionedContentsRecursively(context, child, snapshotContainer);
> // <-- second, identical call
> } {code}
> populateVersionedContentsRecursively is called twice for every child. The
> result of the second call is discarded. The line was introduced with
> NIFI-11464.
>
> *Cause 2: nested flows are resolved twice per level*
> For a group that has VersionedFlowCoordinates,
> populateVersionedContentsRecursively calls fetchFlowContents(context,
> coordinates, true). That goes through clientNode.getFlowContents(...,
> fetchRemoteFlows = true), which already resolves every nested flow below that
> group. The group's contents are then copied over, and the method recurses
> into group.getProcessGroups() again. Those children still carry their
> coordinates, so every nested flow below is fetched again.
> Combining both causes, reading the code suggests that a flow at nesting depth
> d is fetched on the order of 2·3^(d−1) times. If the same flow occurs several
> times in the tree, that number is multiplied again. Nested contents are also
> added to the FlowSnapshotContainer repeatedly.
>
> *Cause 3: every fetch also lists the file's commit history (Git-based
> clients)*
> AbstractGitFlowRegistryClient.getFlowContents calls getLatestVersion on every
> fetch, only to set RegisteredFlowSnapshot.isLatest. That lists the commit
> history of the snapshot file (GitRepositoryClient.getCommits), which may span
> several pages. Each fetch therefore costs at least two REST calls: content
> plus history.
>
> *Observed*
> A flow with three levels of nesting, where one nested flow is used three
> times, on Bitbucket Cloud. A single GET
> /process-groups/\{id}/local-modifications ran for more than 4.5 minutes.
> During that time it fetched individual nested flows dozens of times (e.g.
> Nested.json 44 times, still running). One request thread produced more than
> 1,200 Bitbucket client log entries. Earlier, putting the same group under
> version control fetched the deepest nested flows 108 times each and took ~35
> s.
>
> *Impact*
> * getLocalModifications holds the flow read lock
> (NiFiServiceFacadeLock.proceedWithReadLock) for the whole time. Every write
> request waits behind it, e.g. deleting the very process group.
> * Synchronous REST calls such as starting version control and importing a
> flow can outlast client read timeouts. The operation still completes on the
> server, and a client retry then fails with 409 ("process group is already
> under version control") or creates a duplicate group.
> * The hosted Git APIs see hundreds of calls per user action, which risks
> rate limiting.
>
> *Steps to reproduce*
> # Configure a Git-based Flow Registry Client, e.g. the Bitbucket or GitHub
> client.
> # Version flows B and C. Version flow A so that it contains B, and B
> contains C. Optionally reference B more than once inside A.
> # Import A into the canvas, or call GET
> /nifi-api/process-groups/\{id}/local-modifications on it.
> # Enable DEBUG logging for the registry client. Each nested snapshot file
> is requested many times, and the count grows with depth.
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)