[
https://issues.apache.org/jira/browse/NIFI-14712?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18004768#comment-18004768
]
Eric Secules commented on NIFI-14712:
-------------------------------------
[~tpalfy] not yet, I gotta figure out how to build the nifi docker image from
source first.
> Performance issue with flow-analysis for a process group
> --------------------------------------------------------
>
> Key: NIFI-14712
> URL: https://issues.apache.org/jira/browse/NIFI-14712
> Project: Apache NiFi
> Issue Type: Bug
> Components: Core Framework
> Affects Versions: 2.4.0
> Reporter: Eric Secules
> Priority: Major
> Attachments: Screenshot 2025-07-03 at 7.04.37 PM.png, Screenshot
> 2025-07-03 at 7.05.42 PM-1.png, get_rule_violations_performance_tuning.patch
>
>
> Background: I have a very large flow with 50000 or so processors and the
> flow-analysis endpoint is taking 20 or more seconds to complete. This is
> likely because the endpoint is traversing all the process groups recursively
> each time it is called. While User A had the flow canvas open to the root
> node, User B was unable to make configuration changes to the flow and also
> had a very hard time making updates to the flow.
> This endpoint also holds a read lock on the flow for the entire time, so it
> will prevent any writes.
> Suggestions:
> # Short circuit this endpoint and return a default success value if there
> are no violations at all.
> # -Only show detailed validation errors for the current process group and
> not descendants in the UI. For process groups within the current process
> group, show a cumulative count of violations within each process group.
> Counts would be updated lazily.-
> # -The UI only checks whether there exist violations at the current process
> group and descendants, only load the detailed validation errors if the user
> clicks on the first aid kit icon. A map of process groupID to boolean would
> be updated at the same time violations are updated and a violation on a
> process group would set the boolean to true for its ancestors up to the root.-
> ## -This can be tricky because process groups don't know their parent
> already. Also I think changing the meaning of the "first aid kit" to only
> correspond to the root PG, while it would help this performance situation, it
> would be less intuitive for the user if they're on any other process group
> than the root.-
> # Keep a read through cache of PG id to violations. Empty it whenever the
> primary map of rule violations changes. On the validatior side, add
> exponential sleep between consecutive validation passes until the flow stops
> being modified.
> Screenshots from browser dev tools attached
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
