Rarasinhoo opened a new pull request, #11926: URL: https://github.com/apache/nifi/pull/11926
# Summary [NIFI-16589](https://issues.apache.org/jira/browse/NIFI-16589) Flow synchronization periodically asks the registry client for the latest version of every versioned Process Group, and for the flow itself. For Git-based clients each question reaches the provider even when nothing was committed. With about 1,200 versioned Process Groups that is tens of thousands of GitHub requests per hour, plus a large spike after a restart (see NIFI-16359). This change adds `GitBranchStateCache` to `AbstractGitFlowRegistryClient`: - Per branch, it keeps the answers to `getLatestVersion` and `getFlow`, and checks the branch head at most every 10 seconds. - While the head is unchanged, answers come from the cache. When it moved, the client asks the provider which paths changed between the two commits and drops only those entries; if the provider cannot tell, it drops the whole branch. - `registerFlow`, `deregisterFlow` and `registerFlowSnapshot` invalidate the path they wrote; `invalidateClient` clears everything. The version-conflict check in `registerFlowSnapshot` still asks the repository directly. - `GitRepositoryClient` gains two defaults, `getBranchHeadSha` and `getChangedPaths`, both empty. A client that does not implement them keeps today's behaviour. `GitHubRepositoryClient` implements them with the branch ref and the compare API (capped at 300 files, renames included). No new property is added, which addresses the concern raised on #11708. `GitBranchStateCacheTest` covers the head check interval, partial and full invalidation, and clients without head support. `AbstractGitFlowRegistryClientTest` asserts that repeated questions on an unchanged branch are answered from the cache, and `GitHubRepositoryClientTest` covers the compare mapping. # Tracking Please complete the following tracking steps prior to pull request creation. ### Issue Tracking - [x] [Apache NiFi Jira](https://issues.apache.org/jira/browse/NIFI) issue created ### Pull Request Tracking - [x] Pull Request title starts with Apache NiFi Jira issue number, such as `NIFI-00000` - [x] Pull Request commit message starts with Apache NiFi Jira issue number, as such `NIFI-00000` - [x] Pull request contains [commits signed](https://docs.github.com/en/authentication/managing-commit-signature-verification/signing-commits) with a registered key indicating `Verified` status ### Pull Request Formatting - [x] Pull Request based on current revision of the `main` branch - [x] Pull Request refers to a feature branch with one commit containing changes # Verification Please indicate the verification steps performed prior to pull request creation. ### Build - [x] Build completed using `./mvnw clean install -P contrib-check` (on the GitHub, GitLab, Bitbucket and Azure DevOps registry modules) - [x] JDK 21 - [ ] JDK 25 ### Licensing - [ ] New dependencies are compatible with the [Apache License 2.0](https://apache.org/licenses/LICENSE-2.0) according to the [License Policy](https://www.apache.org/legal/resolved.html) - [ ] New dependencies are documented in applicable `LICENSE` and `NOTICE` files ### Documentation - [ ] Documentation formatting appears as expected in rendered files -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
