Rarasinhoo opened a new pull request, #11926:
URL: https://github.com/apache/nifi/pull/11926

   # Summary
   
   [NIFI-16589](https://issues.apache.org/jira/browse/NIFI-16589)
   
   Flow synchronization periodically asks the registry client for the latest 
version of every versioned Process Group, and for the flow itself. For 
Git-based clients each question reaches the provider even when nothing was 
committed. With about 1,200 versioned Process Groups that is tens of thousands 
of GitHub requests per hour, plus a large spike after a restart (see 
NIFI-16359).
   
   This change adds `GitBranchStateCache` to `AbstractGitFlowRegistryClient`:
   
   - Per branch, it keeps the answers to `getLatestVersion` and `getFlow`, and 
checks the branch head at most every 10 seconds.
   - While the head is unchanged, answers come from the cache. When it moved, 
the client asks the provider which paths changed between the two commits and 
drops only those entries; if the provider cannot tell, it drops the whole 
branch.
   - `registerFlow`, `deregisterFlow` and `registerFlowSnapshot` invalidate the 
path they wrote; `invalidateClient` clears everything. The version-conflict 
check in `registerFlowSnapshot` still asks the repository directly.
   - `GitRepositoryClient` gains two defaults, `getBranchHeadSha` and 
`getChangedPaths`, both empty. A client that does not implement them keeps 
today's behaviour. `GitHubRepositoryClient` implements them with the branch ref 
and the compare API (capped at 300 files, renames included).
   
   No new property is added, which addresses the concern raised on #11708.
   
   `GitBranchStateCacheTest` covers the head check interval, partial and full 
invalidation, and clients without head support. 
`AbstractGitFlowRegistryClientTest` asserts that repeated questions on an 
unchanged branch are answered from the cache, and `GitHubRepositoryClientTest` 
covers the compare mapping.
   
   # Tracking
   
   Please complete the following tracking steps prior to pull request creation.
   
   ### Issue Tracking
   
   - [x] [Apache NiFi Jira](https://issues.apache.org/jira/browse/NIFI) issue 
created
   
   ### Pull Request Tracking
   
   - [x] Pull Request title starts with Apache NiFi Jira issue number, such as 
`NIFI-00000`
   - [x] Pull Request commit message starts with Apache NiFi Jira issue number, 
as such `NIFI-00000`
   - [x] Pull request contains [commits 
signed](https://docs.github.com/en/authentication/managing-commit-signature-verification/signing-commits)
 with a registered key indicating `Verified` status
   
   ### Pull Request Formatting
   
   - [x] Pull Request based on current revision of the `main` branch
   - [x] Pull Request refers to a feature branch with one commit containing 
changes
   
   # Verification
   
   Please indicate the verification steps performed prior to pull request 
creation.
   
   ### Build
   
   - [x] Build completed using `./mvnw clean install -P contrib-check` (on the 
GitHub, GitLab, Bitbucket and Azure DevOps registry modules)
     - [x] JDK 21
     - [ ] JDK 25
   
   ### Licensing
   
   - [ ] New dependencies are compatible with the [Apache License 
2.0](https://apache.org/licenses/LICENSE-2.0) according to the [License 
Policy](https://www.apache.org/legal/resolved.html)
   - [ ] New dependencies are documented in applicable `LICENSE` and `NOTICE` 
files
   
   ### Documentation
   
   - [ ] Documentation formatting appears as expected in rendered files
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to