Rarasinhoo commented on PR #11913: URL: https://github.com/apache/nifi/pull/11913#issuecomment-6926644855
Measurement on a production-shaped flow, for reviewers who want numbers. **Setup.** Synthetic flows reproducing the nesting of a real installation (1,192 versioned Process Groups, 76 top-level, nesting up to 7 levels), stored in a private GitHub repository. A sample of 4 top-level trees (73 versioned Process Groups, nesting up to 6 levels) was imported through the REST API into two local builds: `main` at 659bb60c, and the same commit with #11913, #11925 and #11926 applied. GitHub requests were counted with a proxy between NiFi and the API. Personal Access Token (5,000 requests/hour), Synchronization Interval 1 min. | | `main` | with #11913, #11925, #11926 | |---|---|---| | Import of the 4 trees | not finished after 22 min and 4,920 requests; the token limit was exhausted (403) | 444 requests, 103 s | | Synchronization after import | not reached | 79 requests in 5 min: 74 snapshot reads right after the import, then 1 branch-head check per minute | On `main`, the 4,920 requests were about 1,228 flow fetches, each costing contents + blob + branch ref + commit listing. With the changes each nested flow is fetched once, and while the branch head is unchanged a synchronization costs one request for the whole branch, independent of the number of Process Groups. The synthetic repository has only a few commits, so the commit-author lookups that `main` does per uncached commit were mostly cache hits here; on a repository with long history `main` costs more per fetch. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
