Raul Roberto da Silva created NIFI-16589:
--------------------------------------------
Summary: Git Flow Registry Clients should answer synchronization
from a cache while the branch head is unchanged
Key: NIFI-16589
URL: https://issues.apache.org/jira/browse/NIFI-16589
Project: Apache NiFi
Issue Type: Improvement
Components: Flow Versioning, Extensions
Reporter: Raul Roberto da Silva
The flow synchronization periodically asks the registry client for the latest
version of every versioned process group, and for the flow itself. For
Git-based clients each question goes to the Git provider, even when nothing was
committed. With ~1,200 versioned process groups this means tens of thousands of
requests per hour on GitHub and a large spike after a restart, a common reason
for hitting rate limits (see NIFI-16359).
This change keeps, per branch, the answers to {{getLatestVersion}} and
{{getFlow}}, and checks the branch head at most every 10 seconds. While the
head is unchanged, answers come from the cache. When it moved, the client asks
the provider which paths changed between the two commits and drops only those
entries (everything, if the provider cannot tell). Writes made through the
client invalidate the path they wrote. Providers that cannot report a branch
head keep today's behaviour; GitHub implements head and compare, other clients
can follow.
No new property is added, which addresses the concern raised on PR #11708.
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)