Raul Roberto da Silva created NIFI-16589:
--------------------------------------------

             Summary: Git Flow Registry Clients should answer synchronization 
from a cache while the branch head is unchanged
                 Key: NIFI-16589
                 URL: https://issues.apache.org/jira/browse/NIFI-16589
             Project: Apache NiFi
          Issue Type: Improvement
          Components: Flow Versioning, Extensions
            Reporter: Raul Roberto da Silva


The flow synchronization periodically asks the registry client for the latest 
version of every versioned process group, and for the flow itself. For 
Git-based clients each question goes to the Git provider, even when nothing was 
committed. With ~1,200 versioned process groups this means tens of thousands of 
requests per hour on GitHub and a large spike after a restart, a common reason 
for hitting rate limits (see NIFI-16359).

This change keeps, per branch, the answers to {{getLatestVersion}} and 
{{getFlow}}, and checks the branch head at most every 10 seconds. While the 
head is unchanged, answers come from the cache. When it moved, the client asks 
the provider which paths changed between the two commits and drops only those 
entries (everything, if the provider cannot tell). Writes made through the 
client invalidate the path they wrote. Providers that cannot report a branch 
head keep today's behaviour; GitHub implements head and compare, other clients 
can follow.

No new property is added, which addresses the concern raised on PR #11708.



--
This message was sent by Atlassian Jira
(v10.3.26#10030026)

Reply via email to