Mark Payne created NIFI-5265:
--------------------------------
Summary: Externalize templates from flow.xml.gz
Key: NIFI-5265
URL: https://issues.apache.org/jira/browse/NIFI-5265
Project: Apache NiFi
Issue Type: Improvement
Components: Core Framework
Reporter: Mark Payne
Assignee: Mark Payne
In the 0.x baseline, templates were stored in a configurable directory, each in
its own file. In 1.0, this was changed so that Templates would instead be
stored in the flow.xml.gz directly. This was done for a couple of reasons:
* It made it easier to copy a flow.xml.gz from one node to another if needed
and have all of the templates carried along with it
* It made it easier to include just a single byte[] when inheriting a flow and
fingerprinting it
At that point in time, NiFi had experienced far less adoption than it has now,
however, and the deployments that we envisioned were much smaller than what we
see today. Now, it is not uncommon to see a deployment with many thousand
processors and dozens or hundreds of templates. The cost of serializing these
templates for every single change to the flow is more expensive than was
realized at the time.
On a bare-metal server, with several dozen templates, I can sometimes see the
saving of the flow.xml.gz take several (10-ish) seconds. This causes the UI to
become sluggish at times and can cause cluster instability (though the save
happens in the background, it does done in a lock, which can block other
requests).
As a result, I believe we should consider reverting back to the old behavior of
saving templates into individual files. This would result in better UI
performance and better cluster stability.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
