Mark Payne created NIFI-9069:
--------------------------------
Summary: Allow NiFi to store dataflows using Versioned Components
instead of XML
Key: NIFI-9069
URL: https://issues.apache.org/jira/browse/NIFI-9069
Project: Apache NiFi
Issue Type: Improvement
Components: Core Framework
Reporter: Mark Payne
Assignee: Mark Payne
Currently, we have several different ways of representing flows/sub-flows. We
have the flow.xml.gz, which is used to store the node's flow. We have Templates
that store portions of flows. We have Versioned Components
(VersionedProcessGroup, etc.) that can store sub-flows (and with just a couple
of tweaks could store an entire flow easily).
This becomes a large maintenance burden. Additionally, there are a lot of
improvements that could be made by switching to the Versioned Components model
for serializing the dataflow.
Much of this is discussed in the NiFi 2.0 Proposal
(https://cwiki.apache.org/confluence/display/NIFI/NiFi+2.0+Proposed+Release+Goals)
However, moving purely from XML to JSON & Versioned Components is not without
risk. If there were a bug in the logic, it could result in loss of the
dataflow. As a result, we should build this capability such that we write out
using both the Versioned Component api (JSON) and the existing flow.xml.gz
logic.
Then, on startup, we should load from the Versioned Component model, if it
exists. In doing this, the transition from xml to json will be seamless by just
starting up - the transformation will just happen when the dataflow is written
out again. And this way, if there's an issue loading the dataflow using
Versioned Components, the .json file could be removed and the .xml file used.
We can subsequently drop support for the XML encoding.
This means that the DataFlow that gets serialized to nodes in a cluster will
also transition, and we can use much more powerful capabilities like we do when
inheriting a Versioned Flow in order to have a node rejoin a cluster even when
many things have changed.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
