Ryan LaMothe created NIFI-7631:
----------------------------------
Summary: Create a nifi.content.repository.implementation for
Apache Pulsar
Key: NIFI-7631
URL: https://issues.apache.org/jira/browse/NIFI-7631
Project: Apache NiFi
Issue Type: New Feature
Components: Core Framework
Reporter: Ryan LaMothe
I would like to begin the development of a new
nifi.content.repository.implementation for Apache Pulsar. In our modern,
cloud-based streaming message environments, we are using Apache Pulsar for all
of our persistent message/data and stream management. Apache NiFi currently
supports only local disk (non-volatile) and in-memory (volatile) content
repository implementations. This means that Apache NiFi currently performs
double duty as both a workflow management environment and a message/data
management system, as there are no remote message/data management content
repository implementations available.
The proposed new feature development would create a new content repository
implementation designed around a streaming message/data architecture, in
essence replacing the concept of a "NiFi local queue" with an "Apache Pulsar
remote queue", allowing Apache Pulsar to remotely and independently manage
messages/data on the behalf of NiFi. This would also support NiFi as a pure
workflow management environment, decoupling it from its data management
responsibilities.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
