NiFi Community, I'd like to initiate discussion around a proposal to create our first sub-project of NiFi. A possible name for it is "MiNiFi" a sort of play on Mini-NiFi.
The idea is to provide a complementary data collection agent to NiFi's current approach of dataflow management. As noted in our ASF TLP resolution NiFi is to provide "an automated and durable data broker between systems providing interactive command and control and detailed chain of custody for data." MiNiFi would be consistent with that scope with a specific focus on the first-mile challenge so common in dataflow. Specific goals of MiNiFi would be to provide a small, lightweight, centrally managed agent that natively generates data provenance and seamlessly integrates with NiFi for follow-on dataflow management and maintenance of the chain of custody provided by the powerful data provenance features of NiFi. MiNiFi should be designed to operate directly on or adjacent to the source sensor, system, server generating the events as a resource sensitive tenant. There are numerous agent models in existence today but they do not offer the command and control or provenance that is so important to the philosophy and scope of NiFi. These agents would necessarily have a different interactive command and control model than NiFi as you'd not expect consistent behavior, capability, or accessibility of all instances of the agents at any given time. Multiple implementations of MiNiFi are envisioned including those that operate on the JVM and those that do not. As the discussion advances we can put together wiki pages, concept diagrams, and requirements to help better articulate how this might evolve. We should also discuss the mechanics of how this might work in terms of infrastructure, code repository, and more. Thanks Joe
