Mark I am not quite sure what the best practices are around those questions yet. Have been doing some looking around at how other projects do it. Spark/Hadoop/Camel/etc... there are lots of examples to check into.
Thanks Joe On Sun, Jan 10, 2016 at 10:10 AM, Mark Payne <[email protected]> wrote: > I definitely love the idea, and I think it will provide NiFi a lot more > opportunity > to accomplish some of its goals. > > With this being a sub-project of NiFi, I am guessing that it would have its > own > code repository, separate from the NiFI git repository, similarly to how > nifi-site > and nifi-nar-plugin are their own repositories? How would we keep track of > JIRA's > for MiNiFi? Would it have its own JIRA, or would it perhaps just be a new > 'Component' > of the NiFi JIRA? > > Thanks > -Mark > > >> On Jan 9, 2016, at 7:29 PM, Joe Witt <[email protected]> wrote: >> >> NiFi Community, >> >> I'd like to initiate discussion around a proposal to create our first >> sub-project of NiFi. A possible name for it is "MiNiFi" a sort of >> play on Mini-NiFi. >> >> The idea is to provide a complementary data collection agent to NiFi's >> current approach of dataflow management. As noted in our ASF TLP >> resolution NiFi is to provide "an automated and durable data broker >> between systems providing interactive command and control and detailed >> chain of custody for data." MiNiFi would be consistent with that >> scope with a specific focus on the first-mile challenge so common in >> dataflow. >> >> Specific goals of MiNiFi would be to provide a small, lightweight, >> centrally managed agent that natively generates data provenance and >> seamlessly integrates with NiFi for follow-on dataflow management and >> maintenance of the chain of custody provided by the powerful data >> provenance features of NiFi. >> >> MiNiFi should be designed to operate directly on or adjacent to the >> source sensor, system, server generating the events as a resource >> sensitive tenant. There are numerous agent models in existence today >> but they do not offer the command and control or provenance that is so >> important to the philosophy and scope of NiFi. >> >> These agents would necessarily have a different interactive command >> and control model than NiFi as you'd not expect consistent behavior, >> capability, or accessibility of all instances of the agents at any >> given time. >> >> Multiple implementations of MiNiFi are envisioned including those that >> operate on the JVM and those that do not. >> >> As the discussion advances we can put together wiki pages, concept >> diagrams, and requirements to help better articulate how this might >> evolve. We should also discuss the mechanics of how this might work >> in terms of infrastructure, code repository, and more. >> >> Thanks >> Joe >
