Hello all, I'd like to start a discussion about the current and future Command and Control (C2) capabilities in terms of the NiFi/MiNiFi Java codebase. The current implementation[1] is IMO not a pure reference implementation of the C2 spec [2]. It has fairly limited capability and uses the PullHttpChangeIngestor for the agent to ask for flow updates, which makes any bidirectional communication difficult. Rather than try to build out from what we currently have, I propose a new implementation (deprecating and eventually removing the old one) that is more generic, extensible, etc. This would include the following options:
- Renaming of the minifi-c2 modules and perhaps the release artifacts, updating the documentation accordingly - Creation of a top-level "c2" or "nifi-c2" module that includes an API module, assembly, NAR, implementation, services bundle, etc. as needed - Adding/deleting/changing/using properties related to C2 configuration - Changing the common runtime/loading code to look for a C2 client implementation and instantiate that at startup, configuring it with the appropriate properties and injecting the objects needed for C2 capabilities - Implementing a new C2 server that supports more of the C2 spec, and making adding more of the commands easier going forward I propose the Minimum Viable Product (MVP) C2 Client would have the following capabilities: - A "Manifest Reporter" that can either push a manifest (a document containing all the agent/instance capabilities such as available processors and their metadata, e.g.) and/or respond to a DESCRIBE MANIFEST request (see [2]) - A "Heartbeat Reporter" to send HEARTBEAT messages and also to reply with ACKNOWLEDGEMENT messages and dispatching any commands such as UPDATE - The Heartbeat Reporter is analogous to the PullHttpChangeIngestor in the old system, but is much more capable - Supports HTTPS I propose the MVP C2 Server would have the following capabilities: - Handles manifest messages (push and/or pull) aka DESCRIBE MANIFEST - Handles the HEARTBEAT / UPDATE / ACK exchange, perhaps initially using the same approach as the old C2 system (loading the flow from a file for the UPDATE command) but would be extensible/configurable - Supports HTTPS This should give us not only feature parity with the old C2 system, but also presents a framework to build out the rest of the C2 design and add other capabilities. I should note that this proposal is not specific to MiNiFi Java. It should aim to align and use the same protocol(s) as MiNiFi C++ [3], and should allow us to perhaps add C2 capabilities to NiFi itself in the future. We could track the proposal and progress at [4] as I believe that's what the page is intended for. I'd like to hear your thoughts on these proposals, and I'm happy to answer any questions or clarify as necessary. Also as this proceeds I'm happy to write up any Jira epics/cases/etc. and do the initial project structure and API setup work so others can dig in. Regards, Matt [1] https://github.com/apache/nifi/blob/main/minifi/minifi-c2/README.md [2] https://cwiki.apache.org/confluence/display/MINIFI/C2+Design [3] https://github.com/apache/nifi-minifi-cpp/blob/main/C2.md [4] https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=177048430&flashId=624428108
