I tried copying nifi-kafka-0-10-nar-1.3.0.nar into the lib directory of my MiNiFi installation but MiNiFi was still unable to instantiate the PublishKafka_0_10 processor. Am I including the correct one? Do I need to include any other files or make any other changes?
In our application it is orders of magnitude simpler to make changes to our data processing architecture than it is to make changes on the data source machines, as we have to make much stronger reliability guarantees for any software running on the source machines. However, our customers want the flexibility to send the data to any of an assortment of systems, including NiFi, HDFS, and Kafka, ideally without creating a NiFi cluster just to forward data. We’re looking for an application that’s configurable to send to any of those targets, and are investigating MiNiFi for that purpose. Best, Jeremy From: Aldrin Piri [mailto:[email protected]] Sent: Thursday, August 17, 2017 4:02 PM To: [email protected] Subject: Re: List of processors supported by MiNiFi 0.2.0 Hi Jeremy, Thanks for bringing the lack of a listing to our attention. We actually have documentation created to support this [1] but failed to actually publish and/or reference it anywhere. We address that with MINIFI-379 [2]. To the issue at hand, it is possible to make use of NARs from NiFi in MiNiFi Java given their shared core framework. In this case, you would be able to select one of the NARs from NiFi and place it in the lib folder of your MiNiFi instance. This is labor intensive and hopefully something we can make a more pleasant experience around with the work with Registry and some of the proposals surrounding that [3]. Might you be able to share your use case around your MiNiFi-Kafka interactions? The generated MiNiFi binary release is largely a judgement call on which extensions made the assembly by default and which did not. Would be very helpful to know if we are overlooking applications that we could look to support and assist with better. Thanks for writing in and please let us know if you have any other questions/comments. --aldrin [1] https://github.com/apache/nifi-minifi/blob/master/minifi-docs/src/main/markdown/minifi-java-agent-quick-start.md<https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_apache_nifi-2Dminifi_blob_master_minifi-2Ddocs_src_main_markdown_minifi-2Djava-2Dagent-2Dquick-2Dstart.md&d=DwMFaQ&c=96ZbZZcaMF4w0F4jpN6LZg&r=QlfoufaN_YVRqCsoMf7FH4RAik4DQ787BiN0vEsXA00&m=yYl36ttT9VCOrcSDc3J7ABzs6caIbPP6X2ARNq546UE&s=zba4BtBRzv97Yp_IPb_0Sf5fKy9L49y7z0435dGWvRI&e=> [2] https://issues.apache.org/jira/browse/MINIFI-379<https://urldefense.proofpoint.com/v2/url?u=https-3A__issues.apache.org_jira_browse_MINIFI-2D379&d=DwMFaQ&c=96ZbZZcaMF4w0F4jpN6LZg&r=QlfoufaN_YVRqCsoMf7FH4RAik4DQ787BiN0vEsXA00&m=yYl36ttT9VCOrcSDc3J7ABzs6caIbPP6X2ARNq546UE&s=v-SvEOuctSKaE97jAfZDi0zRmpFKa0xwtc7v9lS3vVY&e=> [3] http://nifi.apache.org/registry<https://urldefense.proofpoint.com/v2/url?u=http-3A__nifi.apache.org_registry&d=DwMFaQ&c=96ZbZZcaMF4w0F4jpN6LZg&r=QlfoufaN_YVRqCsoMf7FH4RAik4DQ787BiN0vEsXA00&m=yYl36ttT9VCOrcSDc3J7ABzs6caIbPP6X2ARNq546UE&s=btVkeUDK2gYldiHfvNcMCeZrfkkipNxtR4PzsMRny1U&e=> On Thu, Aug 17, 2017 at 2:52 PM, Lautman, Jeremy <[email protected]<mailto:[email protected]>> wrote: I’m trying to configure MiNiFi to send data to Kafka instead of to a NiFi remote process group and built a flow using PublishKafka_0_10. When I tried to start it I got the following error in my logs: o.apache.nifi.controller.FlowController Could not create Processor of type org.apache.nifi.processors.kafka.pubsub.PublishKafka_0_10 for ID e1141f47-b1b 8-363a-0000-000000000000; creating "Ghost" implementation org.apache.nifi.controller.exception.ProcessorInstantiationException: Unable to find bundle for coordinate default:unknown:unversioned at org.apache.nifi.controller.FlowController.instantiateProcessor(FlowController.java:1162) at org.apache.nifi.controller.FlowController.createProcessor(FlowController.java:1080) at org.apache.nifi.controller.FlowController.createProcessor(FlowController.java:1053) at org.apache.nifi.controller.StandardFlowSynchronizer.addProcessGroup(StandardFlowSynchronizer.java:1054) at org.apache.nifi.controller.StandardFlowSynchronizer.sync(StandardFlowSynchronizer.java:312) at org.apache.nifi.controller.FlowController.synchronize(FlowController.java:1544) at org.apache.nifi.persistence.StandardXMLFlowConfigurationDAO.load(StandardXMLFlowConfigurationDAO.java:84) at org.apache.nifi.controller.StandardFlowService.loadFromBytes(StandardFlowService.java:720) at org.apache.nifi.controller.StandardFlowService.load(StandardFlowService.java:533) at org.apache.nifi.minifi.MiNiFiServer.start(MiNiFiServer.java:113) at org.apache.nifi.minifi.MiNiFi.<init>(MiNiFi.java:140) at org.apache.nifi.minifi.MiNiFi.main(MiNiFi.java:239) The documentation for the C++ agent includes a list of supported processors, but I can’t find a list of supported processors for MiNiFi Java 0.2.0 anywhere. Does such a list exist? Is there an alternate way of getting MiNiFi to publish to Kafka, or can MiNiFi only send to NiFi? Best, Jeremy
