I'd also keep in mind how partitioning of kafka topics works with
consumers, and whether or not you have a clustered nifi (most people would
for production).
If you have a topic with 1 partition and 3 nifi nodes, only one of the nifi
nodes has a consumer assigned to this partition (assuming 1
Thanks Chris, after looking more closely at the ConsumeKafka processors, I
realized that they don't accept incoming flow files.
It makes the flow more complex, and subject to potential queuing volume
concerns, but another option might be to use the Wait and Notify
processors. At a basic level,
Thanks, Chris,
This is the same problem I figured,
Is it possible to create an executeScript reading Kafka?
With regards,
Mit freundlichem Gruß,
Met vriendelijke groet,
*Ralph Vercauteren*
Technical Architect QAD Automation Solutions
Mobile NL: +31 6 5397 7230
r...@qad.com
This e-mail may
Suspect the problem there would be that the ConsumeKafka processors
(according to the documentation) don't accept incoming connections, so
you've no way of sending the FlowFile from InvokeHttp/HandleHttpRequest to
ConsumeKafka to fetch the messages (which you'd possibly then want to
Ralph,
Thanks for the reply and additional details. I may be misunderstanding,
but just for clarification, in your diagram, are you expecting NiFi to act
as the REST server? If that is the case, then you should be able to use
one of the ConsumeKafka processors to retrieve messages and return
Hi David,
Thanks for responding.
I understand that part, still have the issue to see how we get the kafka
messages from a topic.
[image: image.png]
With regards,
Mit freundlichem Gruß,
Met vriendelijke groet,
*Ralph Vercauteren*
Technical Architect QAD Automation Solutions
Mobile NL: +31 6
Hi Ralph,
If you are looking to create a REST service for receiving and processing
messages in NiFi, take a look at the ListenHTTP processor, as well as the
HandleHttpRequest and HandleHttpResponse processors. The ListenHTTP
processor supports receiving HTTP requests and sending the contents to
