Hello.
I’m doing some experimentations on Apache Nifi to see where we can use it. One idea is to use nifi to feed a spark cluster. So I’m doing some simple test (GenerateFlowFile => spark output port and a simple word count on spark side. I was pretty unhappy with the performance out of the box, so I looked on the net and found almost nothing. So I looked at nifi.properties, and found that some of the following properties have a huge impact on how many messages / second were processed to Spark : nifi.queue.swap.threshold=20000 nifi.swap.in.period=1 sec nifi.swap.in.threads=1 nifi.swap.out.period=1 sec nifi.swap.out.threads=4 The documentation seems unclear on this point for output ports, is anyone have a pointer for me ? Thanks. Aurélien.
