Hi All,
I am a layman to NiFi. I am exploring NiFi as a data flow engine to be
integrated with my Flink processing engine. A brief history of our approach :
We are trying to build a Streaming Data processing engine. We started off with
Flink as the sole core engine, which is responsible for collection(through
Flink Sources) as well as processing the data.
Soon we fumbled onto NiFi and the data flow world.
So far, my understanding is that the NiFi processors are poling processors and
not Pub-Sub processors. That makes me wonder, whats the impact of polling on
latency? I know I can configure my processors to tradeoff latency with
throughput, but is there a hard set limit on the latency I can achieve using
NiFi?
As I said, I am layman as yet. Perhaps my understanding is short here. Any
leads would be much appreciated.
P.S - Not diving much into Event Driven Processors. They look like something
which might clear my thoughts. But since they are marked experimental, would be
more interested in understanding the timer driven processors.
Thanks,
Chirag
