Peter Turcsanyi created NIFI-8085:
-------------------------------------
Summary: Use poll(Duration) in ConsumeKafka_2_x processors
Key: NIFI-8085
URL: https://issues.apache.org/jira/browse/NIFI-8085
Project: Apache NiFi
Issue Type: Improvement
Reporter: Peter Turcsanyi
Assignee: Peter Turcsanyi
The currently used {{KafkaConsumer.poll(long timeout)}} method is deprecated in
2.x. The method waits for the metadata response indefinitely until we get it
(so in case of an unavailable/stopped kafka cluster until the connection gets
recovered). It means the {{poll(long)}} acquires and holds a thread exclusively
for this period which can lead to starvation of other processors for timer
driven threads.
Use poll(Duration timeout) instead which is not simply an overloaded version of
the method but has different semantics
(https://cwiki.apache.org/confluence/display/KAFKA/KIP-266%3A+Fix+consumer+indefinite+blocking+behavior).
If can return from metadata fetch after the timeout expires and does not occupy
a thread continuously and makes the processor able to yield.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
