Github user ijokarumawak commented on the issue:
https://github.com/apache/nifi/pull/1527
@bbende It turned out to be updating Kafka client is sufficient for 0.10.
Confirmed that ConsumeKafka_0_10 can rejoin the consumer group when it resumes
after being back-pressure engaged more than 5 minutes. For 0.9, I've added the
manual connection retaining logic using paused poll.
- For ConsumeKafka_0_10, use latest client library
Above issue has been addressed by KIP-62.
The latest Kafka consumer poll checks if the client instance is still
valid, and rejoin the group if not, before consuming messages.
- For ConsumeKafka (0.9), added manual retention logic using pause/resume
Kafka client 0.9 doesn't have background thread heartbeat, so similar
machanism is added manually.
Use Kafka pause/resume consumer API to tell Kafka server that the
client stops consuming messages but is still alive.
Another internal thread is used to perform paused poll periodically
based on the time passed since the last onTrigger(poll) is executed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
