[
https://issues.apache.org/jira/browse/KAFKA-9638?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17051814#comment-17051814
]
Matthias J. Sax commented on KAFKA-9638:
----------------------------------------
I don't think it would be possible to implement this – the underlying consumer
group management does not really allow this. If a thread dies, it will stop to
send consumer heartbeats and thus the broker side group coordinator will remove
the member from the group and trigger a rebalance.
Not sure if static group membership would help. You can also try to increase
corresponding timeouts, but this may have other undesired side effects.
Hence, if you really want to support anything like this, it is required to
change how consumer groups works. It's not something that can be addressed at
the Streams layer.
> Do not trigger REBALANCING when specific exceptions occur in Kafka Streams
> ---------------------------------------------------------------------------
>
> Key: KAFKA-9638
> URL: https://issues.apache.org/jira/browse/KAFKA-9638
> Project: Kafka
> Issue Type: New Feature
> Components: streams
> Reporter: Levani Kokhreidze
> Priority: Major
>
> As of now, when StreamThread encounters exception in Kafka Streams
> application, it will result in REBALANCING of all the tasks that were
> responsibility of the given thread. Problem with that is, if the exception
> was, lets say some logical exception, like NPE, REBALANCING is pretty much
> useless, cause all other threads will also die with the same NPE. This kind
> of mute rebalancing gives extra costs in terms of network traffic, IOPS, etc
> in case of large stateful applications.
> In addition, this behaviour causes global outage of the Kafka Streams
> application, instead of localized outage of the certain tasks. Would be great
> if Kafka Streams users could specify via some interface, exceptions that must
> not trigger rebalancing of the tasks. StreamThread may still die, but in this
> case, we would have isolated incident.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
