[jira] [Commented] (KAFKA-3879) KafkaConsumer with auto commit enabled gets stuck when killed after broker is dead
[
https://issues.apache.org/jira/browse/KAFKA-3879?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15347069#comment-15347069
]
Ashish K Singh commented on KAFKA-3879:
---
Sure [~hachikuji], linking the JIRAs.
> KafkaConsumer with auto commit enabled gets stuck when killed after broker is
> dead
> --
>
> Key: KAFKA-3879
> URL: https://issues.apache.org/jira/browse/KAFKA-3879
> Project: Kafka
> Issue Type: Bug
>Affects Versions: 0.10.0.0
>Reporter: Ashish K Singh
>Assignee: Ashish K Singh
> Fix For: 0.10.0.1
>
>
> KafkaConsumer with auto commit enabled gets stuck when killed after broker is
> dead.
> * KafkaConsumer on close tries to close coordinator.
> * Coordinator, if auto commit is enabled, tries to commit offsets
> synchronously before closing.
> * While trying to synchronously commit offsets, coordinator checks if
> coordinator is alive by sending {{GroupCoordinatorRequest}}. As brokers are
> dead, this returns {{NoAvailableBrokersException}}, which is a retriable
> exception.
> * Coordinator ready check enters into an infinite loop as it keeps retrying
> to discover group coordinator.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (KAFKA-3879) KafkaConsumer with auto commit enabled gets stuck when killed after broker is dead
[
https://issues.apache.org/jira/browse/KAFKA-3879?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15346819#comment-15346819
]
Jason Gustafson commented on KAFKA-3879:
[~singhashish] I was going to submit a KIP for this eventually, but please go
ahead if you have the time. Maybe we can resolve this issue and have a short
discussion on KAFKA-3822 on the options?
> KafkaConsumer with auto commit enabled gets stuck when killed after broker is
> dead
> --
>
> Key: KAFKA-3879
> URL: https://issues.apache.org/jira/browse/KAFKA-3879
> Project: Kafka
> Issue Type: Bug
>Affects Versions: 0.10.0.0
>Reporter: Ashish K Singh
>Assignee: Ashish K Singh
> Fix For: 0.10.0.1
>
>
> KafkaConsumer with auto commit enabled gets stuck when killed after broker is
> dead.
> * KafkaConsumer on close tries to close coordinator.
> * Coordinator, if auto commit is enabled, tries to commit offsets
> synchronously before closing.
> * While trying to synchronously commit offsets, coordinator checks if
> coordinator is alive by sending {{GroupCoordinatorRequest}}. As brokers are
> dead, this returns {{NoAvailableBrokersException}}, which is a retriable
> exception.
> * Coordinator ready check enters into an infinite loop as it keeps retrying
> to discover group coordinator.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (KAFKA-3879) KafkaConsumer with auto commit enabled gets stuck when killed after broker is dead
[
https://issues.apache.org/jira/browse/KAFKA-3879?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15345609#comment-15345609
]
Ashish K Singh commented on KAFKA-3879:
---
[~hachikuji] yeah KAFKA-3822 is pointing to the same issue. I am trying to
think of a case where someone would want to wait on broker to come up in close,
but no such case to my mind. One can always set {{max.block.ms}} to
{{Long.MAX_VALUE}} to achieve that. I am more inclined towards having something
like {{max.block.ms}}. It will probably require a KIP, are you working on
KAFKA-3822? If not, I can post a short (hopefully :)) KIP and a patch this week.
> KafkaConsumer with auto commit enabled gets stuck when killed after broker is
> dead
> --
>
> Key: KAFKA-3879
> URL: https://issues.apache.org/jira/browse/KAFKA-3879
> Project: Kafka
> Issue Type: Bug
>Affects Versions: 0.10.0.0
>Reporter: Ashish K Singh
>Assignee: Ashish K Singh
> Fix For: 0.10.0.1
>
>
> KafkaConsumer with auto commit enabled gets stuck when killed after broker is
> dead.
> * KafkaConsumer on close tries to close coordinator.
> * Coordinator, if auto commit is enabled, tries to commit offsets
> synchronously before closing.
> * While trying to synchronously commit offsets, coordinator checks if
> coordinator is alive by sending {{GroupCoordinatorRequest}}. As brokers are
> dead, this returns {{NoAvailableBrokersException}}, which is a retriable
> exception.
> * Coordinator ready check enters into an infinite loop as it keeps retrying
> to discover group coordinator.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (KAFKA-3879) KafkaConsumer with auto commit enabled gets stuck when killed after broker is dead
[
https://issues.apache.org/jira/browse/KAFKA-3879?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15345485#comment-15345485
]
Jason Gustafson commented on KAFKA-3879:
[~singhashish] This is a duplicate of KAFKA-3822, right? Yeah, we knew about
it. I think we should either solve it by adding a {{max.block.ms}}
configuration option or an overloaded close() with a timeout.
> KafkaConsumer with auto commit enabled gets stuck when killed after broker is
> dead
> --
>
> Key: KAFKA-3879
> URL: https://issues.apache.org/jira/browse/KAFKA-3879
> Project: Kafka
> Issue Type: Bug
>Affects Versions: 0.10.0.0
>Reporter: Ashish K Singh
>Assignee: Ashish K Singh
> Fix For: 0.10.0.1
>
>
> KafkaConsumer with auto commit enabled gets stuck when killed after broker is
> dead.
> * KafkaConsumer on close tries to close coordinator.
> * Coordinator, if auto commit is enabled, tries to commit offsets
> synchronously before closing.
> * While trying to synchronously commit offsets, coordinator checks if
> coordinator is alive by sending {{GroupCoordinatorRequest}}. As brokers are
> dead, this returns {{NoAvailableBrokersException}}, which is a retriable
> exception.
> * Coordinator ready check enters into an infinite loop as it keeps retrying
> to discover group coordinator.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (KAFKA-3879) KafkaConsumer with auto commit enabled gets stuck when killed after broker is dead
[
https://issues.apache.org/jira/browse/KAFKA-3879?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15340283#comment-15340283
]
Ashish K Singh commented on KAFKA-3879:
---
[~hachikuji] did you guys ever hit this? I am thinking of adding timeout to
{{ConsumerCoordinator.maybeAutoCommitOffsetsSync}}. Wondering if we need a
mechanism to make a retriable error, non-retriable after certain number of
failures.
> KafkaConsumer with auto commit enabled gets stuck when killed after broker is
> dead
> --
>
> Key: KAFKA-3879
> URL: https://issues.apache.org/jira/browse/KAFKA-3879
> Project: Kafka
> Issue Type: Bug
>Affects Versions: 0.10.0.0
>Reporter: Ashish K Singh
>Assignee: Ashish K Singh
> Fix For: 0.10.0.1
>
>
> KafkaConsumer with auto commit enabled gets stuck when killed after broker is
> dead.
> * KafkaConsumer on close tries to close coordinator.
> * Coordinator, if auto commit is enabled, tries to commit offsets
> synchronously before closing.
> * While trying to synchronously commit offsets, coordinator checks if
> coordinator is alive by sending {{GroupCoordinatorRequest}}. As brokers are
> dead, this returns {{NoAvailableBrokersException}}, which is a retriable
> exception.
> * Coordinator ready check enters into an infinite loop as it keeps retrying
> to discover group coordinator.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
