[
https://issues.apache.org/jira/browse/KAFKA-3949?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15427604#comment-15427604
]
ASF GitHub Bot commented on KAFKA-3949:
---------------------------------------
GitHub user hachikuji opened a pull request:
https://github.com/apache/kafka/pull/1762
KAFKA-3949: Fix race condition when metadata update arrives during rebalance
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/hachikuji/kafka KAFKA-3949
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/kafka/pull/1762.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #1762
----
commit 2ce2e1f354740f07899d452e5f2e086cee759ebf
Author: Jason Gustafson <[email protected]>
Date: 2016-08-19T04:26:27Z
KAFKA-3949: Fix race condition when metadata update arrives during rebalance
----
> Consumer topic subscription change may be ignored if a rebalance is in
> progress
> -------------------------------------------------------------------------------
>
> Key: KAFKA-3949
> URL: https://issues.apache.org/jira/browse/KAFKA-3949
> Project: Kafka
> Issue Type: Bug
> Components: consumer
> Affects Versions: 0.9.0.1, 0.10.0.0
> Reporter: Jason Gustafson
> Assignee: Jason Gustafson
>
> The consumer's regex subscription works by matching all topics fetched from a
> metadata update against the provided pattern. When a new topic is created or
> an old topic is deleted, we update the list of subscribed topics and request
> a rebalance by setting the {{needsPartitionAssignment}} flag inside
> {{SubscriptionState}}. On the next call to {{poll()}}, the consumer will
> observe the flag and begin the rebalance by sending a JoinGroup. The problem
> is that it does not account for the fact that a rebalance could already be in
> progress at the time the metadata is updated. This causes the following
> sequence:
> 1. Rebalance begins (needsPartitionAssignment is set True)
> 2. Metadata max age expires and and update is triggered
> 3. Update returns and causes a topic subscription change
> (needsPartitionAssignment set again to True).
> 4. Rebalance completes (needsPartitionAssignment is set False)
> In this situation, we will not request a new rebalance which will prevent us
> from receiving an assignment from any topics added to the consumer's
> subscription when the metadata was updated. This state will persist until
> another event causes the group to rebalance.
> A related problem may occur if a rebalance is interrupted with the wakeup()
> API, and the user calls subscribe(topics) with a change to the subscription
> set.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
