[
https://issues.apache.org/jira/browse/KAFKA-3886?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jason Gustafson updated KAFKA-3886:
-----------------------------------
Description:
If the user calls wakeup() while a rebalance in progress, we currently lose
track of the state of that rebalance. In the worst case, this can result in an
additional unneeded rebalance when the user calls poll() again.
The other thing that can happen is that the rebalance could complete inside
another blocking call (e.g. {{commitSync()}}). There may be scenarios where
this can cause us to commit offsets outside the generation an assignment is
valid for. For example:
1. Consumer is initially assigned partition A
2. The consumer starts rebalancing, but is interrupted with a call to wakeup().
3. User calls commitSync with offsets (A, 5)
4. Before offset commit is sent, an interrupted rebalance completes and changes
the assignment to include only partition B.
5. Now we proceed with the unsafe offset commit on partition A.
In this case, we should probably ensure that it is not possible to commit
offsets after an assignment has been revoked. Other cases, such as position(),
may be handled similarly.
was:
If the user calls wakeup() while a rebalance in progress, we currently lose
track of the state of that rebalance. In the worst case, this can result in an
additional unneeded rebalance when the user calls poll() again.
The other thing that can happen is that the rebalance could complete inside
another blocking call (e.g. {{commitSync()}}). There may be scenarios where
this can cause us to commit offsets outside. For example:
1. Consumer is initially assigned partition A
2. The consumer starts rebalancing, but is interrupted with a call to wakeup().
3. User calls commitSync with offsets (A, 5)
4. Before offset commit is sent, an interrupted rebalance completes and changes
the assignment to include only partition B.
5. Now we proceed with the unsafe offset commit on partition A.
In this case, we should probably ensure that it is not possible to commit
offsets after an assignment has been revoked. Other cases, such as position(),
may be handled similarly.
> Consumer should handle wakeups while rebalancing more gracefully
> ----------------------------------------------------------------
>
> Key: KAFKA-3886
> URL: https://issues.apache.org/jira/browse/KAFKA-3886
> Project: Kafka
> Issue Type: Bug
> Components: consumer
> Affects Versions: 0.10.0.0
> Reporter: Jason Gustafson
> Assignee: Jason Gustafson
>
> If the user calls wakeup() while a rebalance in progress, we currently lose
> track of the state of that rebalance. In the worst case, this can result in
> an additional unneeded rebalance when the user calls poll() again.
> The other thing that can happen is that the rebalance could complete inside
> another blocking call (e.g. {{commitSync()}}). There may be scenarios where
> this can cause us to commit offsets outside the generation an assignment is
> valid for. For example:
> 1. Consumer is initially assigned partition A
> 2. The consumer starts rebalancing, but is interrupted with a call to
> wakeup().
> 3. User calls commitSync with offsets (A, 5)
> 4. Before offset commit is sent, an interrupted rebalance completes and
> changes the assignment to include only partition B.
> 5. Now we proceed with the unsafe offset commit on partition A.
> In this case, we should probably ensure that it is not possible to commit
> offsets after an assignment has been revoked. Other cases, such as
> position(), may be handled similarly.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
