[
https://issues.apache.org/jira/browse/KAFKA-17674?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Lianet Magrans updated KAFKA-17674:
-----------------------------------
Description:
The update fetch positions flow in new consumer attempts to :
# retrieve committed offsets
# reset positions for partitions that may still require one.
In the case that a partition with committed offsets is assigned in-between 1
and 2, the new consumer mistakenly included the newly added partition in the
LIST_OFFSETS request issued as part of 2 (so it updates the new partition
position using the partition offset and not the committed offsets)
We should ensure that we preserve the initial set of partitions we're
initializing, so that the reset excludes partitions that may require a
positions but that are not in the initial set of partitions we're running the
update positions flow for. Those partitions should get a position on the next
poll iteration (going through steps 1 and 2)
was:
The update fetch positions flow in new consumer attempts to :
# retrieve committed offsets
# reset positions for partitions that may still require one.
In the case that a partition is added in-between 1 and 2, the new consumer
mistakenly included the newly added partition in the LIST_OFFSETS request
issued as part of 2 (so it updates the new partition position using the
partition offset and not the committed offsets)
We should ensure that we preserve the initial set of partitions we're
initializing, so that the reset excludes partitions that may require a
positions but that are not in the initial set of partitions we're running the
update positions flow for. Those partitions should get a position on the next
poll iteration (going through steps 1 and 2)
> New consumer reset positions for newly added partitions before retrieving
> committed offsets
> -------------------------------------------------------------------------------------------
>
> Key: KAFKA-17674
> URL: https://issues.apache.org/jira/browse/KAFKA-17674
> Project: Kafka
> Issue Type: Bug
> Components: clients, consumer
> Reporter: Lianet Magrans
> Assignee: Lianet Magrans
> Priority: Major
> Labels: consumer-threading-refactor, kip-848-client-support
>
> The update fetch positions flow in new consumer attempts to :
> # retrieve committed offsets
> # reset positions for partitions that may still require one.
> In the case that a partition with committed offsets is assigned in-between 1
> and 2, the new consumer mistakenly included the newly added partition in the
> LIST_OFFSETS request issued as part of 2 (so it updates the new partition
> position using the partition offset and not the committed offsets)
> We should ensure that we preserve the initial set of partitions we're
> initializing, so that the reset excludes partitions that may require a
> positions but that are not in the initial set of partitions we're running the
> update positions flow for. Those partitions should get a position on the next
> poll iteration (going through steps 1 and 2)
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
