[ 
https://issues.apache.org/jira/browse/KAFKA-13447?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Philip Nee reassigned KAFKA-13447:
----------------------------------

    Assignee: Philip Nee

> Consumer should not reuse committed offset after topic recreation
> -----------------------------------------------------------------
>
>                 Key: KAFKA-13447
>                 URL: https://issues.apache.org/jira/browse/KAFKA-13447
>             Project: Kafka
>          Issue Type: Bug
>            Reporter: Jason Gustafson
>            Assignee: Philip Nee
>            Priority: Major
>              Labels: consumer, needs-kip
>
> KAFKA-12257 fixes an issue in which the consumer is unable to make progress 
> after a topic has been recreated. The problem was that the client could not 
> distinguish between stale metadata with a lower leader epoch and a recreated 
> topic with a lower leader epoch. With TopicId support in KIP-516, the client 
> is able to tell when a topic has been recreated since the new topic will have 
> a different ID. 
> However, what the patch did not fix is the potential reuse of the current 
> offset position on the recreated topic. For example, say that the consumer is 
> at offset N when the topic gets recreated. Currently, the consumer will 
> continue fetching from offset N after detecting the recreation. The most 
> likely result of this is either an offset out of range error or a log 
> truncation error, but it is also possible for the offset position to remain 
> valid on the recreated topic (say for a low-volume topic where the offsets is 
> already low, or a case where the consumer was down for a while). 
> To fix this issue completely, we need to store the topicId along with the 
> committed offset in __consumer_offsets. This would allow the consumer to 
> detect when the offset is no longer relevant for the current topic. We also 
> need to decide how to raise this case to the user. If the user has enabled 
> automatic offset reset, we can probably use that. Otherwise, we might need a 
> new exception type to signal the user that the position needs to be reset.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

Reply via email to