[ https://issues.apache.org/jira/browse/KAFKA-13447?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Philip Nee reassigned KAFKA-13447: ---------------------------------- Assignee: Philip Nee > Consumer should not reuse committed offset after topic recreation > ----------------------------------------------------------------- > > Key: KAFKA-13447 > URL: https://issues.apache.org/jira/browse/KAFKA-13447 > Project: Kafka > Issue Type: Bug > Reporter: Jason Gustafson > Assignee: Philip Nee > Priority: Major > Labels: consumer, needs-kip > > KAFKA-12257 fixes an issue in which the consumer is unable to make progress > after a topic has been recreated. The problem was that the client could not > distinguish between stale metadata with a lower leader epoch and a recreated > topic with a lower leader epoch. With TopicId support in KIP-516, the client > is able to tell when a topic has been recreated since the new topic will have > a different ID. > However, what the patch did not fix is the potential reuse of the current > offset position on the recreated topic. For example, say that the consumer is > at offset N when the topic gets recreated. Currently, the consumer will > continue fetching from offset N after detecting the recreation. The most > likely result of this is either an offset out of range error or a log > truncation error, but it is also possible for the offset position to remain > valid on the recreated topic (say for a low-volume topic where the offsets is > already low, or a case where the consumer was down for a while). > To fix this issue completely, we need to store the topicId along with the > committed offset in __consumer_offsets. This would allow the consumer to > detect when the offset is no longer relevant for the current topic. We also > need to decide how to raise this case to the user. If the user has enabled > automatic offset reset, we can probably use that. Otherwise, we might need a > new exception type to signal the user that the position needs to be reset. -- This message was sent by Atlassian Jira (v8.20.10#820010)