[
https://issues.apache.org/jira/browse/KAFKA-15351?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Kamal Chandraprakash updated KAFKA-15351:
-----------------------------------------
Description:
Case-1:
In the FETCH response, the leader-log-start-offset will be piggy-backed. But,
there can be a scenario:
# Leader deleted the remote log segment and updates it's log-start-offset
# Before the replica-2 update it's log-start-offset via FETCH-request, the
leadership changed to replica-2.
# There are no more eligible segments to delete from remote.
# The log-start-offset will be stale (referring to old log-start-offset but
the data was already removed from remote)
# If the consumer starts to read from the beginning of the topic, it will fail
to read.
See this comment
[https://github.com/apache/kafka/pull/13561#discussion_r1293081560] for more
details.
Case-3:
When tiered storage is enabled on the topic, and the last-standing-replica is
restarted, then the log-start-offset should be updated upto
log-start-offset-checkpoint offset.
was:
Case-1:
In the FETCH response, the leader-log-start-offset will be piggy-backed. But,
there can be a scenario:
# Leader deleted the remote log segment and updates it's log-start-offset
# Before the replica-2 update it's log-start-offset via FETCH-request, the
leadership changed to replica-2.
# There are no more eligible segments to delete from remote.
# The log-start-offset will be stale (referring to old log-start-offset but
the data was already removed from remote)
# If the consumer starts to read from the beginning of the topic, it will fail
to read.
See this comment
[https://github.com/apache/kafka/pull/13561#discussion_r1293081560] for more
details.
Case-2:
The old-leader (follower) can delete the remote log segment in the middle of
leader election. We need to update the log-start-offset metadata for this case.
Case-3:
When tiered storage is enabled on the topic, and the last-standing-replica is
restarted, then the log-start-offset should be updated upto
log-start-offset-checkpoint offset.
> Update log-start-offset after leader election for topics enabled with remote
> storage
> ------------------------------------------------------------------------------------
>
> Key: KAFKA-15351
> URL: https://issues.apache.org/jira/browse/KAFKA-15351
> Project: Kafka
> Issue Type: Sub-task
> Reporter: Kamal Chandraprakash
> Assignee: Kamal Chandraprakash
> Priority: Blocker
> Fix For: 3.6.0
>
>
> Case-1:
> In the FETCH response, the leader-log-start-offset will be piggy-backed. But,
> there can be a scenario:
> # Leader deleted the remote log segment and updates it's log-start-offset
> # Before the replica-2 update it's log-start-offset via FETCH-request, the
> leadership changed to replica-2.
> # There are no more eligible segments to delete from remote.
> # The log-start-offset will be stale (referring to old log-start-offset but
> the data was already removed from remote)
> # If the consumer starts to read from the beginning of the topic, it will
> fail to read.
> See this comment
> [https://github.com/apache/kafka/pull/13561#discussion_r1293081560] for more
> details.
> Case-3:
> When tiered storage is enabled on the topic, and the last-standing-replica is
> restarted, then the log-start-offset should be updated upto
> log-start-offset-checkpoint offset.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
