rajinisivaram commented on a change in pull request #9382:
URL: https://github.com/apache/kafka/pull/9382#discussion_r504729807
##########
File path: core/src/main/scala/kafka/server/AbstractFetcherThread.scala
##########
@@ -432,14 +455,22 @@ abstract class AbstractFetcherThread(name: String,
failedPartitions.removeAll(initialFetchStates.keySet)
initialFetchStates.forKeyValue { (tp, initialFetchState) =>
- // We can skip the truncation step iff the leader epoch matches the
existing epoch
+ // For IBP 2.7 onwards, we can rely on truncation based on diverging
data returned in fetch responses.
+ // For older versions, we can skip the truncation step iff the leader
epoch matches the existing epoch
val currentState = partitionStates.stateValue(tp)
- val updatedState = if (currentState != null &&
currentState.currentLeaderEpoch == initialFetchState.leaderEpoch) {
+ val updatedState = if (initialFetchState.offset >= 0 &&
isTruncationOnFetchSupported && initialFetchState.lastFetchedEpoch.nonEmpty) {
+ if (currentState != null)
+ currentState
Review comment:
I refactored this code a bit and added a check for Fetching state. Not
sure if I have missed something though. I think we can continue to fetch
without truncating if currentState is Fetching when `lastFetchedEpoch` is
known. If we need to truncate, we will do that later when we get told about
diverging epochs. Does that make sense?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
