jax-shopify opened a new pull request, #23650: URL: https://github.com/apache/kafka/pull/23650
When `follower.fetch.last.tiered.offset.enable` is true (KIP-1023), an empty follower can select a fetch start offset equal to the leader's log start offset. The remote range to rebuild the auxiliary state from, `[logStartOffset, fetchStartOffset)`, is then empty. `TierStateMachine.buildRemoteLogAuxState` still looked up remote segment metadata for `fetchStartOffset - 1`. That offset is below the log start offset, and its remote log segment may already be deleted, so the lookup can return empty. When it did, the follower failed with `RemoteStorageException`, retried, and stayed out of the ISR until the selected offset changed. This change handles the equal case before the earlier-epoch and remote metadata lookups: the follower truncates fully to the leader's log start offset and fetches from there, the same as for a topic without tiered storage. The existing error is kept when the remote range is not empty. JIRA: https://issues.apache.org/jira/browse/KAFKA-21190 ### Testing - `TierStateMachineTest.testStartFetchesFromLogStartOffsetWhenRemoteRangeIsEmpty` runs `TierStateMachine` with equal offsets and an empty metadata lookup. It verifies that the follower starts at the log start offset and that no remote metadata or earlier-epoch lookup happens. Without the fix, it fails with the same `RemoteStorageException` we saw in production. - `TierStateMachineTest.testStartFailsWhenRemoteMetadataIsMissingForNonEmptyRemoteRange` verifies that missing metadata is an error when the remote range is not empty. - The shared setup for the `*BuildRemoteLogAuxStateMetrics` tests in `ReplicaManagerTest` used a leader with no remote range (log start offset = local log start offset = 0). These tests expect the aux state build to be attempted and to fail, so the setup now gives the leader a non-empty remote range. - Ran `TierStateMachineTest`, `AbstractFetcherThreadTest`, `ReplicaFetcherThreadTest`, `ReplicaManagerTest`, `ReplicaAlterLogDirsThreadTest`, and the tiered storage integration tests. The existing `AbstractFetcherThreadTest` cases for this scenario (`testLastTieredOffsetWithNonZeroLSOOnNewLeader`, `testLastTieredOffsetWithSlowUploadNoLocalDeletion`) use `MockTierStateMachine`, which skips the metadata lookup, so they did not catch this. Co-Authored-By: Claude Opus 5.5 <[email protected]> -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
