[
https://issues.apache.org/jira/browse/KAFKA-21203?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Nilesh Kumar reassigned KAFKA-21203:
------------------------------------
Assignee: Nilesh Kumar
> recovery could fail when a preallocated segment hits offset overflow
> --------------------------------------------------------------------
>
> Key: KAFKA-21203
> URL: https://issues.apache.org/jira/browse/KAFKA-21203
> Project: Kafka
> Issue Type: Bug
> Reporter: Jun Rao
> Assignee: Nilesh Kumar
> Priority: Minor
>
> During log recovery, we discover offset overflow in a segment. If an overflow
> is detected, it stops the recovery and calls splitOverflowedSegment() to
> split the segment. However, since the segment hasn't been fully recovered,
> there could be untrimmed padding bytes in the segment if preallocation is
> enabled. This will cause an exception in splitOverflowedSegment() and fail
> log loading. The following call sequence shows how this can happen.
>
> {code:java}
> LogLoader.recoverLog() LogLoader.java:456
> └─ retryOnOffsetOverflow(this::recoverLog) :210 (outer) / :330
> catch
> └─ recoverSegment(segment) :400
> └─ LogSegment.recover(...) LogSegment.java:482
> for (RecordBatch batch : log.batches()) :490
> batch.ensureValid() :491
> ensureOffsetInRange(batch.lastOffset()) :492 ──► throws
>
> LogSegmentOffsetOverflowException
> ── caught by retryOnOffsetOverflow ── LogLoader.java:330
> └─ LocalLog.splitOverflowedSegment(segment,...) LocalLog.java:909
> while (position < sourceRecords.sizeInBytes()) :926 ◄──
> sizeInBytes() incl. padding
> sourceRecords.batchesFrom(position).iterator().next() :927
> ──► CorruptRecordException
> newSegment.appendFromFile(sourceRecords, position) :930
> appendChunkFromFile →
> records.batchesFrom(position).iterator() LogSegment.java:294
> {code}
>
> Note that this issue likely won't happen in practice since the offset
> overflow case typically happens on existing rolled segment which has been
> trimmed.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
