Jun Rao created KAFKA-17801:
-------------------------------
Summary: RemoteLogManager may compute inaccurate upperBoundOffset
for aborted txns
Key: KAFKA-17801
URL: https://issues.apache.org/jira/browse/KAFKA-17801
Project: Kafka
Issue Type: Bug
Components: core
Affects Versions: 3.6.0
Reporter: Jun Rao
In RemoteLogManager.read, we compute startPos as the following.
{code:java}
startPos = lookupPositionForOffset(remoteLogSegmentMetadata, offset);{code}
This is the position returned by the offset index. The actual position for the
first batch being read happens in the following, but startPos is not updated
accordingly.
{code:java}
firstBatch = findFirstBatch(remoteLogInputStream, offset);{code}
We then use the inaccurate startPos to create fetchDataInfo.
{code:java}
FetchDataInfo fetchDataInfo = new FetchDataInfo(
new LogOffsetMetadata(offset, remoteLogSegmentMetadata.startOffset(), startPos),
MemoryRecords.readableRecords(buffer));{code}
In addAbortedTransactions(), we use startPos to find the upperBoundOffset to
retrieve the aborted txns.
{code:java}
long upperBoundOffset = offsetIndex.fetchUpperBoundOffset(startOffsetPosition,
fetchSize)
.map(position -> position.offset).orElse(segmentMetadata.endOffset() + 1);{code}
The inaccurate startPos can lead to inaccurate upperBoundOffset, which leads to
inaccurate aborted txns returned to the consumer.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
