[
https://issues.apache.org/jira/browse/KAFKA-21204?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18122130#comment-18122130
]
Jun Rao commented on KAFKA-21204:
---------------------------------
[~nileshk03] : Thank you for your interest. Feel free to assign the ticket to
yourself.
{quote}
One thing to settle first: for MAX_TIMESTAMP, bounding the scan by position is
not enough. The current UnifiedLog path first picks the segment with the
globally greatest maxTimestampSoFar, then looks inside that segment. If that
segment’s max timestamp is beyond the fetchable bound, we could return -1 even
though the correct max timestamp exists in an earlier batch or segment.
{quote}
Yes, this is also an issue. Not sure how to address this efficiently.
{quote}
Also, since maxPosition is batch-granular and HW/LSO can fall mid-batch,
{quote}
Since replication is at batch level, HW is always at batch boundary. Ditto for
LSO since firstUnstableOffset() is at batch boundary.
> ListOffset may return -1 when offset is available
> -------------------------------------------------
>
> Key: KAFKA-21204
> URL: https://issues.apache.org/jira/browse/KAFKA-21204
> Project: Kafka
> Issue Type: Bug
> Reporter: Jun Rao
> Priority: Major
>
> When serving a ListOffset request, the broker eventually calls FileRecords.
> searchForTimestamp(), which calls batchesFrom(). batchesFrom() bounds the
> iteration by the end of the segment, which could include uncommitted data. In
> ReplicaManger.
> fetchOffset(), we bound the returned offset by lastFetchableOffset. However,
> this means that the returned offset is -1, instead of the actual offset. This
> issue can be more likely surfaced when MAX_TIMESTAMP is used in ListOffset
> and records are appended in increasing timestamp order.
>
> {code:java}
> if (resultHolder.lastFetchableOffset.isPresent &&
> timestampAndOffsetOpt.offset >=
> resultHolder.lastFetchableOffset.get) {
> resultHolder.maybeOffsetsError.map(e => throw e)
> } else {{code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
