junrao commented on code in PR #15825: URL: https://github.com/apache/kafka/pull/15825#discussion_r1586835182
########## storage/src/main/java/org/apache/kafka/storage/internals/log/LogOffsetMetadata.java: ########## @@ -28,6 +28,7 @@ public final class LogOffsetMetadata { //TODO KAFKA-14484 remove once UnifiedLog has been moved to the storage module private static final long UNIFIED_LOG_UNKNOWN_OFFSET = -1L; + public static final long REMOTE_LOG_UNKNOWN_OFFSET = -2L; Review Comment: We probably don't need this. The existing UNIFIED_LOG_UNKNOWN_OFFSET should be enough. ########## storage/src/main/java/org/apache/kafka/storage/internals/log/LogOffsetMetadata.java: ########## @@ -51,6 +57,8 @@ public LogOffsetMetadata(long messageOffset, // check if this offset is already on an older segment compared with the given offset public boolean onOlderSegment(LogOffsetMetadata that) { + if (this.segmentBaseOffset == REMOTE_LOG_UNKNOWN_OFFSET || that.segmentBaseOffset == REMOTE_LOG_UNKNOWN_OFFSET) Review Comment: We probably don't need this. If messageOffsetOnly() is true, we can just return false. ########## storage/src/main/java/org/apache/kafka/storage/internals/log/LogOffsetMetadata.java: ########## @@ -65,6 +73,8 @@ private boolean onSameSegment(LogOffsetMetadata that) { // compute the number of bytes between this offset to the given offset // if they are on the same segment and this offset precedes the given offset public int positionDiff(LogOffsetMetadata that) { + if (this.segmentBaseOffset == REMOTE_LOG_UNKNOWN_OFFSET || that.segmentBaseOffset == REMOTE_LOG_UNKNOWN_OFFSET) + return 1; Review Comment: It's a bit hacky to do this here. I was thinking of doing this in DelayedFetch. ``` if (endOffset.messageOffset != fetchOffset.messageOffset) { if (endOffset.messageOnly() || fetchOffset.messageOnly()) { accumulatedSize += 1 } else if (endOffset.onOlderSegment(fetchOffset)) { // Case F, this can happen when the new fetch operation is on a truncated leader ... } ``` ########## core/src/main/scala/kafka/log/UnifiedLog.scala: ########## @@ -1428,7 +1428,11 @@ class UnifiedLog(@volatile var logStartOffset: Long, */ private def convertToOffsetMetadataOrThrow(offset: Long): LogOffsetMetadata = { checkLogStartOffset(offset) - localLog.convertToOffsetMetadataOrThrow(offset) + if (remoteLogEnabled() && offset < localLogStartOffset()) { + new LogOffsetMetadata(offset, LogOffsetMetadata.REMOTE_LOG_UNKNOWN_OFFSET) + } else { + localLog.convertToOffsetMetadataOrThrow(offset) Review Comment: I was thinking that we could change localLog.convertToOffsetMetadataOrThrow() such that if read() throws an exception, it just returns a message-only LogOffsetMetadata. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: jira-unsubscr...@kafka.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org