chia7712 commented on code in PR #22873:
URL: https://github.com/apache/kafka/pull/22873#discussion_r4100427371
##########
storage/src/main/java/org/apache/kafka/storage/internals/log/UnifiedLog.java:
##########
@@ -2000,39 +2000,34 @@ public interface DeletionCondition {
}
private int deleteRetentionMsBreachedSegments() throws IOException {
- long retentionMs = UnifiedLog.localRetentionMs(config(),
remoteLogEnabledAndRemoteCopyEnabled());
+ boolean remoteLogEnabledAndRemoteCopyEnabled =
remoteLogEnabledAndRemoteCopyEnabled();
+ long retentionMs = UnifiedLog.localRetentionMs(config(),
remoteLogEnabledAndRemoteCopyEnabled);
if (retentionMs < 0) return 0;
long startMs = time().milliseconds();
DeletionCondition shouldDelete = (segment, nextSegmentOpt) -> {
- if (startMs < segment.largestTimestamp()) {
- futureTimestampLogger.warn("{} contains future timestamp(s),
making it ineligible to be deleted", segment);
+ long anchorTimestamp = segment.largestTimestamp();
+ if (startMs < anchorTimestamp) {
+ if (remoteLogEnabledAndRemoteCopyEnabled) {
+ anchorTimestamp = segment.lastModified();
Review Comment:
> occurs even when tiered storage is not used
Yes, we had considered it as well. However, the risk is different from the
remote storage case. The segments which have been uploaded can't be deleted
locally. That violates the purpose of remote storage. Also, it will block the
deletion of the following segments, potentially causing a full disk.
We can have a similar fallback on the normal path, but I did not see a
strong reason for it. I can file a JIRA to track it and wait for a good reason.
WDYT?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
