jiafu1115 commented on code in PR #20913:
URL: https://github.com/apache/kafka/pull/20913#discussion_r3259505787
##########
storage/src/main/java/org/apache/kafka/server/log/remote/storage/RemoteLogManager.java:
##########
@@ -937,6 +946,68 @@ List<EnrichedLogSegment> candidateLogSegments(UnifiedLog
log, Long fromOffset, L
return candidateLogSegments;
}
+ private boolean delayCopy(LogConfig logConfig, LogSegment previousSeg,
long currentTimeMs, long totalLogSize, long cumulativeSize) {
+ if (logConfig == null) {
+ return false;
+ }
+
+ long copyLagMs = logConfig.remoteCopyLagMs();
+ long copyLagBytes = logConfig.remoteCopyLagBytes();
+ if (logger.isTraceEnabled()) {
+ logger.trace("delayCopy check for segment {}: copyLagMs={},
copyLagBytes={}, currentTimeMs={}, totalLogSize={}, cumulativeSize={},
sizeLagBytes={}",
+ previousSeg, copyLagMs, copyLagBytes, currentTimeMs,
totalLogSize, cumulativeSize, totalLogSize - cumulativeSize);
+ }
+
+ if (copyLagMs == 0 || copyLagBytes == 0) {
+ return false;
+ }
+
+ boolean needCheckCopyLagMs = copyLagMs > 0;
+ boolean needCheckCopyLagBytes = copyLagBytes > 0;
+
+ // When no lag delay is enabled, upload immediately.
+ if (!needCheckCopyLagMs && !needCheckCopyLagBytes) {
Review Comment:
@kamalcph
Thanks for your review. I already split the test to new dedicated test class
and refactor the code according to your suggestions.
For this quesition. I correct to not upload(false) for this sepecial case
due to it will be easier to be understand: it is delay forever. I also and add
new test case:
https://github.com/apache/kafka/pull/20913/changes#diff-a1977b0f066b04154cf4471e9c949ada43119517a345ea5924685f6d34cf1bb2R340
Thus we have tiny style/performance difference about this case's implement
though the result is same.
my code approach:
I keep using -1 as the effective value when local retention is unlimited and
lag is -1. (Normally. if the local retention is positive value. the -1 already
be switch to it. ) I don't check the configure for if need to upload.
your code approach:
you use MAX value as the value. you always check it and get the result is
false. Take time configure as example: Follow check already been checked.
` shouldUploadNow = segmentAgeMs >= effectiveCopyLagMs;`
At the same time. it seems all similar logics for judgement about storage
also using the retentionMs < 0 (keep -1) instead of using Max Value
```
private int deleteRetentionMsBreachedSegments() throws IOException {
long retentionMs = UnifiedLog.localRetentionMs(config(),
remoteLogEnabledAndRemoteCopyEnabled());
if (retentionMs < 0) return 0;
```
Anyway. you can help to check the latest code.
WDTY? Expect to get your comment for this question. Thanks a lot!
BTW;
I add this missed test case here.
[new test
case](https://github.com/apache/kafka/pull/20913/changes#diff-a1977b0f066b04154cf4471e9c949ada43119517a345ea5924685f6d34cf1bb2R340)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
