jiafu1115 commented on code in PR #20913:
URL: https://github.com/apache/kafka/pull/20913#discussion_r3259505787
##########
storage/src/main/java/org/apache/kafka/server/log/remote/storage/RemoteLogManager.java:
##########
@@ -937,6 +946,68 @@ List<EnrichedLogSegment> candidateLogSegments(UnifiedLog
log, Long fromOffset, L
return candidateLogSegments;
}
+ private boolean delayCopy(LogConfig logConfig, LogSegment previousSeg,
long currentTimeMs, long totalLogSize, long cumulativeSize) {
+ if (logConfig == null) {
+ return false;
+ }
+
+ long copyLagMs = logConfig.remoteCopyLagMs();
+ long copyLagBytes = logConfig.remoteCopyLagBytes();
+ if (logger.isTraceEnabled()) {
+ logger.trace("delayCopy check for segment {}: copyLagMs={},
copyLagBytes={}, currentTimeMs={}, totalLogSize={}, cumulativeSize={},
sizeLagBytes={}",
+ previousSeg, copyLagMs, copyLagBytes, currentTimeMs,
totalLogSize, cumulativeSize, totalLogSize - cumulativeSize);
+ }
+
+ if (copyLagMs == 0 || copyLagBytes == 0) {
+ return false;
+ }
+
+ boolean needCheckCopyLagMs = copyLagMs > 0;
+ boolean needCheckCopyLagBytes = copyLagBytes > 0;
+
+ // When no lag delay is enabled, upload immediately.
+ if (!needCheckCopyLagMs && !needCheckCopyLagBytes) {
Review Comment:
@kamalcph
thanks for your review. I already split the test to new dedicated test class
and refactor the code according to part of suggestions. All tests passed except
flaky tests.
For this question. In your code approach, if the effective localRetentionMs
and localRetentionSize both are -1
then effectiveCopyLagBytes and effectiveCopyLagMs both are Long.MAX_VALUE
then the upload won't be done.
it mean if one topic which want to save forever in local (not sure if it is
the real existed case). it won't be upload any segment with this configure.
right?
in my code approach. it will be upload at once. So that is the only tiny
difference
I add the missed test case. Maybe the only difference about result between
our code is whether this new added test case can passed or not.
https://github.com/apache/kafka/pull/20913/changes#diff-a1977b0f066b04154cf4471e9c949ada43119517a345ea5924685f6d34cf1bb2R340
another tiny difference is little performance instead of result:
my code approach:
if one effective configure is MAX value (You use max value. I keep using -1
as the effective value). I don't check the configure.
your code approach:
if one is MAX value. you still check it and get the result is false. Take
time configure as example: Follow check already been checked.
shouldUploadNow = segmentAgeMs >= effectiveCopyLagMs;
WDTY? Thanks
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
