jiafu1115 commented on code in PR #22394:
URL: https://github.com/apache/kafka/pull/22394#discussion_r3356686260
##########
clients/src/main/java/org/apache/kafka/common/config/TopicConfig.java:
##########
@@ -102,20 +102,22 @@ public class TopicConfig {
"(i.e. retention.ms/bytes).";
public static final String REMOTE_COPY_LAG_MS_CONFIG =
"remote.copy.lag.ms";
- public static final String REMOTE_COPY_LAG_MS_DOC = "Controls how long to
delay uploading segments to remote storage. " +
- "When set to 0, immediate upload without any delay check. " +
- "When set to a positive value (ms), a segment can't become
eligible for upload until the time since the latest record in the segment
reaches the value. " +
- "The value should not exceed the real local retention ms except
the latter is retained indefinitely (-1). " +
- "When set to -1, resolves to the real local retention ms as
maximum delay. " +
- "For how the real local retention time is computed, see
<code>local.retention.ms</code>.";
+ public static final String REMOTE_COPY_LAG_MS_DOC = "Controls one of the
two upload eligibility checks (time and size) for copying segments to remote
storage. " +
Review Comment:
thanks！look more better now.
##########
storage/src/main/java/org/apache/kafka/server/log/remote/storage/RemoteLogManagerConfig.java:
##########
@@ -169,22 +169,26 @@ public final class RemoteLogManagerConfig {
public static final Long DEFAULT_LOG_LOCAL_RETENTION_BYTES = -2L;
public static final String LOG_REMOTE_COPY_LAG_MS_PROP =
"log.remote.copy.lag.ms";
- public static final String LOG_REMOTE_COPY_LAG_MS_DOC = "Controls how long
to delay uploading segments to remote storage. " +
- "When set to 0, immediate upload without any delay check. " +
- "When set to a positive value (ms), a segment can't become
eligible for upload until the time since the latest record in the segment
reaches the value. " +
- "The value should not exceed the real local retention ms except
the latter is retained indefinitely (-1). " +
- "When set to -1, resolves to the real local retention ms as
maximum delay. " +
- "For how the real local retention time is computed, see
<code>log.local.retention.ms</code>.";
+ public static final String LOG_REMOTE_COPY_LAG_MS_DOC = "Controls one of
the two upload eligibility checks (time and size) for copying segments to
remote storage. " +
+ "A non-active segment is upload-eligible when either this
time-based check or <code>log.remote.copy.lag.bytes</code> is satisfied. " +
+ "When set to 0, uploads are immediately eligible regardless of lag
checks. " +
+ "When set to a positive value (ms), the segment is time-eligible
once elapsed time since its latest record reaches this value. " +
+ "When set to -1, this value is derived from effective local
retention time (<code>log.local.retention.ms</code>). " +
+ "If that effective local retention time is unlimited (-1), this
time-based check is not applied. " +
+ "A positive value should not exceed effective local retention time
unless local retention is unlimited (-1).";
+ public static final Long MAX_LOG_REMOTE_COPY_LAG_MS = -1L; // It indicates
the value depends on log.local.retention.ms
public static final Long DEFAULT_LOG_REMOTE_COPY_LAG_MS = 0L;
public static final String LOG_REMOTE_COPY_LAG_BYTES_PROP =
"log.remote.copy.lag.bytes";
- public static final String LOG_REMOTE_COPY_LAG_BYTES_DOC = "Controls
size-based delay for uploading segments to remote storage. " +
- "When set to 0, immediate upload without any delay check. " +
- "When set to a positive value (bytes), a segment can't become
eligible for upload until the total bytes of log data after the segment reach
the value. " +
- "The value should not exceed the real local retention bytes except
the latter is retained indefinitely (-1). " +
- "When set to -1, resolves to the real local retention bytes as
maximum delay. " +
- "For how the real local retention size is computed, see
<code>log.local.retention.bytes</code>.";
- public static final Long DEFAULT_LOG_REMOTE_COPY_LAG_BYTES = 0L;
+ public static final String LOG_REMOTE_COPY_LAG_BYTES_DOC = "Controls one
of the two upload eligibility checks (time and size) for copying segments to
remote storage. " +
+ "A non-active segment is upload-eligible when either this
size-based check or <code>log.remote.copy.lag.ms</code> is satisfied. " +
+ "When set to 0, uploads are immediately eligible regardless of lag
checks. " +
+ "When set to a positive value (bytes), the segment is
size-eligible once bytes of newer local log data after that segment reaches
this value. " +
+ "When set to -1, this value is derived from effective local
retention size (<code>log.local.retention.bytes</code>). " +
+ "If that effective local retention size is unlimited (-1), this
size-based check is not applied. " +
+ "A positive value should not exceed effective local retention size
unless local retention is unlimited (-1).";
+ public static final Long MAX_LOG_REMOTE_COPY_LAG_BYTES = -1L; // It
indicates the value depends on log.local.retention.bytes
+ public static final Long DEFAULT_LOG_REMOTE_COPY_LAG_BYTES =
MAX_LOG_REMOTE_COPY_LAG_BYTES;
Review Comment:
@kamalcph Others look good to me. I think we can remove this statement
` // The default value of logRemoteCopyLagBytes is -1, so remote-copy
uploads are controlled by the time-based criterion only.`
because. it isn't always true. if the effective retention is N gb. -1 need
to check. So to avoid classify too many detail. we can remove the first
statement. just keep the second on. WDTY?
Thanks
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
