This is an automated email from the ASF dual-hosted git repository.
mjsax pushed a commit to branch trunk
in repository https://gitbox.apache.org/repos/asf/kafka.git
The following commit(s) were added to refs/heads/trunk by this push:
new cd5ce522408 KAFKA-20664: Clarify docs on max compaction lag,
segment.ms, and segment.bytes for active segment rolling (#22489)
cd5ce522408 is described below
commit cd5ce522408d03c05726d71084ff4e632f378dbd
Author: Alan Lau <[email protected]>
AuthorDate: Fri Jun 19 19:21:25 2026 -0400
KAFKA-20664: Clarify docs on max compaction lag, segment.ms, and
segment.bytes for active segment rolling (#22489)
Improves the documentation for `segment.bytes`, `segment.ms`, and
`max.compaction.lag.ms` with respect to active segment rolling.
Reviewers: Lucy Liu <[email protected]>, Matthias J. Sax
<[email protected]>
---
.../org/apache/kafka/common/config/TopicConfig.java | 17 ++++++++++++++---
docs/design/design.md | 6 +++++-
2 files changed, 19 insertions(+), 4 deletions(-)
diff --git
a/clients/src/main/java/org/apache/kafka/common/config/TopicConfig.java
b/clients/src/main/java/org/apache/kafka/common/config/TopicConfig.java
index c3a6ce5e50b..a1d075ae051 100755
--- a/clients/src/main/java/org/apache/kafka/common/config/TopicConfig.java
+++ b/clients/src/main/java/org/apache/kafka/common/config/TopicConfig.java
@@ -31,12 +31,17 @@ public class TopicConfig {
public static final String SEGMENT_BYTES_CONFIG = "segment.bytes";
public static final String SEGMENT_BYTES_DOC = "This configuration
controls the segment file size for " +
"the log. Retention and cleaning is always done a file at a time so a
larger segment size means " +
- "fewer files but less granular control over retention.";
+ "fewer files but less granular control over retention. " +
+ "The active segment is rolled once it reaches this size.";
public static final String SEGMENT_MS_CONFIG = "segment.ms";
public static final String SEGMENT_MS_DOC = "This configuration controls
the period of time after " +
"which Kafka will force the log to roll even if the segment file isn't
full to ensure that retention " +
- "can delete or compact old data.";
+ "can delete or compact old data. " +
+ "This forces active segment rolling by time, even if the active
segment has not reached " +
+ "<code>segment.bytes</code>. For compacted topics,
<code>max.compaction.lag.ms</code> can trigger " +
+ "active segment rolling sooner: the effective time-based roll
threshold is the smaller of " +
+ "<code>segment.ms</code> and <code>max.compaction.lag.ms</code>.";
public static final String SEGMENT_JITTER_MS_CONFIG = "segment.jitter.ms";
public static final String SEGMENT_JITTER_MS_DOC = "The maximum random
jitter subtracted from the scheduled " +
@@ -151,7 +156,13 @@ public class TopicConfig {
public static final String MAX_COMPACTION_LAG_MS_CONFIG =
"max.compaction.lag.ms";
public static final String MAX_COMPACTION_LAG_MS_DOC = "The maximum time a
message will remain " +
- "ineligible for compaction in the log. Only applicable for logs that
are being compacted.";
+ "ineligible for compaction in the log. Only applicable for logs that
are being compacted. " +
+ "Because the active segment is never compacted, for compacted topics
this value also drives " +
+ "active segment rolling: the effective time-based roll threshold is
the smaller of " +
+ "<code>segment.ms</code> and <code>max.compaction.lag.ms</code>.
Active segment rolling moves " +
+ "records out of the active segment, after which
<code>max.compaction.lag.ms</code> makes them " +
+ "eligible for compaction even if
<code>min.cleanable.dirty.ratio</code> is not met. See " +
+ "<a href=\"https://kafka.apache.org/documentation/#compaction\";>log
compaction</a>.";
public static final String MIN_CLEANABLE_DIRTY_RATIO_CONFIG =
"min.cleanable.dirty.ratio";
public static final String MIN_CLEANABLE_DIRTY_RATIO_DOC = "This
configuration controls how frequently " +
diff --git a/docs/design/design.md b/docs/design/design.md
index 21b4145f704..587f985f842 100644
--- a/docs/design/design.md
+++ b/docs/design/design.md
@@ -454,7 +454,11 @@ This can be used to prevent messages newer than a minimum
message age from being
log.cleaner.max.compaction.lag.ms
```
-This can be used to prevent log with low produce rate from remaining
ineligible for compaction for an unbounded duration. If not set, logs that do
not exceed min.cleanable.dirty.ratio are not compacted. Note that this
compaction deadline is not a hard guarantee since it is still subjected to the
availability of log cleaner threads and the actual compaction time. You will
want to monitor the uncleanable-partitions-count, max-clean-time-secs and
max-compaction-delay-secs metrics.
+This can be used to prevent log with low produce rate from remaining
ineligible for compaction for an unbounded duration. If not set, logs that do
not exceed min.cleanable.dirty.ratio are not compacted.
+
+Because the active segment is never compacted (as noted above), records become
eligible for compaction only through active segment rolling. For a compacted
topic the active segment is rolled when the first of these is reached: it grows
to segment.bytes, or its age reaches the smaller of segment.ms and
max.compaction.lag.ms. So max.compaction.lag.ms governs two distinct things.
First, it triggers active segment rolling by lowering the effective time-based
roll threshold to the smaller of [...]
+
+Note that this compaction deadline is not a hard guarantee since it is still
subjected to the availability of log cleaner threads and the actual compaction
time. You will want to monitor the uncleanable-partitions-count,
max-clean-time-secs and max-compaction-delay-secs metrics.
Further cleaner configurations are described
[here](/documentation.html#brokerconfigs).
