[ https://issues.apache.org/jira/browse/KAFKA-14321?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17619904#comment-17619904 ]
Jun Rao commented on KAFKA-14321: --------------------------------- A potential solution is to implement the time based segment rolling based on segment creation time. > max.compaction.lag.ms is not enforced accurately > ------------------------------------------------ > > Key: KAFKA-14321 > URL: https://issues.apache.org/jira/browse/KAFKA-14321 > Project: Kafka > Issue Type: Bug > Reporter: Jun Rao > Priority: Major > > Compaction only cleans data in non-active segments. When > max.compaction.lag.ms is set, we use it to set segment.ms to force segment > rolling by time. However, the current implementation of time-based segment > roll is not precise. It only rolls a segment if the new record's timestamp > differs from the timestamp of the first record in the segment by more than > segment.ms. If we have a bunch of records appended within segment.ms and then > stop producing new records, all those records could remain in the active > segments forever, which prevents the records to be cleaned. -- This message was sent by Atlassian Jira (v8.20.10#820010)