[
https://issues.apache.org/jira/browse/KAFKA-16385?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17830199#comment-17830199
]
Kuan Po Tseng commented on KAFKA-16385:
---------------------------------------
{quote}
I, as well, agree that we should include the active segment on the retention
checks; but would like to also discuss whether we should align active segment
rotation for size-based retention as well.
{quote}
Thank you [~jeqo]. Indeed there are some inconsistent behavior between
retention.ms and retention.bytes regrading the expiration of active segments.
That might need more discussion if we want to align their behavior. Before we
conclude this discussion, we should document these differences so users don't
get confused. I've address a PR in https://github.com/apache/kafka/pull/15588
and add more description mentioned in this JIRA discussion.
> Segment is rolled before segment.ms or segment.bytes breached
> -------------------------------------------------------------
>
> Key: KAFKA-16385
> URL: https://issues.apache.org/jira/browse/KAFKA-16385
> Project: Kafka
> Issue Type: Bug
> Affects Versions: 3.5.1, 3.7.0
> Reporter: Luke Chen
> Assignee: Kuan Po Tseng
> Priority: Major
>
> Steps to reproduce:
> 0. Startup a broker with `log.retention.check.interval.ms=1000` to speed up
> the test.
> 1. Creating a topic with the config: segment.ms=7days , segment.bytes=1GB,
> retention.ms=1sec .
> 2. Send a record "aaa" to the topic
> 3. Wait for 1 second
> Will this segment will rolled? I thought no.
> But what I have tested is it will roll:
> {code:java}
> [2024-03-19 15:23:13,924] INFO [LocalLog partition=t2-1,
> dir=/tmp/kafka-logs_jbod] Rolled new log segment at offset 1 in 3 ms.
> (kafka.log.LocalLog)
> [2024-03-19 15:23:13,925] INFO [ProducerStateManager partition=t2-1] Wrote
> producer snapshot at offset 1 with 1 producer ids in 1 ms.
> (org.apache.kafka.storage.internals.log.ProducerStateManager)
> [2024-03-19 15:23:13,925] INFO [UnifiedLog partition=t2-1,
> dir=/tmp/kafka-logs_jbod] Deleting segment LogSegment(baseOffset=0, size=71,
> lastModifiedTime=1710832993131, largestRecordTimestamp=1710832992125) due to
> log retention time 1000ms breach based on the largest record timestamp in the
> segment (kafka.log.UnifiedLog)
> {code}
> The segment is rolled due to log retention time 1000ms breached, which is
> unexpected.
> Tested in v3.5.1, it has the same issue.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
