[
https://issues.apache.org/jira/browse/KAFKA-16414?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17847836#comment-17847836
]
Chia-Ping Tsai commented on KAFKA-16414:
----------------------------------------
{quote}
In any real use-case, setting retention.ms|bytes to 1 is not that common, but
local.retention.ms|bytes to 1 can be more frequently used (i.e. keep as minimal
data locally as possible) so my guess is that more users will find this
inconsistent behavior as tiered storage gets more adopted.
{quote}
That is a good point. Also, please consider making the configs more
straightforward. If there is no known overhead/disadvantage, I'm +1 to make
retention.bytes includes the active segment. That can bring following benefits
# consistent with retention.ms
# easy to understand
# easy to control the local segments size (with/without remote storage)
About the IT of storage, it seems the purpose is to keep "only" active segment
in the local. With the new behavior, we can set the retention.bytes equal to
segment.bytes + n to be close to the previous behavior. And then we all can
work together to deal with remaining flaky :)
> Inconsistent active segment expiration behavior between retention.ms and
> retention.bytes
> ----------------------------------------------------------------------------------------
>
> Key: KAFKA-16414
> URL: https://issues.apache.org/jira/browse/KAFKA-16414
> Project: Kafka
> Issue Type: Improvement
> Affects Versions: 3.6.1
> Reporter: Kuan Po Tseng
> Assignee: Kuan Po Tseng
> Priority: Major
>
> This is a follow up issue on KAFKA-16385.
> Currently, there's a difference between how retention.ms and retention.bytes
> handle active segment expiration:
> - retention.ms always expire active segment when max segment timestamp
> matches the condition.
> - retention.bytes only expire active segment when retention.bytes is
> configured to zero.
> The behavior should be either rotate active segments for both retention
> configurations or none at all.
> For more details, see
> https://issues.apache.org/jira/browse/KAFKA-16385?focusedCommentId=17829682&page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-17829682
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
