[
https://issues.apache.org/jira/browse/KAFKA-70?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13102399#comment-13102399
]
Prashanth Menon commented on KAFKA-70:
--------------------------------------
Haha, this is quite uncanny. I had the same discussion regarding log.file.size
and log.retention.size after I made the patch and concluded (I believe
incorrectly) that the retention size should always be larger than the segment
size. I see both sides of the argument but I'm not really in a position to
make a decision (due to me being rather new and lacking the knowledge of the
breadth of use cases it's being used under). From my point of view having the
retention size be larger than the log file size is the natural solution;
implementing logic to trim the log to log.retention.size but leaving the active
segment undeleted is confusing since it doesn't really live by the rule its
namesake configuration suggests. As a user, I will still see the log file
greater than the retention size and be confused (perhaps this can be solved
with a simple rename of the configuration?). Just my two cents. I'd agree
with you though, dealing with bytes rather than files is a more intuitive
way.
In any case, let me know which solution you would like to go with. As it is
right now, I made the retention size > log size rule implicit, but I can add it
into the code and throw an error or warning.
I'd definitely like to work on the per-topic retention feature request, it'll
give me a better chance to dig through the code :) My preference would be to
create a new bug for it (assign it to me?).
Cheers,
Prashanth
> Introduce retention setting that depends on space
> -------------------------------------------------
>
> Key: KAFKA-70
> URL: https://issues.apache.org/jira/browse/KAFKA-70
> Project: Kafka
> Issue Type: New Feature
> Affects Versions: 0.8
> Labels: log, roll
> Attachments: KAFKA-70.patch
>
>
> Currently there is this setting:
> log.retention.hours
> introduce:
> log.retention.size
> Semantic would be, either size is reached or time is reached, oldest messages
> will be deleted.
> This does not break back-ward compatibility and would make the system robust
> under scenarios where message size is not deterministic over time.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
