Re: Timestamp based log compaction

2018-08-06 Thread Jan Lukavský
Hi Rajini, thanks for pointing out, that looks like exactly what I had in mind. I wasn't able to google that. Jan On 08/06/2018 12:31 PM, Rajini Sivaram wrote: Can you take a look at KIP-280: https://cwiki.apache.org/confluence/display/KAFKA/KIP-280%3A+Enhanced+log+compaction ? On Mon,

Re: Timestamp based log compaction

2018-08-06 Thread Rajini Sivaram
Can you take a look at KIP-280: https://cwiki.apache.org/confluence/display/KAFKA/KIP-280%3A+Enhanced+log+compaction ? On Mon, Aug 6, 2018 at 10:55 AM, Jan Lukavský wrote: > Hi, > > I have a question about log compaction. LogCleaner's JavaDoc states that: > > {quote} > > A message with key K

Timestamp based log compaction

2018-08-06 Thread Jan Lukavský
Hi, I have a question about log compaction. LogCleaner's JavaDoc states that: {quote} A message with key K and offset O is obsolete if there exists a message with key K and offset O' such that O < O'. {/quote} That works fine if messages are arriving "in-order", i.e. with timestamp

[jira] [Created] (KAFKA-5533) Timestamp-based log compaction

2017-06-28 Thread Eno Thereska (JIRA)
Eno Thereska created KAFKA-5533: --- Summary: Timestamp-based log compaction Key: KAFKA-5533 URL: https://issues.apache.org/jira/browse/KAFKA-5533 Project: Kafka Issue Type: New Feature