Re: Kafka retention policy per topic not working as expected

2020-05-24 Thread Christopher Smith
It's a bit weird having the default policy for your brokers being compact, but yes, the policy for the topic overrides the broker policy. What you are seeing on with your example of the segment files makes sense to me. With a 121000ms retention policy you've actually got 400 seconds beyond

How should I measure the time spent in various stages of kafka

2020-05-24 Thread 张瑞
Hi,we have some questions about Kafka and want to ask you, hope this email does not disturb your life. We are doing some work based on kakfa, and we measured the time spent at each stage in the workflow of Kafka when producing and consuming messages. The results show that network transmission

Re: Kafka retention policy per topic not working as expected

2020-05-24 Thread Don Ky
Hello all, any thoughts on this? Just pinging if anyone out there has any pointers on where to go from here? I was wondering if it could be that the topic is actually being compacted and not a retention policy of delete? Could this be because that the topics were initially created with a compact