Re: Weird behavior of topic retention - some are cleaned up too often, some are not at all

Thank you, Ran, yes, I understand about the segment size impact - however, if you look at the configuration of those topics (see below) - you will see they have exactly the same config, with the same segment.bytes setting (to 1G, default), and all other settings are also the same - except for

Re: Weird behavior of topic retention - some are cleaned up too often, some are not at all

Sorry I did not see all the info at first, what do you mean by topic getting cleaned, you have setting to check retention every 5 minutes, the data that getting "cleaned" is the older data which is 30 days old... am I missing something? בתאריך יום ג׳, 25 במאי 2021, 23:04, מאת Ran Lupovich ‏: >

Re: Weird behavior of topic retention - some are cleaned up too often, some are not at all

By the segment size you are "delete" after 1 giga bytes is full , per partition, you need to remmber the retention is done when segments closed , per partition בתאריך יום ג׳, 25 במאי 2021, 22:59, מאת Ran Lupovich ‏: > Have you checked the segment size? Did you decribe the topic >

Re: Weird behavior of topic retention - some are cleaned up too often, some are not at all

Have you checked the segment size? Did you decribe the topic configuration?maybe you created it with some settings you dont remember בתאריך יום ג׳, 25 במאי 2021, 19:51, מאת Marina Popova ‏: > > Any idea what is wrong here? I have restarted Kafka brokers a few times, > and all other Confluent

Re: Weird behavior of topic retention - some are cleaned up too often, some are not at all

Any idea what is wrong here? I have restarted Kafka brokers a few times, and all other Confluent services like KSQL - but I see exactly the same behavior - one topic gets its logs cleaned up every 5 minutes, while the other one - does not get cleaned up at all Is there anything else I

Re: [External] Monitoring Consumer

If you know the offset and partition of the message then you could use the api to determine when a consumer has acknowledged the message. From: Jason Turim Date: Tuesday, May 25, 2021 at 7:31 AM To: users@kafka.apache.org Subject: Re: [External] Monitoring Consumer One approach is to have the

Re: [External] Monitoring Consumer

One approach is to have the consumer publish notifications about messages it has processed to a new topic (or other storage mechanism). You may be able to use the admin api, but I don't think it's a standard use case. On Tue, May 25, 2021, 8:21 AM Tauzell, Dave wrote: > I don’t know about

Re: [External] Monitoring Consumer

I don’t know about monitoring when a particular message is reads but you can use something like https://github.com/linkedin/Burrow to monitor consumer lag. Basically you can see that consumer Y has not yet read X number of messages that are ready. -Dave From: Alberto Moio Date: Tuesday,

Monitoring Consumer

Hello, I have a question about monitoring. Can I know when a consumer or consumer-group reads a message in a topic? Is there an API REST or a way to do this monitoring? Thank you [image: Techedge | Inspiring Trust. Globally.] *Alberto Moio* *Data Analytics | Data