Re: Frequent "offset out of range" messages, partitions deserted by consumer

2018-06-14 Thread Lawrence Weikum
unsubscribe

Re: Frequent "offset out of range" messages, partitions deserted by consumer

2018-06-14 Thread Shantanu Deshmukh
Any help please. On Thu, Jun 14, 2018 at 2:39 PM Shantanu Deshmukh wrote: > We have a consumer application which has a single consumer group > connecting to multiple topics. We are seeing strange behaviour in consumer > logs. With these lines > > Fetch offset 1109143 is out of range for

Avro Schema Generation from Json/Csv/SQL for Kafka topics

2018-06-14 Thread Rahul Singh
I have a Kafka related challenge and hoping someone else has faced this or has some pointers. This is NOT a *schema registry* question, it is a question regarding the generation of schemas. I already know how I’m managing these schemas once they are created. I need to manage potentially

RE: Details of segment deletion

2018-06-14 Thread Simon Cooper
Thanks, that's answered all my questions! Simon -Original Message- From: Gwen Shapira Sent: 13 June 2018 02:42 To: Users Subject: Re: Details of segment deletion See below: On Mon, Jun 11, 2018 at 3:36 AM, Simon Cooper < simon.coo...@featurespace.co.uk> wrote: > Hi, > > I've ben

Re: How set log compaction policies at cluster level

2018-06-14 Thread Manikumar
Those configs are topic-level config names. To configure in server.properties, we need to use server config property names (log.cleanup.policy, log.cleaner.delete.retention.ms etc..). check the "SERVER DEFAULT PROPERTY" column in the table given in the below link

How set log compaction policies at cluster level

2018-06-14 Thread David Espinosa
Hi all, I would like to apply log compaction configuration for any topic in my kafka cluster, as default properties. These configuration properties are: - cleanup.policy - delete.retention.ms - segment.ms - min.cleanable.dirty.ratio I have tried to place them in the

Frequent "offset out of range" messages, partitions deserted by consumer

2018-06-14 Thread Shantanu Deshmukh
We have a consumer application which has a single consumer group connecting to multiple topics. We are seeing strange behaviour in consumer logs. With these lines Fetch offset 1109143 is out of range for partition otp-email-4, resetting offset Fetch offset 952168 is out of range for partition