Re: Kafka java consumer processes duplicate messages

2016-08-02 Thread Amit K
gt; updated? > > > > > > On Mon, Aug 1, 2016 at 11:35 AM, Tauzell, Dave < > > > dave.tauz...@surescripts.com > > > > wrote: > > > > > > > If you kill a broker, then any uncommitted messages will be replayed. > > > > > &g

Re: Kafka java consumer processes duplicate messages

2016-08-02 Thread R Krishna
offsets may not be successfully > > updated? > > > > On Mon, Aug 1, 2016 at 11:35 AM, Tauzell, Dave < > > dave.tauz...@surescripts.com > > > wrote: > > > > > If you kill a broker, then any uncommitted messages will be replayed. > > > > > &g

Re: Kafka java consumer processes duplicate messages

2016-08-01 Thread R Krishna
messages will be replayed. > > -Dave > > From: R Krishna <krishna...@gmail.com> > Sent: Monday, August 1, 2016 1:32 PM > To: users@kafka.apache.org > Subject: Re: Kafka java consumer processes duplicate messages > > Remem

Re: Kafka java consumer processes duplicate messages

2016-08-01 Thread Tauzell, Dave
If you kill a broker, then any uncommitted messages will be replayed. -Dave From: R Krishna <krishna...@gmail.com> Sent: Monday, August 1, 2016 1:32 PM To: users@kafka.apache.org Subject: Re: Kafka java consumer processes duplicate messages Re

Re: Kafka java consumer processes duplicate messages

2016-08-01 Thread R Krishna
Remember reading about these options for higher consumer guarantees: Unclean.leader.election = false Auto.offset.commit = falseconsumer side Commit after processingsyncCommit() regularly What about your producer, does it wait until it reaches all replicas in ISR, i.e., ack=all or none?

Kafka java consumer processes duplicate messages

2016-08-01 Thread Amit K
Hi, I am kind of new to Kafka. I have set up a 3 node kafka (1 broker per machine) cluster with 3 node zookeer cluster. I am using Kafka 0.9.0.0 version. The set up works fine wherein from my single producer I am pushing a JSON string to Kafka to a topic with 3 partitions and replication factor

kafka java consumer not consuming messsages produced by remote client

2015-11-23 Thread Kudumula, Surender
consumer works but my kafka java consumer doesn't consume messages produced by remote client but when I produce messages from command line in my local machine where kafka client is subscribed to the same topic it works and consumes all the messages. bin/kafka-console-consumer.sh --zookeeper ap3

Re: Kafka java consumer

2015-08-14 Thread Abhijith Prabhakar
Thanks Ewen. Any idea when we can expect 0.8.3? On Aug 14, 2015, at 5:36 PM, Ewen Cheslack-Postava e...@confluent.io wrote: Hi Abhijith, You should be using KafkaProducer, but KafkaConsumer is not ready yet. The APIs are included in 0.8.2.1, but the implementation is not ready. Until

Re: Kafka java consumer

2015-08-14 Thread Ewen Cheslack-Postava
There's not a precise date for the release, ~1.5 or 2 months from now. On Fri, Aug 14, 2015 at 3:45 PM, Abhijith Prabhakar abhi.preda...@gmail.com wrote: Thanks Ewen. Any idea when we can expect 0.8.3? On Aug 14, 2015, at 5:36 PM, Ewen Cheslack-Postava e...@confluent.io wrote: Hi

Kafka java consumer

2015-08-14 Thread Abhijith Prabhakar
Hi All, I am newbie to Kafka and was looking to use java client implementation org.apache.kafka:kafka-clients:0.8.2.1. I was trying to write a consumer group using example given here: https://cwiki.apache.org/confluence/display/KAFKA/Consumer+Group+Example

Re: Kafka java consumer

2015-08-14 Thread Ewen Cheslack-Postava
Hi Abhijith, You should be using KafkaProducer, but KafkaConsumer is not ready yet. The APIs are included in 0.8.2.1, but the implementation is not ready. Until 0.8.3 is released, you cannot rely only on kafka-clients if you want to write a consumer. You'll need to depend on the main kafka jar