Re: Kafka consumer cluster setup

2016-10-19 Thread sat
s Confluent, Inc > On Oct 16, 2016, at 11:17 PM, sat <sathish.al...@gmail.com> wrote: > > Hi, > > We are planning to run Kafka servers along with Zookeeper in 3 remote > machines for our cluster setup. > > We have our application with Kafka Consumer and Publisher runnin

Kafka consumer cluster setup

2016-10-16 Thread sat
Hi, We are planning to run Kafka servers along with Zookeeper in 3 remote machines for our cluster setup. We have our application with Kafka Consumer and Publisher running in another/separate 2 machines. We will be configuring 1 or 2 out of 3 Kafka servers as bootstrap servers in Kafka

Consumer skip/ignore uncommitted message

2016-08-10 Thread sat
Hi, We have come up with Kafka consumer and we only have 1 consumer in the group. We are using Kafka 0.9. We are able to consume messages as soon as we start polling, however for some reason our consumer gets crashed after processing the message but before manually committing it. When we restart

Re: Kafka consumers in cluster

2016-08-06 Thread sat
Thanks Kamal, >Yes, it gets called on every re-balance. >-- Kamal On Thu, Aug 4, 2016 at 10:54 AM, sat <sathish.al...@gmail.com> wrote: > Hi Kamal, > > Thanks for your prompt response. Does our custom partition assignor gets > called during every rebalancing

Re: Kafka consumers in cluster

2016-08-04 Thread sat
artitions to the first consumer instance in >the group. >See 'partition.assignment.strategy' config in the consumer configs [1] >[1]: http://kafka.apache.org/documentation.html#newconsumerconfigs >On Thu, Aug 4, 2016 at 8:54 AM, sat <sathish.al...@gmail.com> wrote: > Hi, > >

Re: Kafka Consumer poll

2016-08-03 Thread sat
it split into 1min intervals of 10 times do{ if( pollForMessageIfAny()){ return; } Thread.sleep(1000); timeremaining = curretime - timeout; }while(timeremainning > 0) Thanks and Regards A.SathishKumar On Wed, Aug 3, 2016 at 5:46 PM, sat <sathish.al...@gmail.com> wro

Kafka consumers in cluster

2016-08-03 Thread sat
Hi, We have Kafka server/broker running in a seperate machine (say machine A), for now we are planning to have in one node. We have multiple topics and all topics have only 1 partition for now. We have our application which includes Kafka consumers installed in machine B and machine C. Our

Re: Kafka Consumer poll

2016-08-03 Thread sat
ent of temporary broker outage, it doesn't help if you >accidentally >specified wrong broker url. Oleg > On Aug 2, 2016, at 10:27, Kamal C <kamaltar...@gmail.com> wrote: > > See the answers inline. > >> On Tue, Aug 2, 2016 at 12:23 AM, sat <sathish.al...@gmail.com> wrot

Kafka Consumer poll

2016-08-01 Thread sat
Hi, I am new to Kafka. We are planning to use Kafka messaging for our application. I was playing with Kafka 0.9.0.1 version and i have following queries. Sorry for asking basic questions. 1) I have instantiated Kafka Consumer and invoked consumer.poll(Long.MAX_VALUE). Although i have specified