Re: Multiple consumer groups with same group id on a single topic

2015-03-10 Thread Jiangjie Qin
nal Message- >> From: Phill Tomlinson [mailto:philltomlin...@fico.com] >> Sent: 10 March 2015 08:18 >> To: users@kafka.apache.org >> Subject: RE: Multiple consumer groups with same group id on a single >>topic >> >> Hi, >> >> Code snippet bel

Re: Multiple consumer groups with same group id on a single topic

2015-03-10 Thread tao xiao
; > -Original Message- > From: Phill Tomlinson [mailto:philltomlin...@fico.com] > Sent: 10 March 2015 08:18 > To: users@kafka.apache.org > Subject: RE: Multiple consumer groups with same group id on a single topic > > Hi, > > Code snippet below. This creates two consumers wi

RE: Multiple consumer groups with same group id on a single topic

2015-03-10 Thread Phill Tomlinson
executorService2.submit(new CcsKafkaStreamConsumer(stream, ("common-topic", messageWrapper)); } -Original Message----- From: Kevin Scaldeferri [mailto:ke...@scaldeferri.com] Sent: 09 March 2015 18:13 To: users@kafka.apache.org Subject: Re: Multiple consumer groups with same group id o

RE: Multiple consumer groups with same group id on a single topic

2015-03-10 Thread Phill Tomlinson
ot;, messageWrapper)); } -Original Message- From: Kevin Scaldeferri [mailto:ke...@scaldeferri.com] Sent: 09 March 2015 18:13 To: users@kafka.apache.org Subject: Re: Multiple consumer groups with same group id on a single topic On Mon, Mar 9, 2015 at 10:38 AM, Phill Tomlinson wrote: > H

Re: Multiple consumer groups with same group id on a single topic

2015-03-09 Thread Jiangjie Qin
Yes, Kevin is right. It does not matter whether you run the consumer from the same JVM or not, as long as the consumers has same group id, they are in the same group. So in your case, you have 6 consumers in the same consumer group. Since you have 6 partitions in the topic, assuming you have only o

Re: Multiple consumer groups with same group id on a single topic

2015-03-09 Thread Kevin Scaldeferri
On Mon, Mar 9, 2015 at 10:38 AM, Phill Tomlinson wrote: > Hi, > > I have two separate consumer groups on different JVM processes, but both > have the same "group.id". You've said this twice, and I think it's creating some confusion, because the group.id is exactly what determines the members o

RE: Multiple consumer groups with same group id on a single topic

2015-03-09 Thread Phill Tomlinson
e consumer groups with same group id on a single topic If you have 2 consumer groups, each group will read from all partitions automaticcally if you are using HighLevel consumer ( In your case it would be each consumer gets 2 partitons). You don't have to specify the partitions it should rea

Re: Multiple consumer groups with same group id on a single topic

2015-03-09 Thread Mayuresh Gharat
If you have 2 consumer groups, each group will read from all partitions automaticcally if you are using HighLevel consumer ( In your case it would be each consumer gets 2 partitons). You don't have to specify the partitions it should read from. Thanks, Mayuresh On Mon, Mar 9, 2015 at 9:59 AM, Ji

Re: Multiple consumer groups with same group id on a single topic

2015-03-09 Thread Jiangjie Qin
HI Phill, Do you mean you are using 6 consumers with the same group id? Or you have 3 consumers using one group id, and another 3 using another different group id? For the example you mentioned, what you can do is to run several consumers on different physical machine with the same group id, they