We are doing one scala consumer and one java consumer who listen on same topic with different group id.The issue is if I start both second one fails with this error.If we run just one its working fine.
Attaching Scala code and Java code where we call createMessageStreams.It seems like issue with API not recognizing that its different group of consumers that are started because createMessageStream doesnot accept group.id but should refer already passed group.id to make sure second on is new stream and not related even we listen on same topic. Please let me know if it is API issue or way we handle API. -----Original Message----- From: Joe Stein [mailto:joe.st...@stealth.ly] Sent: Friday, December 13, 2013 4:36 PM To: users@kafka.apache.org Subject: Re: Unable to start consumers in Tomcat you do this at the consumer connector level not at the message stream level so one propsA.put("group.id", "groupA") be val configA = new ConsumerConfig(propsA) val one = Consumer.create(configA) and another propsB.put("group.id", "groupB") be val configB = new ConsumerConfig(propsB) val two = Consumer.create(configB) so you then manage each Consumer connector created by each of those .create in separate threads (or actors or processes or however you are doing concurrency) and you only call the create message stream each once but each once on each of them makes sense? /******************************************* Joe Stein Founder, Principal Consultant Big Data Open Source Security LLC http://www.stealth.ly Twitter: @allthingshadoop <http://www.twitter.com/allthingshadoop> ********************************************/ On Fri, Dec 13, 2013 at 6:29 PM, Seshadri, Balaji <balaji.sesha...@dish.com>wrote: > Cant we create message stream for same topic but different consumer > group ?. > > -----Original Message----- > From: Joe Stein [mailto:joe.st...@stealth.ly] > Sent: Friday, December 13, 2013 4:23 PM > To: users@kafka.apache.org > Subject: Re: Unable to start consumers in Tomcat > > You should check your code to verify that it is only called once per > instantiated consumer connector. Here is where the exception is > thrown > https://github.com/apache/kafka/blob/0.8/core/src/main/scala/kafka/con > sumer/ZookeeperConsumerConnector.scala#L133from > > /******************************************* > Joe Stein > Founder, Principal Consultant > Big Data Open Source Security LLC > http://www.stealth.ly > Twitter: @allthingshadoop <http://www.twitter.com/allthingshadoop> > ********************************************/ > > > On Fri, Dec 13, 2013 at 6:17 PM, Seshadri, Balaji > <balaji.sesha...@dish.com>wrote: > > > We needed HTTP interface to start our consumers using REST interface > > for management that why we chose tomcat to run our consumers. > > > > We create streams only once.when we initially start consumer. > > > > -----Original Message----- > > From: Joe Stein [mailto:joe.st...@stealth.ly] > > Sent: Friday, December 13, 2013 4:12 PM > > To: users@kafka.apache.org > > Subject: Re: Unable to start consumers in Tomcat > > > > That error comes if you are trying to create a message/stream on a > > consumer instantiated object more than once. > > > > Why are you using tomcat for the consumers? Is it to see the results > > of messages? if so you need to isolate the Consumer in some way so > > there is a singleton (assuming one partition or if more than one > > partition some type of consumer pool) and the messages being > > consumed write to a thread safe memory structure and then the UI can > > read from that thread safe memory structure. > > > > /******************************************* > > Joe Stein > > Founder, Principal Consultant > > Big Data Open Source Security LLC > > http://www.stealth.ly > > Twitter: @allthingshadoop <http://www.twitter.com/allthingshadoop> > > ********************************************/ > > > > > > On Fri, Dec 13, 2013 at 5:42 PM, Seshadri, Balaji > > <balaji.sesha...@dish.com>wrote: > > > > > 0.8 > > > > > > -----Original Message----- > > > From: Neha Narkhede [mailto:neha.narkh...@gmail.com] > > > Sent: Friday, December 13, 2013 3:33 PM > > > To: users@kafka.apache.org > > > Subject: Re: Unable to start consumers in Tomcat > > > > > > Which version of kafka are you using? > > > > > > > > > On Fri, Dec 13, 2013 at 2:29 PM, Seshadri, Balaji > > > <balaji.sesha...@dish.com>wrote: > > > > > > > Any idea on this error guys ?. > > > > > > > > -----Original Message----- > > > > From: Seshadri, Balaji [mailto:balaji.sesha...@dish.com] > > > > Sent: Friday, December 13, 2013 9:32 AM > > > > To: 'users@kafka.apache.org' > > > > Subject: Unable to start consumers in Tomcat > > > > > > > > Hi , > > > > > > > > Can you guys lets us know why are we getting this error when try > > > > to spawn a consumer ?. > > > > > > > > ZookeeperConsumerConnector can create message streams at most > > > > once > > > > > > > > Balaji > > > > > > > > > >
