Florian, Please can you file a bug, with information on how to reproduce it ?
Thanks, Neha On Fri, Dec 9, 2011 at 3:35 PM, Florian Leibert <f...@leibert.de> wrote: > I'm using Zookeeper - so no static lists at all. It's not really > reproducable because it occurs only after a couple of hundred messages have > been sent. > But it usually occurs within the first 1000. I tried it with github/trunk > as well as with 0.6. > > I literally copied the code from the simple producer / consumer example. > > On Fri, Dec 9, 2011 at 3:10 PM, Neha Narkhede <neha.narkh...@gmail.com > >wrote: > > > >> Adding a broker after the producer stopped and > > restarting the producer (consumer still running) throws on the consumer > > side the InvalidMessageSize exception - which continues to happen on > > subsequent produce/consume cycles as long as the second broker is > running. > > > > What version of Kafka are you running ? Are you using the producer with > > zk.connect or broker.list ? If you are using broker.list, then you must > > modify it after deploying a second broker. > > > > Is the InvalidMessageSizeException reproducible using our > > producer/simple-consumer shells ? > > > > Thanks, > > Neha > > > > On Fri, Dec 9, 2011 at 1:12 PM, Florian Leibert <f...@leibert.de> wrote: > > > > > My observations so far: > > > > > > If I start with 1 broker, 1 consumer (code is pretty much the same as > in > > > the simple-consumer-demo) and 1 producer I can do arbitrary many runs > and > > > everything works fine. Adding a broker after the producer stopped and > > > restarting the producer (consumer still running) throws on the consumer > > > side the InvalidMessageSize exception - which continues to happen on > > > subsequent produce/consume cycles as long as the second broker is > > running. > > > > > > I looked around Zookeeper - there are 8 partitions when one broker is > > > active and 16 with two... i see all the topics are registered in > > zookeeper > > > and overall zookeeper logs are clean. > > > > > > After this fact I tried to see if two brokers and one partition each > > would > > > work so I shutdown the brokers, changed the global partition size to 1 > in > > > each of the configs and restarted them. The result is the same, I still > > get > > > InvalidMessageSize exceptions on the consumer size. > > > > > > On Fri, Dec 9, 2011 at 11:10 AM, Florian Leibert <f...@leibert.de> > wrote: > > > > > > > On that note - another error I see right around the same time is > below > > - > > > i > > > > see it on the consumer end. Note that in this case I have one > consumer > > > with > > > > a single iterator - and on each broker 8 partitions. When printing > > > > messages, I see an empty message on occasion... > > > > > > > > Exception in thread "main" kafka.common.InvalidMessageSizeException > > > > at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native > > Method) > > > > at > > > > > > > > > > sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:39) > > > > at > > > > > > > > > > sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:27) > > > > at java.lang.reflect.Constructor.newInstance(Constructor.java:513) > > > > at java.lang.Class.newInstance0(Class.java:355) > > > > at java.lang.Class.newInstance(Class.java:308) > > > > at > > kafka.common.ErrorMapping$.maybeThrowException(ErrorMapping.scala:53) > > > > at > > > > > > > > > > kafka.message.ByteBufferMessageSet.iterator(ByteBufferMessageSet.scala:60) > > > > at > kafka.consumer.ConsumerIterator.makeNext(ConsumerIterator.scala:67) > > > > at > kafka.consumer.ConsumerIterator.makeNext(ConsumerIterator.scala:30) > > > > at > > > > kafka.utils.IteratorTemplate.maybeComputeNext(IteratorTemplate.scala:58) > > > > at kafka.utils.IteratorTemplate.hasNext(IteratorTemplate.scala:50) > > > > at com.airbnb.logging.util.ConsumerTest.main(ConsumerTest.java:50) > > > > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > > > > at > > > > > > > > > > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) > > > > at > > > > > > > > > > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) > > > > at java.lang.reflect.Method.invoke(Method.java:597) > > > > at > > com.intellij.rt.execution.application.AppMain.main(AppMain.java:120) > > > > > > > > > > > > > > > > > -- > > > Best regards, > > > > > > Florian > > > http://twitter.com/flo <http://twitter.com/floleibert> > > > http://flori.posterous.com/ > > > > > > > > > -- > Best regards, > > Florian > http://twitter.com/flo <http://twitter.com/floleibert> > http://flori.posterous.com/ >
