Hi, Thanks for all the replies. It helped me understand the system. I appreciate it.
I tried change the producer properties to async and also changed queue.enqueueTimeout.ms=-1. But I still get the exception. I then changed the producer queue size to 20K and 30K. My hope is by making the queue size bigger, it will buy the producer more time to digest the total amount of messages. But that didn't help in either 20K or 30K instances. I then increased max queue time to 5000ms in hopes that would hold some of the data. But in all these case, I still get the exception. I looked the console producer code and it uses synchronous producer class. In this case, does it make any difference if I change the producer's properties (e.g., to async)? I guess if the answer is no, it would explain what saw the above. Also is there any definition or explanation on the Java API? As I read the doc page on the site, there is none... Thanks Jamie -----Original Message----- From: Neha Narkhede [mailto:[email protected]] Sent: Monday, November 26, 2012 4:40 PM To: [email protected] Subject: Re: producer queue size This will happen if you push data to the producer at a higher rate than it is able to send to the server. queue.size allows you to configure the size of the producer queue (defaults to 10K). Also, queue.enqueueTimeout.ms, if set to -1 will lead to blocking behaviour instead of the producer throwing QueueFullExceptions. Thanks, Neha On Mon, Nov 26, 2012 at 4:06 PM, Joel Koshy <[email protected]> wrote: > To use async, set producer.type to async ; The default queue size is 10000; > and the default batch size is 200. > > > On Mon, Nov 26, 2012 at 2:28 PM, Jamie Wang <[email protected]> wrote: > >> Hi, I am running the console demo comes with 0.7.2 in getting started >> guide. All is working fine. Then I use stdio redirect a file of 30K lines >> of messages into the producer and I received an error "ERROR Event queue >> is full of unsent messages, could not send event:..." and exception stack >> shows: >> Exception in thread "main" kafka.producer.async.QueueFullException: Event >> queue is full of unsent messages, could not send event:... >> >> Is there a way in the produce.properties to configure the size of producer >> queue? I know the producer is running in synchronous mode. How do I solve >> this problem or my usage is totally wrong? >> >> Thanks >> Jamie >>
