Hmm, that should be big enough. Thanks,
Jun On Thu, Nov 29, 2012 at 10:03 AM, Jamie Wang <[email protected]> wrote: > log.flush.interval is set to 10000. Should I increase or decrease? > > thanks > Jamie > > -----Original Message----- > From: Jun Rao [mailto:[email protected]] > Sent: Wednesday, November 28, 2012 10:00 PM > To: [email protected] > Subject: Re: producer queue size > > Is the server busy on I/O? What's log.flush.interval on the broker? For > better performance, you need to set it to a few hundreds or more. > > Thanks, > > Jun > > On Wed, Nov 28, 2012 at 7:12 PM, Jamie Wang <[email protected]> > wrote: > > > Hi, > > > > Thanks for all the replies. It helped me understand the system. I > > appreciate it. > > > > I tried change the producer properties to async and also changed > > queue.enqueueTimeout.ms=-1. But I still get the exception. I then > changed > > the producer queue size to 20K and 30K. My hope is by making the queue > size > > bigger, it will buy the producer more time to digest the total amount of > > messages. But that didn't help in either 20K or 30K instances. I then > > increased max queue time to 5000ms in hopes that would hold some of the > > data. But in all these case, I still get the exception. > > > > I looked the console producer code and it uses synchronous producer > class. > > In this case, does it make any difference if I change the producer's > > properties (e.g., to async)? I guess if the answer is no, it would > explain > > what saw the above. > > > > Also is there any definition or explanation on the Java API? As I read > the > > doc page on the site, there is none... > > > > Thanks > > Jamie > > > > > > > > -----Original Message----- > > From: Neha Narkhede [mailto:[email protected]] > > Sent: Monday, November 26, 2012 4:40 PM > > To: [email protected] > > Subject: Re: producer queue size > > > > This will happen if you push data to the producer at a higher rate > > than it is able to send to the server. > > queue.size allows you to configure the size of the producer queue > > (defaults to 10K). Also, > > queue.enqueueTimeout.ms, if set to -1 will lead to blocking behaviour > > instead of the producer throwing > > QueueFullExceptions. > > > > Thanks, > > Neha > > > > On Mon, Nov 26, 2012 at 4:06 PM, Joel Koshy <[email protected]> wrote: > > > To use async, set producer.type to async ; The default queue size is > > 10000; > > > and the default batch size is 200. > > > > > > > > > On Mon, Nov 26, 2012 at 2:28 PM, Jamie Wang <[email protected]> > > wrote: > > > > > >> Hi, I am running the console demo comes with 0.7.2 in getting started > > >> guide. All is working fine. Then I use stdio redirect a file of 30K > > lines > > >> of messages into the producer and I received an error "ERROR Event > > queue > > >> is full of unsent messages, could not send event:..." and exception > > stack > > >> shows: > > >> Exception in thread "main" kafka.producer.async.QueueFullException: > > Event > > >> queue is full of unsent messages, could not send event:... > > >> > > >> Is there a way in the produce.properties to configure the size of > > producer > > >> queue? I know the producer is running in synchronous mode. How do I > > solve > > >> this problem or my usage is totally wrong? > > >> > > >> Thanks > > >> Jamie > > >> > > >
