Thanks Joel. That seemed to do it. Since I wasn't handling the exception, the close() method never got called and some background threads must have been hanging around, keeping the JVM running. Here is the working producer code now:
https://gist.github.com/4082757 Since I couldn't find any full examples, I based the code largely on the ConsoleProducer which also appears to have this problem. I am happy to submit a patch for the ConsoleProducer and/or also put a full example on the wiki if that would be helpful. Thanks, Paul On 11/15/12 6:32 PM, "Joel Koshy" <jjkosh...@gmail.com> wrote: >From the threaddump it looks like your app did get the exception and the >main thread exited - i.e., it does not appear in the stacktrace. You >should >enclose the producer send with a try/catch and finally that closes the >producer. > > >On Thu, Nov 15, 2012 at 3:19 PM, Paul Mackles <pmack...@adobe.com> wrote: > >> Thanks. Thread dump from hung producer app here: >> >> https://gist.github.com/4082265 >> >> In case anyone has any insights. >> >> On 11/15/12 5:30 PM, "Neha Narkhede" <neha.narkh...@gmail.com> wrote: >> >> >Paul, >> > >> >This behavior is unexpected and the code suggests QueueFullException >> >is thrown back to the caller. I would take a thread dump when the >> >producer is blocked to see what's causing the blocking behavior. >> > >> >Thanks, >> >Neha >> > >> >On Thu, Nov 15, 2012 at 12:53 PM, Paul Mackles <pmack...@adobe.com> >> wrote: >> >> I am tinkering with the java async producer from kafka 7.2 and I am >> >>trying to write a client that will never block when sending messages >>to >> >>a kafka broker (dropping them would be better than blocking). I can >>send >> >>messages at a rate that is fast enough to trigger a QueueFullException >> >>but the exception never propagates up to my client code. In other >>words, >> >>I see the stack trace and the error message on the console but the >>call >> >>to Producer.send() that triggered the exception just hangs >>indefinitely. >> >>In this setup, queue.enqueueTimeout.ms=0. I get the same result when I >> >>set queue.enqueueTimeout.ms to a very low number positive number. Is >> >>this the intended behavior? Is there another way to guarantee your >> >>client can never block? If I had to guess, maybe the exception is >> >>happening in a background thread of some kind and that is why I never >> >>see it? My knowledge of scala is extremely limited. Here is the code I >> >>am using: >> >> >> >> https://gist.github.com/4081183 >> >> >> >> Thanks, >> >> Paul >> >> >> >>
