Hello Prashant, You can use on the producer failure sensors to do the monitoring.
http://kafka.apache.org/documentation.html#monitoring Guozhang On Thu, Jul 10, 2014 at 6:11 AM, Prashant Prakash <prash.i...@gmail.com> wrote: > Dear Kafka Users, > > We are using kafka 0.8.0 is our application development. To keep message > delivery reliable we want to detect any failure while sending message. To > get high throughput we are using async producer. > > As of kafka 0.8.0 async producer implementation, failure to send to message > is logged but not thrown back to producer client. > > Snippet from ProducerSendThread - > > def tryToHandle(events: Seq[KeyedMessage[K,V]]) { > val size = events.size > try { > debug("Handling " + size + " events") > if(size > 0) > handler.handle(events) > }catch { > case e: Throwable => error("Error in handling batch of " + size + " > events", e) > } > } > > Is there any better solution to monitor the async send failure ? > > Few of the solution we came up are : > 1. Create message buffer in our application logic. Instantiate configurable > pool of sync producer which reads and send message from buffer. (Sync > producer throws back exception to the caller class) > 2. Extend DefaultEventHandler and pass an exception handler callback. On > encountering exception the callback method -handleException() will be > called notifying application code of any failure. > > Please share views, solutions which you have used to tack async send > failure. > > Thank you very much. > > Best regards > Prashant > -- -- Guozhang