Hello Prashant,

You can use on the producer failure sensors to do the monitoring.

http://kafka.apache.org/documentation.html#monitoring

Guozhang


On Thu, Jul 10, 2014 at 6:11 AM, Prashant Prakash <prash.i...@gmail.com>
wrote:

> Dear Kafka Users,
>
> We are using kafka 0.8.0 is our application development. To keep message
> delivery reliable we want to detect any failure while sending message. To
> get high throughput we are using async producer.
>
> As of kafka 0.8.0 async producer implementation, failure to send to message
> is logged but not thrown back to producer client.
>
> Snippet from ProducerSendThread -
>
> def tryToHandle(events: Seq[KeyedMessage[K,V]]) {
>     val size = events.size
>     try {
>       debug("Handling " + size + " events")
>       if(size > 0)
>         handler.handle(events)
>     }catch {
>       case e: Throwable => error("Error in handling batch of " + size + "
> events", e)
>     }
>   }
>
> Is there any better solution to monitor the async send failure ?
>
> Few of the solution we came up are :
> 1. Create message buffer in our application logic. Instantiate configurable
> pool of  sync producer which reads and send message from buffer. (Sync
> producer throws back exception to the caller class)
> 2. Extend DefaultEventHandler and pass an exception handler callback. On
> encountering exception the callback  method -handleException() will be
> called notifying application code of any failure.
>
> Please share views, solutions which you have used to tack async send
> failure.
>
> Thank you very much.
>
> Best regards
> Prashant
>



-- 
-- Guozhang

Reply via email to