>>0.8 has the same DefaultEventHandler logic as 0.7, so can you try to do the >>same in 0.8 ? We will make this easier to achieve in 0.9.
The "event.handler" configuration from v0.7 is gone from v0.8. So I would not be able to inject a custom event handler. Would you consider adding some kind of hook into 0.8 to allow something like this: def handle(events: Seq[KeyedMessage[K,V]]) { try { monitorBefore(events); super.handle(events); monitorAfter(events); catch { case t => monitorOnError(events, e); throw t; } It's not perfect, but it could be good enough for writing an Async REST GW until 0.9 comes out. It would also cover some of Drew's use cases described below. Thanks, Itai -----Original Message----- From: Neha Narkhede [mailto:neha.narkh...@gmail.com] Sent: Saturday, April 20, 2013 8:09 AM To: users@kafka.apache.org Subject: Re: kafka.javaapi.producer.SyncProducer Confusion >> There must be some way I can extend/hack/fork into the Producer in the async >> background thread in version 0.8 (and not wait until 0.9 is released). For >> example in 0.7, it looked like extending DefaultEventHandler could do the >> trick. 0.8 has the same DefaultEventHandler logic as 0.7, so can you try to do the same in 0.8 ? We will make this easier to achieve in 0.9. Thanks, Neha On Fri, Apr 19, 2013 at 2:25 PM, Itai Frenkel <it...@gigaspaces.com> wrote: > I would like to write a simple REST GW for event producers outside the > firewall, and that the REST would acknowledge that the items were persisted > in Kafka (so the REST client could decide to retry if an error occured). > Without any callback, I need to revert to the Producer sync API which means I > would be limited by the number of Java threads, which is mapped in Linux to > light processes, which is limited by Linux ulimit.... which limits > scalability. > > There must be some way I can extend/hack/fork into the Producer in the async > background thread in version 0.8 (and not wait until 0.9 is released). For > example in 0.7, it looked like extending DefaultEventHandler could do the > trick. Could you give some pointers for 0.8? > > Itai > > -----Original Message----- > From: Drew Daugherty [mailto:drew.daughe...@returnpath.com] > Sent: Friday, April 19, 2013 7:55 PM > To: users@kafka.apache.org > Subject: RE: kafka.javaapi.producer.SyncProducer Confusion > > I would second this request. We would like to gather metrics on the > SyncProducer (bytes in/out, messages in/out, time spent in preparing/sending > messages, etc.) when we are sending asynchronously. Or would you recommend > something like Scala actors as a better solution? > > -drew > ________________________________________ > From: Neha Narkhede [neha.narkh...@gmail.com] > Sent: Friday, April 19, 2013 10:20 AM > To: users@kafka.apache.org > Subject: Re: kafka.javaapi.producer.SyncProducer Confusion > > In 0.8, the producer API does not have callbacks to notify you of the status > of the messages. However, I think we want to rethink the client APIs for 0.9. > Let us know if you have feedback. > > Thanks, > Neha > > On Fri, Apr 19, 2013 at 4:15 AM, Itai Frenkel <it...@gigaspaces.com> wrote: >> Could you give some pointers (or test case perhaps) how to detect that a >> message was sent successfully to the broker, when sending the data >> asynchronously? >> >> -----Original Message----- >> From: Swapnil Ghike [mailto:sgh...@linkedin.com] >> Sent: Friday, April 19, 2013 10:24 AM >> To: users@kafka.apache.org >> Subject: Re: kafka.javaapi.producer.SyncProducer Confusion >> >> Hi Samir, >> >> In 0.8, we don't have SyncProducer as a public api. You can use >> kafka.javaapi.producer.Producer to send the data synchronously or >> asynchronously. >> >> The quick start page is outdated, and we plan to update a few pages >> soon (before the 0.8 BETA release). In fact, we would love to have >> contribution from the community for updating the website. :) >> >> Thanks, >> Swapnil >> >> >> >> On 4/18/13 11:30 PM, "Samir Madhavan" >> <samir.madha...@fluturasolutions.com> wrote: >> >>>Hi, >>> >>>I'm using kafka-0.8 Snapshot. In the quick start, there is an example >>>producer code in which SyncProducer class has been mentioned but when >>>I checked the kafka.javaapi.producer, there is only Producer class >>>and no SyncProducer or ProducerData class. >>> >>>I went through the apache github source code and also the svn trunk >>>code but these classes are not there. >>> >>>In the api documentation of apache kafka site, these classes have >>>been shown. >>> >>>Do these classes still exist or am I going wrong somewhere? >>> >>>Regards, >>>Samir >> >> >> > >