I would like to write a simple REST GW for event producers outside the firewall, and that the REST would acknowledge that the items were persisted in Kafka (so the REST client could decide to retry if an error occured). Without any callback, I need to revert to the Producer sync API which means I would be limited by the number of Java threads, which is mapped in Linux to light processes, which is limited by Linux ulimit.... which limits scalability.
There must be some way I can extend/hack/fork into the Producer in the async background thread in version 0.8 (and not wait until 0.9 is released). For example in 0.7, it looked like extending DefaultEventHandler could do the trick. Could you give some pointers for 0.8? Itai -----Original Message----- From: Drew Daugherty [mailto:drew.daughe...@returnpath.com] Sent: Friday, April 19, 2013 7:55 PM To: users@kafka.apache.org Subject: RE: kafka.javaapi.producer.SyncProducer Confusion I would second this request. We would like to gather metrics on the SyncProducer (bytes in/out, messages in/out, time spent in preparing/sending messages, etc.) when we are sending asynchronously. Or would you recommend something like Scala actors as a better solution? -drew ________________________________________ From: Neha Narkhede [neha.narkh...@gmail.com] Sent: Friday, April 19, 2013 10:20 AM To: users@kafka.apache.org Subject: Re: kafka.javaapi.producer.SyncProducer Confusion In 0.8, the producer API does not have callbacks to notify you of the status of the messages. However, I think we want to rethink the client APIs for 0.9. Let us know if you have feedback. Thanks, Neha On Fri, Apr 19, 2013 at 4:15 AM, Itai Frenkel <it...@gigaspaces.com> wrote: > Could you give some pointers (or test case perhaps) how to detect that a > message was sent successfully to the broker, when sending the data > asynchronously? > > -----Original Message----- > From: Swapnil Ghike [mailto:sgh...@linkedin.com] > Sent: Friday, April 19, 2013 10:24 AM > To: users@kafka.apache.org > Subject: Re: kafka.javaapi.producer.SyncProducer Confusion > > Hi Samir, > > In 0.8, we don't have SyncProducer as a public api. You can use > kafka.javaapi.producer.Producer to send the data synchronously or > asynchronously. > > The quick start page is outdated, and we plan to update a few pages > soon (before the 0.8 BETA release). In fact, we would love to have > contribution from the community for updating the website. :) > > Thanks, > Swapnil > > > > On 4/18/13 11:30 PM, "Samir Madhavan" > <samir.madha...@fluturasolutions.com> wrote: > >>Hi, >> >>I'm using kafka-0.8 Snapshot. In the quick start, there is an example >>producer code in which SyncProducer class has been mentioned but when >>I checked the kafka.javaapi.producer, there is only Producer class and >>no SyncProducer or ProducerData class. >> >>I went through the apache github source code and also the svn trunk >>code but these classes are not there. >> >>In the api documentation of apache kafka site, these classes have been >>shown. >> >>Do these classes still exist or am I going wrong somewhere? >> >>Regards, >>Samir > > >
