Yeah I think this is a reasonable strategy for reducing client side logic. For the java client this is not needed as we have an async mode meant specifically for this that will batch messages together and send out batched requests in a background thread.
I am not aware of anyone having open sourced what you describe. If anyone is interested one nice way to implement this would be to implement a daemon that could interact with the syslogd protocol over a unix domain socket. Many scripting languages have good syslogd integration and this would probably be a good strategy for supporting logging from this kind of process. -Jay On Tue, Oct 9, 2012 at 8:53 AM, howard chen <howac...@gmail.com> wrote: > Hi, > > > On Tue, Oct 9, 2012 at 11:51 PM, Neha Narkhede <neha.narkh...@gmail.com> > wrote: > > Howard, > > > > Kafka consumers fetch data in bulk, controlled by the consumer's > > fetch.size, to reduce the overhead of a network roundtrip. Hence, > > there is no need for any special agent between a Kafka broker and > > consumer. > > My concern is on the producer hosts, i.e. calling localhost should > minimize the network latency, similar to a localhost SMTP relay. > > Thanks. >
