Sure, the custom producer and consumer library I'm updating is Krackle: https://github.com/blackberry/Krackle
It's a very limited low overhead library that we're using to get syslog style messages from the Blackberry service provider infrastructure into Kafka. We have dozens of pools of servers containing dozens of hosts each which each host receiving and forwarding ~50K msg/sec into Kafka. We found that neither the Kafka proper producers nor Kafka could keep up with the stock packages when we initially looked (pre 0.8) ________________________________________ From: [email protected] [[email protected]] on behalf of Ismael Juma [[email protected]] Sent: Thursday, December 10, 2015 12:44 PM To: [email protected] Subject: Re: Interacting with a secured Kafka cluster via GSS-API On Thu, Dec 10, 2015 at 5:24 PM, Dave Ariens <[email protected]> wrote: > Absolutely, currently I'm hoping to get authentication working and then > ultimately work towards encryption. We're also testing performance of > more out-of-the-box Kafka components but I fear our message volume will > require us to maintain our custom producers and consumers. > If it turns out that the out of the box components are not good enough, it would be interesting to know how your custom clients differ. Is that documented somewhere? Thanks, Ismael
