Hey folks, If anyone can quickly comment on the below producer calls? If this is worth more discussion I'll raise it on the users list under a more appropriate subject.
Thanks ________________________________________ From: Dave Ariens Sent: Thursday, December 10, 2015 1:40 PM To: dev@kafka.apache.org Subject: RE: Interacting with a secured Kafka cluster via GSS-API > Interested to find out if the new producer and consumer fare > better Regarding that, what's the best way to use the new producer with as little overhead as possible? import org.apache.kafka.clients.producer.KafkaProducer; import org.apache.kafka.clients.producer.ProducerRecord; producer = new KafkaProducer<byte[], byte[]>(conf.getProperties()); The code containing the producer is listening on a TCP socket and reading into a byte buffer until there's a new line character. It then treats the contents of the byte array up to the position of the buffer as a message. byte[] payload = Arrays.copyOf(bytes, sendBuffer.position()); ProducerRecord<byte[], byte[]> record = new ProducerRecord<>(topic, payload); producer.send(record); ...is there a more elegant way to produce that message? Having to create a new object by itself would be too costly in our use case, let alone the array copy... (another disclaimer: I really haven't played around with the new producer much at all--and even my 0.8.2.1 knowledge is mainly focused on Krackle) ________________________________________ From: isma...@gmail.com [isma...@gmail.com] on behalf of Ismael Juma [ism...@juma.me.uk] Sent: Thursday, December 10, 2015 1:10 PM To: dev@kafka.apache.org Subject: Re: Interacting with a secured Kafka cluster via GSS-API On Thu, Dec 10, 2015 at 6:06 PM, Dave Ariens <dari...@blackberry.com> wrote: > Sure, the custom producer and consumer library I'm updating is Krackle: > https://github.com/blackberry/Krackle > ... > We found that neither the Kafka proper producers nor Kafka could keep up > with the stock packages when we initially looked (pre 0.8) > Thanks, Dave. Interested to find out if the new producer and consumer fare better. :) Ismael
