Hi Yang, In the code suggested by Manikumar, yes, it is possible message 3 still got sent even message 2 failed. There is no single line code for send a batch of message synchronously now, but after KAFKA-1660 is checked in, you may be able to achieve this by doing the following: Set a callback for each send, in the callback, if exception occurs, call producer.close(-1) - this will prevent further messages to be sent. Then you can the following code:
For(record: records) Producer.send(record, callback) Producer.flush() Jiangjie (Becket) Qin On 3/9/15, 8:49 AM, "Yu Yang" <yuyan...@gmail.com> wrote: >If a send request in the middle of the list fails, will all send requests >that follows it fail? Or only the messages that are put in the same batch >by the underneath transportation layer fail? > >On Mon, Mar 9, 2015 at 1:31 AM, Manikumar Reddy <ku...@nmsworks.co.in> >wrote: > >> 1. We can send list of messages and wait on the returned futures >> >> List responses = new ArrayList(); >> for(input: recordBatch) >> responses.add(producer.send(input)); >> for(response: responses) >> response.get >> >> 2. messages will be send in the submission order. >> >> On Mon, Mar 9, 2015 at 1:56 PM, Manikumar Reddy <ku...@nmsworks.co.in> >> wrote: >> >> > 1 . >> > >> > >> > >> > On Mon, Mar 9, 2015 at 1:03 PM, Yu Yang <yuyan...@gmail.com> wrote: >> > >> >> The confluent blog >> >> < >> http://blog.confluent.io/2014/12/02/whats-coming-in-apache-kafka-0-8-2/> >> >> mentions >> >> that the the batching is done whenever possible now. "The sync >>producer, >> >> under load, can get performance as good as the async producer. " >>Does >> it >> >> mean that kafka 0.8.2 guarantees that the sequence of >> >> broker-received-message is the same as the async-call sequence by the >> >> producer? If I have code as follows to send messages to the same >>topic >> >> partition: >> >> >> >> producer.send(msg1); >> >> prodcuer.send(msg2); >> >> producer.send(msg3); >> >> >> >> If all three calls succeed, is it possible that broker received msg3 >> >> before >> >> msg2? >> >> >> >> >> >> >> >> >> >> On Mon, Mar 9, 2015 at 12:17 AM, Yu Yang <yuyan...@gmail.com> wrote: >> >> >> >> > Hi, >> >> > >> >> > Kafka 0.8.1.1 allows us to send a list of messages in sync mode: >> >> > >> >> > public void send(List<KeyedMessage<K,V> messages); >> >> > >> >> > I did not find a counter-part of this api in the new producer that >>is >> >> > introduced in kafka 0.8.2. It seems that we can use the following >> >> method >> >> > to do sync send in kafka 0.8.2: >> >> > >> >> > producer.send(new ProducerRecord(...))).get(); >> >> > >> >> > My understanding is that we can only send one message at a time in >> >> > sync-mode. This will limit the throughput of kafka producer. Is >>there >> a >> >> way >> >> > to send a batch of messages in sync mode using kafka 0.8.2 >>producer? >> Is >> >> > there any study on the throughput of Kafka 0.8.2 producer, >>comparing >> >> with >> >> > kafka 0.8.1.1? >> >> > >> >> > Thanks! >> >> > >> >> > Regards, >> >> > Yu >> >> > >> >> > >> >> > >> >> >> > >> > >>
