Up to Ben to clarify, but I'd think that in this case, it is up to the logic of B to decide what to do. B knows that the offset isn't what it expects, so it can react accordingly. If it chooses to try again, then it should not violate any application invariant.
-Flavio On Fri, Jul 17, 2015 at 9:49 PM, Ashish Singh <asi...@cloudera.com> wrote: > Good concept. I have a question though. > > Say there are two producers A and B. Both producers are producing to same > partition. > - A sends a message with expected offset, x1 > - Broker accepts is and sends an Ack > - B sends a message with expected offset, x1 > - Broker rejects it, sends nack > - B sends message again with expected offset, x1+1 > - Broker accepts it and sends Ack > I guess this is what this KIP suggests, right? If yes, then how does this > ensure that same message will not be written twice when two producers are > producing to same partition? Producer on receiving a nack will try again > with next offset and will keep doing so till the message is accepted. Am I > missing something? > > Also, you have mentioned on KIP, "it imposes little to no runtime cost in > memory or time", I think that is not true for time. With this approach > producers' performance will reduce proportionally to number of producers > writing to same partition. Please correct me if I am missing out something. > > > On Fri, Jul 17, 2015 at 11:32 AM, Mayuresh Gharat < > gharatmayures...@gmail.com> wrote: > > > If we have 2 producers producing to a partition, they can be out of > order, > > then how does one producer know what offset to expect as it does not > > interact with other producer? > > > > Can you give an example flow that explains how it works with single > > producer and with multiple producers? > > > > > > Thanks, > > > > Mayuresh > > > > On Fri, Jul 17, 2015 at 10:28 AM, Flavio Junqueira < > > fpjunque...@yahoo.com.invalid> wrote: > > > > > I like this feature, it reminds me of conditional updates in zookeeper. > > > I'm not sure if it'd be best to have some mechanism for fencing rather > > than > > > a conditional write like you're proposing. The reason I'm saying this > is > > > that the conditional write applies to requests individually, while it > > > sounds like you want to make sure that there is a single client writing > > so > > > over multiple requests. > > > > > > -Flavio > > > > > > > On 17 Jul 2015, at 07:30, Ben Kirwin <b...@kirw.in> wrote: > > > > > > > > Hi there, > > > > > > > > I just added a KIP for a 'conditional publish' operation: a simple > > > > CAS-like mechanism for the Kafka producer. The wiki page is here: > > > > > > > > > > > > > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-27+-+Conditional+Publish > > > > > > > > And there's some previous discussion on the ticket and the users > list: > > > > > > > > https://issues.apache.org/jira/browse/KAFKA-2260 > > > > > > > > > > > > > > https://mail-archives.apache.org/mod_mbox/kafka-users/201506.mbox/%3CCAAeOB6ccyAA13YNPqVQv2o-mT5r=c9v7a+55sf2wp93qg7+...@mail.gmail.com%3E > > > > > > > > As always, comments and suggestions are very welcome. > > > > > > > > Thanks, > > > > Ben > > > > > > > > > > > > -- > > -Regards, > > Mayuresh R. Gharat > > (862) 250-7125 > > > > > > -- > > Regards, > Ashish >
