Great idea. I’ve been talking about this for 2 years, and I’m glad someone is finally picking it up. Will take a look at the KIP at some point shortly.
-Todd On Mon, Jan 25, 2016 at 11:24 AM, Jay Kreps <j...@confluent.io> wrote: > Hey Becket, > > Yeah this is really similar to the callback. The difference is really in > who sets the behavior. The idea of the interceptor is that it doesn't > require any code change in apps so you can globally add behavior to your > Kafka usage without changing app code. Whereas the callback is added by the > app. The idea is to kind of obviate the need for the wrapper code that e.g. > LinkedIn maintains to hold this kind of stuff. > > -Jay > > On Sun, Jan 24, 2016 at 4:21 PM, Becket Qin <becket....@gmail.com> wrote: > > > This could be a useful feature. And I think there are some use cases to > > mutate the data like rejected alternative one mentioned. > > > > I am wondering if there is functional overlapping between > > ProducerInterceptor.onAcknowledgement() and the producer callback? I can > > see that the Callback could be a per record setting while > > onAcknowledgement() is a producer level setting. Other than that, is > there > > any difference between them? > > > > Thanks, > > > > Jiangjie (Becket) Qin > > > > On Fri, Jan 22, 2016 at 6:21 PM, Neha Narkhede <n...@confluent.io> > wrote: > > > > > James, > > > > > > That is one of the many monitoring use cases for the interceptor > > interface. > > > > > > Thanks, > > > Neha > > > > > > On Fri, Jan 22, 2016 at 6:18 PM, James Cheng <jch...@tivo.com> wrote: > > > > > > > Anna, > > > > > > > > I'm trying to understand a concrete use case. It sounds like producer > > > > interceptors could be used to implement part of LinkedIn's Kafak > Audit > > > > tool? https://engineering.linkedin.com/kafka/running-kafka-scale > > > > > > > > Part of that is done by a wrapper library around the kafka producer > > that > > > > keeps a count of the number of messages produced, and then sends that > > > count > > > > to a side-topic. It sounds like the producer interceptors could > > possibly > > > be > > > > used to implement that? > > > > > > > > -James > > > > > > > > > On Jan 22, 2016, at 4:33 PM, Anna Povzner <a...@confluent.io> > wrote: > > > > > > > > > > Hi, > > > > > > > > > > I just created a KIP-42 for adding producer and consumer > interceptors > > > for > > > > > intercepting messages at different points on producer and consumer. > > > > > > > > > > > > > > > > > > > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-42%3A+Add+Producer+and+Consumer+Interceptors > > > > > > > > > > Comments and suggestions are welcome! > > > > > > > > > > Thanks, > > > > > Anna > > > > > > > > > > > > ________________________________ > > > > > > > > This email and any attachments may contain confidential and > privileged > > > > material for the sole use of the intended recipient. Any review, > > copying, > > > > or distribution of this email (or any attachments) by others is > > > prohibited. > > > > If you are not the intended recipient, please contact the sender > > > > immediately and permanently delete this email and any attachments. No > > > > employee or agent of TiVo Inc. is authorized to conclude any binding > > > > agreement on behalf of TiVo Inc. by email. Binding agreements with > TiVo > > > > Inc. may only be made by a signed written agreement. > > > > > > > > > > > > > > > > -- > > > Thanks, > > > Neha > > > > > > -- *—-* *Todd Palino* Staff Site Reliability Engineer Data Infrastructure Streaming linkedin.com/in/toddpalino
