Hello folks, I have updated the KIP page with some detailed API / architecture / packaging proposals, along with the long promised first patch in PR:
https://cwiki.apache.org/confluence/display/KAFKA/KIP-28+-+Add+a+processor+client https://github.com/apache/kafka/pull/130 Any feedbacks / comments are more than welcomed. Guozhang On Mon, Aug 10, 2015 at 6:55 PM, Guozhang Wang <wangg...@gmail.com> wrote: > Hi Jun, > > 1. I have removed the streamTime in punctuate() since it is not only > triggered by clock time, detailed explanation can be found here: > > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-28+-+Add+a+processor+client#KIP-28-Addaprocessorclient-StreamTime > > 2. Yes, if users do not schedule a task, then punctuate will never fire. > > 3. Yes, I agree. The reason it was implemented in this way is that the > state store registration call is triggered by the users. However I think it > is doable to change that API so that it will be more natural to have sth. > like: > > context.createStore(store-name, store-type). > > Guozhang > > On Tue, Aug 4, 2015 at 9:17 AM, Jun Rao <j...@confluent.io> wrote: > >> A few questions/comments. >> >> 1. What's streamTime passed to punctuate()? Is that just the current time? >> 2. Is punctuate() only called if schedule() is called? >> 3. The way the KeyValueStore is created seems a bit weird. Since this is >> part of the internal state managed by KafkaProcessorContext, it seems >> there >> should be an api to create the KeyValueStore from KafkaProcessorContext, >> instead of passing context to the constructor of KeyValueStore? >> >> Thanks, >> >> Jun >> >> On Thu, Jul 23, 2015 at 5:59 PM, Guozhang Wang <wangg...@gmail.com> >> wrote: >> >> > Hi all, >> > >> > I just posted KIP-28: Add a transform client for data processing >> > < >> > >> https://cwiki.apache.org/confluence/display/KAFKA/KIP-28+-+Add+a+transform+client+for+data+processing >> > > >> > . >> > >> > The wiki page does not yet have the full design / implementation >> details, >> > and this email is to kick-off the conversation on whether we should add >> > this new client with the described motivations, and if yes what >> features / >> > functionalities should be included. >> > >> > Looking forward to your feedback! >> > >> > -- Guozhang >> > >> > > > > -- > -- Guozhang > -- -- Guozhang
