Re: KTables + aggregation - how to make lots of ppl happy

2016-12-02 Thread Guozhang Wang
Thanks Jon for bringing this up. We have seen the community discussing about explicit triggers for Kafka Streams, and some of them have been covered in KIP-63: https://cwiki.apache.org/confluence/display/KAFKA/KIP-63%3A+Unify+store+and+downstream+caching+in+streams Guozhang On Thu, Dec 1, 20

KTables + aggregation - how to make lots of ppl happy

2016-12-01 Thread Jon Yeargers
Seems like there are many questions on SO and related about "how do I know when my windowed aggregation is 'done'?" The answer has always been "It's never done. You're thinking about it the wrong way." I propose a new function for KStream: finiteAggregateByKey(Initializer, Aggregator, Windows, Ti