Re: Is there a way to control pipeline flow to downstream

Hi, Thanks for the link. What I understand is that when cache.max.bytes.buffering value is reached it will push the aggregation to downstream. What is the default value for the same? And how can I determine my cache size for current stream so as to set an optimal value. I also suppose the push

Re: Is there a way to control pipeline flow to downstream

Hi Sachin, This landed in 0.10.1, so the docs are at http://kafka.apache.org/0101/javadoc/index.html . This wiki has a good description of how this works:

Re: Is there a way to control pipeline flow to downstream

Hi, I checked the docs http://kafka.apache.org/0100/javadoc/index.html class StreamsConfig but did not find this CACHE_MAX_BYTES_BUFFERING_CONFIG setting. Also on the first option: use the record cache to dedup messages with the same key before sending downstream I did not understand this. How

Re: Is there a way to control pipeline flow to downstream

Hi Sachin, If you are using the DSL, currently there is no way to do fine-grained control of the downstream sending. There is some coarse-grained control in that you can use the record cache to dedup messages with the same key before sending downstream, or you can choose to get all records by

Is there a way to control pipeline flow to downstream

Hi all, Say I have a pipleline like this topic.aggregateByKey( ...) => to downstream Now for every message in topic it will call aggregateByKey and send it to downstream Is there a way to tell the pipeline that if it gets a certain message then only push the current aggregation result to