Re: Kafka-streams: mix Processor API with windowed grouping

2018-04-07 Thread Matthias J. Sax
>> ok, question then - is it possible to use state store with .aggregate()? Not sure what you exactly mean by this. An aggregations always uses a store; it's a stateful operation and cannot be computed without a store. For TopN, if you get the hit-count as input, you can use a `.aggregate()`

Re: kafka in unrecoverable state

2018-04-07 Thread SenthilKumar K
I observed below error in one of the broker , and it is unresponsive ... 2018-04-07 12:51:39,830] ERROR [Replica Manager on Broker 3]: Error processing append operation on partition __consumer_offsets-27 (kafka.server.ReplicaManager) org.apache.kafka.common.errors.NotEnoughReplicasException:

Re: Kafka Streams Internal Topic Retention not applied

2018-04-07 Thread Björn Häuser
Hello Matthias, thank you very much for your patience. I am still trying to understand the complete picture here. So, a streams internal topic for aggregation will be of cleanup.policy = compact. Which means that while doing aggregation ignoring tombstones records will cause havoc, right?