Re: Kafka Stream Deadman

2017-10-08 Thread Guozhang Wang
Hello Trevor, You may be interested in the punctuate API of the processor: https://kafka.apache.org/0110/javadoc/index.html?org/apache/kafka/streams/processor/Processor.html#punctuate Note that in the coming 1.0.0 version users can schedule a punctuate function either based on stream time

Re: Improving Kafka State Store performance

2017-10-08 Thread Guozhang Wang
Hello, Since you are running on EBS not SSD, the first suspicion I'd have is its write and storage amplification. This can possibly be verified from RocksDB's own stats, and Bill once shared the code to expose such metrics for investigaion:

Re: NPE on ConsumerRecords$ConcatenatedIterable$1.makeNext() while iterating records

2017-10-08 Thread Ted Yu
Was there any consumer interceptor involved ? Cheers On Sun, Oct 8, 2017 at 6:29 AM, Michael Keinan wrote: > Hi > Using Kafka 0.10.2.0 > I get a NPE while iterating the records after polling them using poll > method. > - Any idea where does it come from ? > - How can I

NPE on ConsumerRecords$ConcatenatedIterable$1.makeNext() while iterating records

2017-10-08 Thread Michael Keinan
Hi Using Kafka 0.10.2.0 I get a NPE while iterating the records after polling them using poll method. - Any idea where does it come from ? - How can I open an issue to Kafka team ? Stacktrace: java.lang.NullPointerException at

Re: Kafka Streams with embedded RocksDB

2017-10-08 Thread Sachin Mittal
I have checked rocksdbjni-5.0.1.jar included in kafka_2.12-0.11.0.0.tgz and librocksdbjni dll is included. I don't think you have to build anything extra fir windows OS. On Sat, Oct 7, 2017 at 11:39 PM, Valentin Forst wrote: > Hi Ted, > > Thank you for your reply... > Yes,