Re: scala.NotImplementedError: put() should not be called on an EmptyStateMap while doing stateful computation on spark streaming

Are you using 1.6.1 ? If not, does the problem persist when you use 1.6.1 ? Thanks > On Jun 20, 2016, at 11:16 PM, umanga wrote: > > I am getting following warning while running stateful computation. The state > consists of BloomFilter (stream-lib) as Value and Integer

Re: scala.NotImplementedError: put() should not be called on an EmptyStateMap while doing stateful computation on spark streaming

further descriptions: Environment: Spark cluster running in standalone mode with 1 master, 5 slaves, each has 4 vCPUS, 8GB RAM data is being streamed from 3 node kafka cluster (managed by 3 node zk cluster). Checkpointing is being done at hadoop-cluster, plus we are also saving state in HBase

scala.NotImplementedError: put() should not be called on an EmptyStateMap while doing stateful computation on spark streaming

I am getting following warning while running stateful computation. The state consists of BloomFilter (stream-lib) as Value and Integer as key. The program runs smoothly for few minutes and after that, i am getting this warning, and streaming app becomes unstable (processing time increases