Re: mapWithState and context start when checkpoint exists

Hey Andrey, `ConstantInputDStream` doesn't support checkpoint as it contains an RDD field. It cannot resume from checkpoints. On Mon, Jan 25, 2016 at 1:12 PM, Andrey Yegorov wrote: > Hi, > > I am new to spark (and scala) and hope someone can help me with the issue > I

mapWithState and context start when checkpoint exists

Hi, I am new to spark (and scala) and hope someone can help me with the issue I got stuck on in my experiments/learning. mapWithState from spark 1.6 seems to be a great way for the task I want to implement with spark but unfortunately I am getting error "RDD transformations and actions can only

Re: mapWithState and context start when checkpoint exists

Thank you! what would be the best alternative to simulate a stream for testing purposes from e.g. sequence or a text file? In production I'll use kafka as a source but locally I wanted to mock it. Worst case scenario I'll have setup/tear down kafka cluster in tests but I think having a mock will