Any kind of changes to the jvm classes will make it fail. By checkpointing the data you mean using checkpoint with updateStateByKey? Here's a similar discussion happened earlier which will clear your doubts i guess http://mail-archives.us.apache.org/mod_mbox/spark-user/201507.mbox/%3CCA+AHuK=xoy8dsdaobmgm935goqytaaqkpqsvdaqpmojottj...@mail.gmail.com%3E
Thanks Best Regards On Thu, Sep 17, 2015 at 10:01 AM, Bin Wang <wbi...@gmail.com> wrote: > And here is another question. If I load the DStream from database every > time I start the job, will the data be loaded when the job is failed and > auto restart? If so, both the checkpoint data and database data are loaded, > won't this a problem? > > > > Bin Wang <wbi...@gmail.com>于2015年9月16日周三 下午8:40写道： > >> Will StreamingContex.getOrCreate do this work？What kind of code change >> will make it cannot load？ >> >> Akhil Das <ak...@sigmoidanalytics.com>于2015年9月16日周三 20:20写道： >> >>> You can't really recover from checkpoint if you alter the code. A better >>> approach would be to use some sort of external storage (like a db or >>> zookeeper etc) to keep the state (the indexes etc) and then when you deploy >>> new code they can be easily recovered. >>> >>> Thanks >>> Best Regards >>> >>> On Wed, Sep 16, 2015 at 3:52 PM, Bin Wang <wbi...@gmail.com> wrote: >>> >>>> I'd like to know if there is a way to recovery dstream from checkpoint. >>>> >>>> Because I stores state in DStream, I'd like the state to be recovered >>>> when I restart the application and deploy new code. >>>> >>> >>>
