Hi, is there any provision in spark batch for checkpoint.
I am having huge data, it takes more than 3 hours to process all data. I am currently having 100 partitions. if the job fails after two hours, lets say it has processed 70 partition. should i start spark job from the beginning or is there way for checkpoint provision. Checkpoint,what i am expecting is start from 71 partition to till end. Please give me your suggestions. -- Selvam Raman "லஞ்சம் தவிர்த்து நெஞ்சம் நிமிர்த்து"
