Re: Driver not able to restart the job automatically after the application of Streaming with Kafka Direct went down

2016-02-05 Thread swetha kasireddy
Following is the error that I see when it retries. org.apache.spark.SparkException: Failed to read checkpoint from directory /share/checkpointDir at org.apache.spark.streaming.CheckpointReader$.read(Checkpoint.scala:342) at

Driver not able to restart the job automatically after the application of Streaming with Kafka Direct went down

2016-02-04 Thread SRK
Hi, I have the Streaming job running in qa/prod. Due to Kafka issues both the jobs went down. After the Kafka issues got resolved and after the deletion of the checkpoint directory the driver in the qa job restarted the job automatically and the application UI was up. But, in the prod job, the