Re: Unable to catch SparkContext methods exceptions

2015-08-24 Thread Burak Yavuz
textFile is a lazy operation. It doesn't evaluate until you call an action on it, such as .count(). Therefore, you won't catch the exception there. Best, Burak On Mon, Aug 24, 2015 at 9:09 AM, Roberto Coluccio roberto.coluc...@gmail.com wrote: Hello folks, I'm experiencing an unexpected

Re: Unable to catch SparkContext methods exceptions

2015-08-24 Thread Burak Yavuz
The laziness is hard to deal with in these situations. I would suggest trying to handle expected cases FileNotFound, etc using other methods before even starting a Spark job. If you really want to try.catch a specific portion of a Spark job, one way is to just follow it with an action. You can