Re: Understanding Spark Task failures

2016-01-28 Thread Tathagata Das
That is hard to guarantee by the system, and it is upto the app developer to ensure that this is not . For example, if the data in a message is corrupted, unless the app code is robust towards handling such data, the system will fail every time it retries that app code. On Thu, Jan 28, 2016 at

Re: Understanding Spark Task failures

2016-01-28 Thread Patrick McGloin
Hi Tathagata, Thanks for the response. I can add in a try catch myself and handle user exceptions, that's true, so maybe my example wasn't a very good one. I'm more worried about OOM exceptions and other run-time exceptions (that could happen outside my try catch). For example, I have this