Re: Spark Yarn closing sparkContext

2016-04-14 Thread Ted Yu
Can you pastebin the failure message ? Did you happen to take jstack during the close ? Which Hadoop version do you use ? Thanks > On Apr 14, 2016, at 5:53 AM, nihed mbarek wrote: > > Hi, > I have an issue with closing my application context, the process take a long > time with a fail at t

Spark Yarn closing sparkContext

2016-04-14 Thread nihed mbarek
Hi, I have an issue with closing my application context, the process take a long time with a fail at the end. In other part, my result was generate in the write folder and _SUCESS file was created. I'm using spark 1.6 with yarn. any idea ? regards, -- MBAREK Med Nihed, Fedora Ambassador, TUNIS