Steve & Adam, I would be interesting in hearing the outcome here as well. I am seeing some similar issues in my 1.4.1 pipeline, using stateful functions (reduceByKeyAndWindow and updateStateByKey).
Regards, Bryan Jeffrey On Mon, Mar 14, 2016 at 6:45 AM, Steve Loughran <ste...@hortonworks.com> wrote: > > > On 14 Mar 2016, at 09:41, adamreith <r.corbe...@reply.it> wrote: > > > > I dumped the heap of the driver process and seems that 486.2 MB on 512 > MB of > > the available memory is used by an instance of the class > > /org.apache.spark.deploy.yarn.history.YarnHistoryService/. I'm trying to > > figure out how to solve the issue but till now i didn't found a solution. > > > > Could someone help me to sort out the issue? > > > > I've emailed you direct to look @ this > > --------------------------------------------------------------------- > To unsubscribe, e-mail: user-unsubscr...@spark.apache.org > For additional commands, e-mail: user-h...@spark.apache.org > >
