Somewhat related, though this JIRA is on 1.6. https://issues.apache.org/jira/browse/SPARK-13288#
JESSE CHEN
Big Data Performance | IBM Analytics
Office: 408 463 2296
Mobile: 408 828 9068
Email: [email protected]
From: Bryan Jeffrey <[email protected]>
To: Steve Loughran <[email protected]>
Cc: adamreith <[email protected]>, "[email protected]"
<[email protected]>
Date: 03/14/2016 06:33 AM
Subject: Re: OOM Exception in my spark streaming application
Steve & Adam,
I would be interesting in hearing the outcome here as well. I am seeing
some similar issues in my 1.4.1 pipeline, using stateful functions
(reduceByKeyAndWindow and updateStateByKey).
Regards,
Bryan Jeffrey
On Mon, Mar 14, 2016 at 6:45 AM, Steve Loughran <[email protected]>
wrote:
> On 14 Mar 2016, at 09:41, adamreith <[email protected]> wrote:
>
> I dumped the heap of the driver process and seems that 486.2 MB on 512
MB of
> the available memory is used by an instance of the class
> /org.apache.spark.deploy.yarn.history.YarnHistoryService/. I'm trying
to
> figure out how to solve the issue but till now i didn't found a
solution.
>
> Could someone help me to sort out the issue?
>
I've emailed you direct to look @ this
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
