:dev@spark.apache.org>
Subject: Re: timeout in shuffle problem
Are you running on YARN? Another possibility here is that your shuffle managers
are facing GC pain and becoming less responsive, thus missing timeouts. Can you
try increasing the memory on the node managers and see if that helps?
On S
Are you running on YARN? Another possibility here is that your shuffle
managers are facing GC pain and becoming less responsive, thus missing
timeouts. Can you try increasing the memory on the node managers and see if
that helps?
On Sun, Jan 24, 2016 at 4:58 PM Ted Yu wrote:
> Cycling past bits:
Cycling past bits:
http://search-hadoop.com/m/q3RTtU5CRU1KKVA42&subj=RE+shuffle+FetchFailedException+in+spark+on+YARN+job
On Sun, Jan 24, 2016 at 5:52 AM, wangzhenhua (G)
wrote:
> Hi,
>
> I have a problem of time out in shuffle, it happened after shuffle write
> and at the start of shuffle read,
Hi,
I have a problem of time out in shuffle, it happened after shuffle write and at
the start of shuffle read,
logs on driver and executors are shown as below. Spark version is 1.5. Looking
forward to your replys. Thanks!
logs on driver only have warnings:
WARN TaskSetManager: Lost task 38.0 in
