e the executor memory overhead.
Good luck.
-Mike
From: Sandy Ryza
To: Umesh Kacha
Cc: "user@spark.apache.org"
Sent: Thursday, August 20, 2015 5:21 PM
Subject: Re: How to avoid executor time out on yarn spark while dealing with
large shuffle skewed data?
GC wouldn't n
>
>>>>>>> cluster.yarnScheduler : Removing an executor 14 65 timeout
>>>>>>> exceeds
>>>>>>> 60 seconds
>>>>>>>
>>>>>>> and because of above error couple of chained errors starts to com
;>>>> peer,
>>>>>> IOException etc
>>>>>>
>>>>>> Please see the following UI page I have noticed when shuffle
>>>>>> read/write
>>>>>> starts to increase more than 10 GB executors starts getting lost
>&
>>>>> IOException etc
>>>>>
>>>>> Please see the following UI page I have noticed when shuffle read/write
>>>>> starts to increase more than 10 GB executors starts getting lost
>>>>> because of
>>>>> timeout
> Please
> guide.
>
> IMG_20150819_231418358.jpg
> <
> http://apache-spark-user-list.1001560.n3.nabble.com/file/n24345/IMG_20150819_231418358.jpg
> >
>
>
>
>
> --
> View this message in context:
> http://apache-spark-user-list.1001560.n3.nabble.com/H
k-user-list.1001560.n3.nabble.com/How-to-avoid-executor-time-out-on-yarn-spark-while-dealing-with-large-shuffle-skewed-data-tp24345.html
Sent from the Apache Spark User List mailing list archive at Nabble.com.
-
To unsubscribe,
