Looks like the spark history server should take the lost exectuors into
account by analyzing the output from 'yarn logs applicationId' command.
Cheers
On Thu, Oct 1, 2015 at 11:46 AM, Lan Jiang wrote:
> Ted,
>
> Thanks for your reply.
>
> First of all, after sending email to the mailing list,
Ted,
Thanks for your reply.
First of all, after sending email to the mailing list, I use yarn logs
applicationId to retrieve the aggregated log
successfully. I found the exceptions I am looking for.
Now as to your suggestion, when I go to the YARN RM UI, I can only see the
"Tracking URL" in t
Can you go to YARN RM UI to find all the attempts for this Spark Job ?
The two lost executors should be found there.
On Thu, Oct 1, 2015 at 10:30 AM, Lan Jiang wrote:
> Hi, there
>
> When running a Spark job on YARN, 2 executors somehow got lost during the
> execution. The message on the histor
Hi, there
When running a Spark job on YARN, 2 executors somehow got lost during the
execution. The message on the history server GUI is “CANNOT find address”. Two
extra executors were launched by YARN and eventually finished the job. Usually
I go to the “Executors” tab on the UI to check the e