> Subject: Re: Origin of failed tasks
> From: hit...@apache.org
> Date: Wed, 12 Oct 2016 08:57:12 -0700
> To: firstname.lastname@example.org
> If you have the logs for the application master, you can try the following:
> grep “[HISTORY]” | grep “TASK_ATTEMPT_FINISHED”
> This will give you info on any failed task attempts.
> The AM logs have history events being published to them. You can do grep
> “[HISTORY]” | grep “<entity type>_<event type>” where entity type is one of
> DAG, VERTEX, TASK, TASK_ATTEMPT and event type is STARTED or FINISHED.
> The logs are also split into diff files. e.g.
> The AM logs use a syslog_dag… format to split across dags.
> Task/Container logs use syslog_attempt* format to split out logs for
> different task attempts.
> If you have YARN timeline enabled, you can use the analyzers to do more
> analysis on the dag specific data. These are more related to perf tuning and
> not failure diagnostics though.
> ― Hitesh
> > On Oct 11, 2016, at 5:09 PM, Allan Wilson <wilsoncr...@gmail.com> wrote:
> > Use the yarn logs command. That's your only chance without the TEZ UI. I
> > setup the TEZ UI
> > In our shop and it is really nice.
> > Allan
> > Sent from my iPhone
> >> On Oct 11, 2016, at 5:05 PM, Jan Morlock <jan.morl...@googlemail.com>
> >> wrote:
> >> Hi,
> >> currently failed tasks occur during the execution of my Hive/Tez job.
> >> However in the end, the overall job succeeds. Is it possible to find out
> >> afterwards about the origin of those failed tasks (without using the Tez
> >> UI) just by analyzing the output log files?
> >> Best regards
> >> Jan