> Subject: Re: Origin of failed tasks
> From: hit...@apache.org
> Date: Wed, 12 Oct 2016 08:57:12 -0700
> To: user@tez.apache.org
> 
> If you have the logs for the application master, you can try the following: 
> 
> grep “[HISTORY]” | grep “TASK_ATTEMPT_FINISHED”
> 
> This will give you info on any failed task attempts. 
> 
> The AM logs have history events being published to them. You can do grep 
> “[HISTORY]” | grep “<entity type>_<event type>” where entity type is one of 
> DAG, VERTEX, TASK, TASK_ATTEMPT and event type is STARTED or FINISHED. 
> 
> The logs are also split into diff files. e.g. 
> The AM logs use a syslog_dag… format to split across dags. 
> Task/Container logs use syslog_attempt* format to split out logs for 
> different task attempts. 
> 
> If you have YARN timeline enabled, you can use the analyzers to do more 
> analysis on the dag specific data. These are more related to perf tuning and 
> not failure diagnostics though.
> 
> thanks
> ― Hitesh
> 
> 
> > On Oct 11, 2016, at 5:09 PM, Allan Wilson <wilsoncr...@gmail.com> wrote:
> > 
> > Use the yarn logs command.  That's your only chance without the TEZ UI.  I 
> > setup the TEZ UI
> > In our shop and it is really nice.
> > 
> > Allan
> > Sent from my iPhone
> > 
> >> On Oct 11, 2016, at 5:05 PM, Jan Morlock <jan.morl...@googlemail.com> 
> >> wrote:
> >> 
> >> Hi,
> >> 
> >> currently failed tasks occur during the execution of my Hive/Tez job.
> >> However in the end, the overall job succeeds. Is it possible to find out
> >> afterwards about the origin of those failed tasks (without using the Tez
> >> UI) just by analyzing the output log files?
> >> 
> >> Best regards
> >> Jan
> 
                                          

Reply via email to