If you have the logs for the application master, you can try the following:
grep “[HISTORY]” | grep “TASK_ATTEMPT_FINISHED”
This will give you info on any failed task attempts.
The AM logs have history events being published to them. You can do grep
“[HISTORY]” | grep “<entity type>_<event type>” where entity type is one of
DAG, VERTEX, TASK, TASK_ATTEMPT and event type is STARTED or FINISHED.
The logs are also split into diff files. e.g.
The AM logs use a syslog_dag… format to split across dags.
Task/Container logs use syslog_attempt* format to split out logs for different
If you have YARN timeline enabled, you can use the analyzers to do more
analysis on the dag specific data. These are more related to perf tuning and
not failure diagnostics though.
> On Oct 11, 2016, at 5:09 PM, Allan Wilson <wilsoncr...@gmail.com> wrote:
> Use the yarn logs command. That's your only chance without the TEZ UI. I
> setup the TEZ UI
> In our shop and it is really nice.
> Sent from my iPhone
>> On Oct 11, 2016, at 5:05 PM, Jan Morlock <jan.morl...@googlemail.com> wrote:
>> currently failed tasks occur during the execution of my Hive/Tez job.
>> However in the end, the overall job succeeds. Is it possible to find out
>> afterwards about the origin of those failed tasks (without using the Tez
>> UI) just by analyzing the output log files?
>> Best regards