Rajesh Balamohan created TEZ-2612:
-------------------------------------
Summary: Critical path analyzer for DAGs
Key: TEZ-2612
URL: https://issues.apache.org/jira/browse/TEZ-2612
Project: Apache Tez
Issue Type: Bug
Reporter: Rajesh Balamohan
This analyzer plugin/tool can be used to understand the important
vertices/tasks of interest in large DAG for perf analysis / finding
bottlenecks. It can be used to find out 1.input dependency, 2. failure
dependency, 3. scheduling dependency (may be at later stage).
Creating this as a uber ticket. Getting this detail at vertex level might be
possible with the existing logs derived from ATS. For task level analysis,
certain more details are required.
1. Timeline details like when fetch/merge/compute/sort etc are not captured
now. These details can possibly be added in TaskCompletionEvent
2. Need additional details like the last event that completed processing in the
input (for tracing at task level)
3. Add downstream task attempt that caused the higher level task to get
rescheduled/restarted. This can be used in terms of understanding in cases
where the task failed due to read-error.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
