[
https://issues.apache.org/jira/browse/TEZ-2690?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14702282#comment-14702282
]
Bikas Saha edited comment on TEZ-2690 at 8/19/15 1:39 AM:
----------------------------------------------------------
Adds a task attempt level critical path analyzer.
Uses the scheduling and data event dependencies to walk from the last attempt
completion to first attempt creation to account for the time taken in the job.
The output of the analyzer is an svg rendering of the critical path. Attached
sample. The svg code has been re-written to generate svg directly instead of
using jaxb because of missing features in jaxb (e.g. setting the value of a
text field).
Renames existing critical path analyzer to vertex level.
Adds an AnalyzerDriver to allow running analyzers from the command line using
hadoop jar command. Only the latest CriticalPathAnalyzer has been added to the
driver because I am not sure how the other analyzers would behave on the
command line. They are written to output csv results. Perhaps we can create a
base Csv analyzer that can take the csv results and output them on the console
or write them to a file. Then they could be run on the command line.
The goal is to get this in and have motivated developers start running it and
finding issues/improvements.
[~rajesh.balamohan] Please review.
!criticalPath.png|thumbnail!
was (Author: bikassaha):
Adds a task attempt level critical path analyzer.
Uses the scheduling and data event dependencies to walk from the last attempt
completion to first attempt creation to account for the time taken in the job.
The output of the analyzer is an svg rendering of the critical path. Attached
sample. The svg code has been re-written to generate svg directly instead of
using jaxb because of missing features in jaxb (e.g. setting the value of a
text field).
Renames existing critical path analyzer to vertex level.
Adds an AnalyzerDriver to allow running analyzers from the command line using
hadoop jar command. Only the latest CriticalPathAnalyzer has been added to the
driver because I am not sure how the other analyzers would behave on the
command line. They are written to output csv results. Perhaps we can create a
base Csv analyzer that can take the csv results and output them on the console
or write them to a file. Then they could be run on the command line.
The goal is to get this in and have motivated developers start running it and
finding issues/improvements.
[~rajesh.balamohan] Please review.
> Add critical path analyser
> --------------------------
>
> Key: TEZ-2690
> URL: https://issues.apache.org/jira/browse/TEZ-2690
> Project: Apache Tez
> Issue Type: Sub-task
> Reporter: Bikas Saha
> Assignee: Bikas Saha
> Attachments: TEZ-2690.1.patch, criticalPath.png
>
>
> Use input and scheduling dependencies to create critical path for a DAG.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
