[
https://issues.apache.org/jira/browse/TEZ-2278?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15210921#comment-15210921
]
Hitesh Shah commented on TEZ-2278:
----------------------------------
\cc [~jlowe] [~rohini] [~jeagles] [~bikassaha] [~sseth] [~Sreenath] [~hagleitn]
[~gopalv] [~rajesh.balamohan]
I guess the main question is how we calculate duration for a vertex and task:
- For a task, there is a point when the task is schedulable, then when first
attempt is launched, when it goes into a running state and eventually when the
task attempt succeeds and task finishes.
- For a vertex, there is initialization, followed by a wait when it is ready
to run ( sources are started ), when the first task really gets scheduled and
started, all tasks finished, commit phase which eventually implies vertex
completion.
There are various ways we can consider duration of the object in question vs
actual run time, etc. I have cc'ed most of the folks who usually look at this
data from a higher level and use it for debugging, perf analysis, etc. Lets try
and come to a conclusion on what makes sense and we can change the UI to use
that as a default.
Options:
Task Duration:
- first task attempt schedulable to final task attempt succeeded ( accounts
for both scheduling delays and actual run time )
- actual run time of i.e. time taken by only the successful task attempt
- actual run time of all task attempts
Vertex Duration:
- total task attempt run time i.e. first task attempt start to last task
attempt finish
- init time + above run time
Other options/comments? Most of the above do not really consider gaps in time
such as retrospective failures, AM recovery, vertex re-runs after completing so
those would need some thought as well.
> Tez UI start/end time and duration shown are wrong for tasks
> ------------------------------------------------------------
>
> Key: TEZ-2278
> URL: https://issues.apache.org/jira/browse/TEZ-2278
> Project: Apache Tez
> Issue Type: Bug
> Components: UI
> Affects Versions: 0.6.0
> Reporter: Rohini Palaniswamy
> Assignee: Jason Lowe
> Attachments: TEZ-2278.001.patch, screenshot-1.png, screenshot-2.png,
> screenshot-3.png, screenshot-4.png
>
>
> Observing lot of time discrepancies between vertex, task and swinlane views.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
