[
https://issues.apache.org/jira/browse/TEZ-2076?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14335484#comment-14335484
]
Gopal V commented on TEZ-2076:
------------------------------
{code}
scala> import java.io._;
import java.io._
scala> import org.apache.tez.history.parser._;
import org.apache.tez.history.parser._
scala> val p = new ATSFileParser(new File("dag_1421164610335_0820_1.zip"))
p: org.apache.tez.history.parser.ATSFileParser =
org.apache.tez.history.parser.ATSFileParser@25d0cb3a
scala> val d = p.getDAGData("dag_1421164610335_0820_1")
d: org.apache.tez.history.parser.DagInfo = [dagID=dag_1421164610335_0820_1,
dagName=rajesh_20150210175454_bec56061-02d6-4730-bdf6-89a455625a0f:1,
status=SUCCEEDED, startTime=0, submitTime=-1423619702215, endTime=1010246,
timeTaken=1010246, diagnostics=, vertexNameIDMapping={Map
2=vertex_1421164610335_0820_1_03, Reducer 3=vertex_1421164610335_0820_1_04, Map
1=vertex_1421164610335_0820_1_02, Map 4=vertex_1421164610335_0820_1_01, Map
5=vertex_1421164610335_0820_1_00}, failedTasks=0, events=[], status=SUCCEEDED]
scala> println(d.getTimeTaken())
1010246
scala> d.getFailedTasks()
res6: Int = 0
scala> d.getContainersToTaskAttemptMapping()
res7:
com.google.common.collect.Multimap[org.apache.tez.history.parser.Container,org.apache.tez.history.parser.TaskAttemptInfo]
= {[id=container_1421164610335_0820_01_000074,
host=cn053-10.l42scl.hortonworks.com]=[[taskAttemptId=attempt_1421164610335_0820_1_03_000106_0,
scheduledTime=0, startTime=1423619705486, finishTime=1423620148368,
timeTaken=442882, events=[[info={}, type=TASK_ATTEMPT_FINISHED,
time=1423620148368], [info={}, type=TASK_ATTEMPT_STARTED, time=1423619705486]],
diagnostics=, successfulAttempId=,
container=[id=container_1421164610335_0820_01_000074,
host=cn053-10.l42scl.hortonworks.com], nodeId=cn053-10.l42scl.hortonworks.com,
logURL=http://cn042-10.l42scl.hortonworks.com:19888/jobhistory/logs/cn053-10.l42scl.hortonworks.com:43277/container_1421164610335_0820_01_000074/v...
{code}
> Tez framework to extract/analyze data stored in ATS for specific dag
> --------------------------------------------------------------------
>
> Key: TEZ-2076
> URL: https://issues.apache.org/jira/browse/TEZ-2076
> Project: Apache Tez
> Issue Type: Improvement
> Reporter: Rajesh Balamohan
> Assignee: Rajesh Balamohan
> Attachments: TEZ-2076.WIP.2.patch, TEZ-2076.WIP.3.patch,
> TEZ-2076.WIP.patch
>
>
> - Users should be able to download ATS data pertaining to a DAG from Tez-UI
> (more like a zip file containing DAG/Vertex/Task/TaskAttempt info).
> - This can be plugged to an analyzer which parses the data, adds semantics
> and provides an in-memory representation for further analysis.
> - This will enable to write different analyzer rules, which can be run on top
> of this in-memory representation to come up with analysis on the DAG.
> - Results of this analyzer rules can be rendered on to UI (standalone webapp)
> later point in time.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
