[
https://issues.apache.org/jira/browse/TEZ-2076?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14336930#comment-14336930
]
Bikas Saha commented on TEZ-2076:
---------------------------------
[~rajesh.balamohan] For the benefit of folks interested in this, could you
please paint an overall picture of what is available post this patch and what
needs to be done by the developer/user to use the facilities provided by this
patch? I quickly scanned the patch but did not see any documentation for the
website or in any other form such as a readme etc. So having a description on
the jira would help provide a logical understanding. If the code is stable
enough then perhaps we can add some formal documentation that could be
committed or even added to the website section of the docs.
Essentially, the patch is enabling a library to download events from ATS
(SimpleHistoryFile from HDFS also?) and using that data to create a
post-execution Java model of the DAG for further custom post-processing that is
user defined?
> Tez framework to extract/analyze data stored in ATS for specific dag
> --------------------------------------------------------------------
>
> Key: TEZ-2076
> URL: https://issues.apache.org/jira/browse/TEZ-2076
> Project: Apache Tez
> Issue Type: Improvement
> Reporter: Rajesh Balamohan
> Assignee: Rajesh Balamohan
> Attachments: TEZ-2076.1.patch, TEZ-2076.2.patch, TEZ-2076.3.patch,
> TEZ-2076.WIP.2.patch, TEZ-2076.WIP.3.patch, TEZ-2076.WIP.patch
>
>
> - Users should be able to download ATS data pertaining to a DAG from Tez-UI
> (more like a zip file containing DAG/Vertex/Task/TaskAttempt info).
> - This can be plugged to an analyzer which parses the data, adds semantics
> and provides an in-memory representation for further analysis.
> - This will enable to write different analyzer rules, which can be run on top
> of this in-memory representation to come up with analysis on the DAG.
> - Results of this analyzer rules can be rendered on to UI (standalone webapp)
> later point in time.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
