[
https://issues.apache.org/jira/browse/TEZ-2076?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14549387#comment-14549387
]
Hitesh Shah commented on TEZ-2076:
----------------------------------
Minor comments:
{code}
String help = LINE_SEPARATOR
385 + "java -jar tez-history-parser-x.y.z-jar-with-dependencies.jar"
386 + LINE_SEPARATOR
387 + "OR"
388 + LINE_SEPARATOR
389 + "java -cp tez-history-parser-x.y.z-jar-with-dependencies.jar
org.apache.tez.history.ATSImportTool"
390 + LINE_SEPARATOR
391 + "OR"
392 + LINE_SEPARATOR
393 +
"HADOOP_CLASSPATH=$TEZ_HOME/*:$TEZ_HOME/lib/*:$HADOOP_CLASSPATH hadoop jar "
394 + "tez-history-parser-x.y.z.jar " +
ATSImportTool.class.getName()
395 + LINE_SEPARATOR;
396 formatter.printHelp(240, help, "Options",
{code}
- Not sure why there are repetitive options ( with/without class name ) - can
this be reduced to 2 i.e. one via java -cp and the other via hadoop jar?
Also, [~gopalv] raised this point offline too. There needs to be a way to warn
the user if the import tool is used on an in-progress dag - maybe this tool
should just error out for an in-progress dag and only work if there is a
special --allow-incomplete-data or similar flag set. Could be done in a
follow-up jira. \cc [~pramachandran] as the same is applicable to the UI
download tool.
> Tez framework to extract/analyze data stored in ATS for specific dag
> --------------------------------------------------------------------
>
> Key: TEZ-2076
> URL: https://issues.apache.org/jira/browse/TEZ-2076
> Project: Apache Tez
> Issue Type: Improvement
> Reporter: Rajesh Balamohan
> Assignee: Rajesh Balamohan
> Attachments: TEZ-2076.1.patch, TEZ-2076.10.patch, TEZ-2076.11.patch,
> TEZ-2076.12.patch, TEZ-2076.13.patch, TEZ-2076.14.patch, TEZ-2076.15.patch,
> TEZ-2076.2.patch, TEZ-2076.3.patch, TEZ-2076.4.patch, TEZ-2076.5.patch,
> TEZ-2076.6.patch, TEZ-2076.7.patch, TEZ-2076.8.patch, TEZ-2076.9.patch,
> TEZ-2076.WIP.2.patch, TEZ-2076.WIP.3.patch, TEZ-2076.WIP.patch
>
>
> - Users should be able to download ATS data pertaining to a DAG from Tez-UI
> (more like a zip file containing DAG/Vertex/Task/TaskAttempt info).
> - This can be plugged to an analyzer which parses the data, adds semantics
> and provides an in-memory representation for further analysis.
> - This will enable to write different analyzer rules, which can be run on top
> of this in-memory representation to come up with analysis on the DAG.
> - Results of this analyzer rules can be rendered on to UI (standalone webapp)
> later point in time.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
