[
https://issues.apache.org/jira/browse/TEZ-986?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14377568#comment-14377568
]
Rohini Palaniswamy commented on TEZ-986:
----------------------------------------
bq. viewable in Tez UI after the job completes. This is very essential for
debugging jobs.
Just wanted to mention that we need this for Pig and would be good to have
it in one of the upcoming releases. While debugging some of the recent issues,
realized that I don't have access to pig script if user ran it from a gateway
(for Oozie I can get it from launcher job), because pig.script setting is only
set on the DAG config and few other settings useful for debugging like pig
version. For now, I have other workarounds to get this info or resort to asking
the user.
The vertex config also has some important debugging info like what feature is
being run (group by, etc), input/output dirs, etc. Even for this can manage for
the short term and figure these out with the explain output of the script. But
life would be easier if those are shown in the UI.
> Make conf set on DAG and vertex available in jobhistory
> -------------------------------------------------------
>
> Key: TEZ-986
> URL: https://issues.apache.org/jira/browse/TEZ-986
> Project: Apache Tez
> Issue Type: Sub-task
> Components: UI
> Reporter: Rohini Palaniswamy
> Priority: Blocker
>
> Would like to have the conf set on DAG and Vertex
> 1) viewable in Tez UI after the job completes. This is very essential for
> debugging jobs.
> 2) We have processes, that parse jobconf.xml from job history (hdfs) and
> load them into hive tables for analysis. Would like to have Tez also make all
> the configuration (byte array) available in job history so that we can
> similarly parse them. 1) mandates that you store it in hdfs. 2) is just to
> say make the format stored as a contract others can rely on for parsing.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
