[jira] [Commented] (TEZ-986) Make conf set on DAG and vertex available in jobhistory
[ https://issues.apache.org/jira/browse/TEZ-986?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14378304#comment-14378304 ] Hitesh Shah commented on TEZ-986: - [~rohini] Agreed. Just moved the target version as I am not sure if [~Sreenath] has had a chance to work on it and given that [~jeagles] is looking to turn around a 0.6.1 release soon, this jira should likely be moved out if no one volunteers to work on it within a short timeframe. > Make conf set on DAG and vertex available in jobhistory > --- > > Key: TEZ-986 > URL: https://issues.apache.org/jira/browse/TEZ-986 > Project: Apache Tez > Issue Type: Sub-task > Components: UI >Reporter: Rohini Palaniswamy >Priority: Blocker > > Would like to have the conf set on DAG and Vertex > 1) viewable in Tez UI after the job completes. This is very essential for > debugging jobs. > 2) We have processes, that parse jobconf.xml from job history (hdfs) and > load them into hive tables for analysis. Would like to have Tez also make all > the configuration (byte array) available in job history so that we can > similarly parse them. 1) mandates that you store it in hdfs. 2) is just to > say make the format stored as a contract others can rely on for parsing. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (TEZ-986) Make conf set on DAG and vertex available in jobhistory
[ https://issues.apache.org/jira/browse/TEZ-986?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14377568#comment-14377568 ] Rohini Palaniswamy commented on TEZ-986: bq. viewable in Tez UI after the job completes. This is very essential for debugging jobs. Just wanted to mention that we need this for Pig and would be good to have it in one of the upcoming releases. While debugging some of the recent issues, realized that I don't have access to pig script if user ran it from a gateway (for Oozie I can get it from launcher job), because pig.script setting is only set on the DAG config and few other settings useful for debugging like pig version. For now, I have other workarounds to get this info or resort to asking the user. The vertex config also has some important debugging info like what feature is being run (group by, etc), input/output dirs, etc. Even for this can manage for the short term and figure these out with the explain output of the script. But life would be easier if those are shown in the UI. > Make conf set on DAG and vertex available in jobhistory > --- > > Key: TEZ-986 > URL: https://issues.apache.org/jira/browse/TEZ-986 > Project: Apache Tez > Issue Type: Sub-task > Components: UI >Reporter: Rohini Palaniswamy >Priority: Blocker > > Would like to have the conf set on DAG and Vertex > 1) viewable in Tez UI after the job completes. This is very essential for > debugging jobs. > 2) We have processes, that parse jobconf.xml from job history (hdfs) and > load them into hive tables for analysis. Would like to have Tez also make all > the configuration (byte array) available in job history so that we can > similarly parse them. 1) mandates that you store it in hdfs. 2) is just to > say make the format stored as a contract others can rely on for parsing. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (TEZ-986) Make conf set on DAG and vertex available in jobhistory
[ https://issues.apache.org/jira/browse/TEZ-986?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14377271#comment-14377271 ] Hitesh Shah commented on TEZ-986: - Moving this out to 0.6.2. Not sure if [~Sreenath] has had a chance to look at this jira. > Make conf set on DAG and vertex available in jobhistory > --- > > Key: TEZ-986 > URL: https://issues.apache.org/jira/browse/TEZ-986 > Project: Apache Tez > Issue Type: Sub-task > Components: UI >Reporter: Rohini Palaniswamy >Priority: Blocker > > Would like to have the conf set on DAG and Vertex > 1) viewable in Tez UI after the job completes. This is very essential for > debugging jobs. > 2) We have processes, that parse jobconf.xml from job history (hdfs) and > load them into hive tables for analysis. Would like to have Tez also make all > the configuration (byte array) available in job history so that we can > similarly parse them. 1) mandates that you store it in hdfs. 2) is just to > say make the format stored as a contract others can rely on for parsing. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
