[jira] [Commented] (YARN-3670) Add JobConf XML link in Yarn RM UI
[ https://issues.apache.org/jira/browse/YARN-3670?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14548047#comment-14548047 ] Jason Lowe commented on YARN-3670: -- Sure, we could solve this by adding APIs for applications to convey job-specific settings or metrics. However this is far from a minor improvement and could be quite involved, including questions like where will these settings be stored (probably timelineserver), limits on the key/value sizes, limits on total amount of data that can be stored, etc. Add JobConf XML link in Yarn RM UI -- Key: YARN-3670 URL: https://issues.apache.org/jira/browse/YARN-3670 Project: Hadoop YARN Issue Type: Improvement Components: webapp Affects Versions: 2.6.0 Environment: HDP 2.2 Reporter: Hari Sekhon Priority: Minor Request to add JobConf xml link for each application in the RM UI so I don't have to keep recovering it from HDFS to debug if job settings are taking effect. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-3670) Add JobConf XML link in Yarn RM UI
[ https://issues.apache.org/jira/browse/YARN-3670?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14548039#comment-14548039 ] Hari Sekhon commented on YARN-3670: --- I understand (you can't imagine how often I've had to refrain from asking for Maps and Reduces to be available in UI like trusty old MRv1). At the moment it's too opaque, which is a first generation side-effect of Yarn being a generally abstracting resource manager. However, perhaps it would be possible to create a generic mechanism that allows jobs to publish key-value pairs which could be populated with either settings or counters to be displayed via a new named link also specified per set of key-value pairs, such as a job could publish counters or configuration. In this way both the MapReduce client, the Tez client and any other YARN apps would have the ability to publish any arbitrary key-value pair table of information to YARN to display in the UI? This would help immensely with debugging yarn jobs. Add JobConf XML link in Yarn RM UI -- Key: YARN-3670 URL: https://issues.apache.org/jira/browse/YARN-3670 Project: Hadoop YARN Issue Type: Improvement Components: webapp Affects Versions: 2.6.0 Environment: HDP 2.2 Reporter: Hari Sekhon Priority: Minor Request to add JobConf xml link for each application in the RM UI so I don't have to keep recovering it from HDFS to debug if job settings are taking effect. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-3670) Add JobConf XML link in Yarn RM UI
[ https://issues.apache.org/jira/browse/YARN-3670?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14548020#comment-14548020 ] Jason Lowe commented on YARN-3670: -- This is not easily implementable in the general case, since not all YARN applications have a job conf that is an XML file and accessible via a link. While this is true for MapReduce jobs, YARN is not MapReduce-specific. This is similar to asking for the number of maps and reduces to be available on the RM UI. Add JobConf XML link in Yarn RM UI -- Key: YARN-3670 URL: https://issues.apache.org/jira/browse/YARN-3670 Project: Hadoop YARN Issue Type: Improvement Components: webapp Affects Versions: 2.6.0 Environment: HDP 2.2 Reporter: Hari Sekhon Priority: Minor Request to add JobConf xml link for each application in the RM UI so I don't have to keep recovering it from HDFS to debug if job settings are taking effect. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
