[jira] [Commented] (YARN-3869) Add app name to RM audit log
[ https://issues.apache.org/jira/browse/YARN-3869?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14966673#comment-14966673 ] Shay Rojansky commented on YARN-3869: - Any chance we can advance on this? Note that the resource manager emits a summary message with lots of info, including the application name, at the very end of the application (see below). A message like this would be ideal at the *beginning* of the application lifecycle. 2015-10-21 08:52:12,727 INFO org.apache.hadoop.yarn.server.resourcemanager.RMAppManager$ApplicationSummary: appId=application_1444927016569_0035,name=Dummy,user=roji,queue=root.roji,state=FINISHED,trackingUrl=http://**/proxy/application_1444927016569_0035/,appMasterHost=,startTime=1445417521080,finishTime=1445417531344,finalStatus=SUCCEEDED,memorySeconds=19316,vcoreSeconds=18,preemptedAMContainers=0,preemptedNonAMContainers=0,preemptedResources=,applicationType=SPARK > Add app name to RM audit log > > > Key: YARN-3869 > URL: https://issues.apache.org/jira/browse/YARN-3869 > Project: Hadoop YARN > Issue Type: Improvement >Reporter: Shay Rojansky >Priority: Minor > > The YARN resource manager audit log currently includes useful info such as > APPID, USER, etc. One crucial piece of information missing is the > user-supplied application name. > Users are familiar with their application name as shown in the YARN UI, etc. > It's vital for something like logstash to be able to associate logs with the > application name for later searching in something like kibana. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-3869) Add app name to RM audit log
[ https://issues.apache.org/jira/browse/YARN-3869?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14616521#comment-14616521 ] Shay Rojansky commented on YARN-3869: - That's a good point, I'm not sure. Not that there's no requirement for appnames to be unique - I can submit two apps with the same name - so that's a problem I'd have to deal with anyway in logstash or whatever. In any case I wouldn't want to be penalized and lose appnames in logs just because Hive does something its own way. Regardless, this does make me think that the appname should probably be surrounded by quotes (may contain spaces etc) and possibly also escaped. As you say, let's wait for some other opinions. > Add app name to RM audit log > > > Key: YARN-3869 > URL: https://issues.apache.org/jira/browse/YARN-3869 > Project: Hadoop YARN > Issue Type: Improvement >Reporter: Shay Rojansky >Assignee: nijel >Priority: Minor > > The YARN resource manager audit log currently includes useful info such as > APPID, USER, etc. One crucial piece of information missing is the > user-supplied application name. > Users are familiar with their application name as shown in the YARN UI, etc. > It's vital for something like logstash to be able to associate logs with the > application name for later searching in something like kibana. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-3869) Add app name to RM audit log
[ https://issues.apache.org/jira/browse/YARN-3869?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14616323#comment-14616323 ] Shay Rojansky commented on YARN-3869: - Wasn't aware of this case... How is this handled within the Yarn wen UI, which also displays app names? One thought is to simply truncate to some max char count when outputting to the logs... > Add app name to RM audit log > > > Key: YARN-3869 > URL: https://issues.apache.org/jira/browse/YARN-3869 > Project: Hadoop YARN > Issue Type: Improvement >Reporter: Shay Rojansky >Assignee: nijel >Priority: Minor > > The YARN resource manager audit log currently includes useful info such as > APPID, USER, etc. One crucial piece of information missing is the > user-supplied application name. > Users are familiar with their application name as shown in the YARN UI, etc. > It's vital for something like logstash to be able to associate logs with the > application name for later searching in something like kibana. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (YARN-3869) Add app name to RM audit log
[ https://issues.apache.org/jira/browse/YARN-3869?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shay Rojansky updated YARN-3869: Description: The YARN resource manager audit log currently includes useful info such as APPID, USER, etc. One crucial piece of information missing is the user-supplied application name. Users are familiar with their application name as shown in the YARN UI, etc. It's vital for something like logstash to be able to associate logs with the application name for later searching in something like kibana. was: The YARN resource manager audit log currently includes useful info such as APPID, USER, etc. One crucial piece of information missing is the user-supplied application name. Users are familiar with their application name as shown in the YARN UI, etc. It's vital for something like logstash to be able to associated logs with the application name for later searching in something like kibana. > Add app name to RM audit log > > > Key: YARN-3869 > URL: https://issues.apache.org/jira/browse/YARN-3869 > Project: Hadoop YARN > Issue Type: Improvement >Reporter: Shay Rojansky >Priority: Minor > > The YARN resource manager audit log currently includes useful info such as > APPID, USER, etc. One crucial piece of information missing is the > user-supplied application name. > Users are familiar with their application name as shown in the YARN UI, etc. > It's vital for something like logstash to be able to associate logs with the > application name for later searching in something like kibana. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-3869) Add app name to RM audit log
[ https://issues.apache.org/jira/browse/YARN-3869?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14609814#comment-14609814 ] Shay Rojansky commented on YARN-3869: - Nope, feel free to go ahead! > Add app name to RM audit log > > > Key: YARN-3869 > URL: https://issues.apache.org/jira/browse/YARN-3869 > Project: Hadoop YARN > Issue Type: Improvement >Reporter: Shay Rojansky >Priority: Minor > > The YARN resource manager audit log currently includes useful info such as > APPID, USER, etc. One crucial piece of information missing is the > user-supplied application name. > Users are familiar with their application name as shown in the YARN UI, etc. > It's vital for something like logstash to be able to associated logs with the > application name for later searching in something like kibana. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (YARN-3869) Add app name to RM audit log
Shay Rojansky created YARN-3869: --- Summary: Add app name to RM audit log Key: YARN-3869 URL: https://issues.apache.org/jira/browse/YARN-3869 Project: Hadoop YARN Issue Type: Improvement Reporter: Shay Rojansky Priority: Minor The YARN resource manager audit log currently includes useful info such as APPID, USER, etc. One crucial piece of information missing is the user-supplied application name. Users are familiar with their application name as shown in the YARN UI, etc. It's vital for something like logstash to be able to associated logs with the application name for later searching in something like kibana. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
