[
https://issues.apache.org/jira/browse/YARN-5382?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vrushali C updated YARN-5382:
-----------------------------
Attachment: YARN-5382-branch-2.7.03.patch
Uploading patch v3.
Changes made:
- Added a RMAppKillEvent that contains the callerUGI and the remote IP
- Added a method (log success) to print the ip (I am wondering if we need a
corresponding log failure method?)
- Updated the diagnostic message be similar to YARN-5053
- Tested on local machine in a pseudo distributed setup as well through unit
test TestRMAppTransitions.
I see the following in the RM log when I ran and killed a sleep job:
{code}
vrushali[hostname-vrushali hadoop-2.7.4-SNAPSHOT (branch-2.7)]$ grep -rni
auditlogger logs/yarn-vvrushali-resourcemanager-hostname-vrushali.log | grep
-i "kill app"
logs/yarn-vvrushali-resourcemanager-hostname-vrushali.log:204:2016-07-21
00:53:07,493 INFO org.apache.hadoop.yarn.server.resourcemanager.RMAuditLogger:
USER=vvrushali IP=127.0.0.1 OPERATION=Kill Application Request
TARGET=RMAppImpl RESULT=SUCCESS APPID=application_1469087565504_0001
[hostname-vrushali hadoop-2.7.4-SNAPSHOT (branch-2.7)]$
{code}
I see the following in the unit test log:
{code}
[hostname-vrushali hadoop-yarn-server-resourcemanager (branch-2.7)]$ grep
"Kill Application "
target/surefire-reports/org.apache.hadoop.yarn.server.resourcemanager.rmapp.TestRMAppTransitions-output.txt
2016-07-21 00:54:02,761 INFO [Thread-5] resourcemanager.RMAuditLogger
(RMAuditLogger.java:logSuccess(192)) - USER=fooTestAppNewKill IP=127.0.0.1
OPERATION=Kill Application Request TARGET=RMAppImpl RESULT=SUCCESS
APPID=application_1469087642647_0001
2016-07-21 00:54:03,086 INFO [main] resourcemanager.RMAuditLogger
(RMAuditLogger.java:logSuccess(192)) - USER=fooTestAppSubmittedKill
IP=127.0.0.1 OPERATION=Kill Application Request TARGET=RMAppImpl
RESULT=SUCCESS APPID=application_1469087642647_0007
2016-07-21 00:54:03,155 INFO [main] resourcemanager.RMAuditLogger
(RMAuditLogger.java:logSuccess(192)) - USER=fooTestAppAcceptedKill
IP=127.0.0.1 OPERATION=Kill Application Request TARGET=RMAppImpl
RESULT=SUCCESS APPID=application_1469087642647_0009
2016-07-21 00:54:03,258 INFO [Thread-53] resourcemanager.RMAuditLogger
(RMAuditLogger.java:logSuccess(192)) - USER=fooTestAppNewSavingKill
IP=127.0.0.1 OPERATION=Kill Application Request TARGET=RMAppImpl
RESULT=SUCCESS APPID=application_1469087642647_0012
2016-07-21 00:54:03,328 INFO [main] resourcemanager.RMAuditLogger
(RMAuditLogger.java:logSuccess(192)) - USER=fooTestAppRunningKill
IP=127.0.0.1 OPERATION=Kill Application Request TARGET=RMAppImpl
RESULT=SUCCESS APPID=application_1469087642647_0014
2016-07-21 00:54:03,587 INFO [Thread-102] resourcemanager.RMAuditLogger
(RMAuditLogger.java:logSuccess(192)) - USER=fooTestAppNewKill IP=127.0.0.1
OPERATION=Kill Application Request TARGET=RMAppImpl RESULT=SUCCESS
APPID=application_1469087642647_0026
2016-07-21 00:54:03,754 INFO [main] resourcemanager.RMAuditLogger
(RMAuditLogger.java:logSuccess(192)) - USER=fooTestAppSubmittedKill
IP=127.0.0.1 OPERATION=Kill Application Request TARGET=RMAppImpl
RESULT=SUCCESS APPID=application_1469087642647_0032
2016-07-21 00:54:03,807 INFO [main] resourcemanager.RMAuditLogger
(RMAuditLogger.java:logSuccess(192)) - USER=fooTestAppAcceptedKill
IP=127.0.0.1 OPERATION=Kill Application Request TARGET=RMAppImpl
RESULT=SUCCESS APPID=application_1469087642647_0034
2016-07-21 00:54:03,882 INFO [Thread-150] resourcemanager.RMAuditLogger
(RMAuditLogger.java:logSuccess(192)) - USER=fooTestAppNewSavingKill
IP=127.0.0.1 OPERATION=Kill Application Request TARGET=RMAppImpl
RESULT=SUCCESS APPID=application_1469087642647_0037
2016-07-21 00:54:03,926 INFO [main] resourcemanager.RMAuditLogger
(RMAuditLogger.java:logSuccess(192)) - USER=fooTestAppRunningKill
IP=127.0.0.1 OPERATION=Kill Application Request TARGET=RMAppImpl
RESULT=SUCCESS APPID=application_1469087642647_0039
[hostname-vrushali hadoop-yarn-server-resourcemanager (branch-2.7)]$
{code}
> RM does not audit log kill request for active applications
> ----------------------------------------------------------
>
> Key: YARN-5382
> URL: https://issues.apache.org/jira/browse/YARN-5382
> Project: Hadoop YARN
> Issue Type: Bug
> Components: resourcemanager
> Affects Versions: 2.7.2
> Reporter: Jason Lowe
> Assignee: Vrushali C
> Attachments: YARN-5382-branch-2.7.01.patch,
> YARN-5382-branch-2.7.02.patch, YARN-5382-branch-2.7.03.patch
>
>
> ClientRMService will audit a kill request but only if it either fails to
> issue the kill or if the kill is sent to an already finished application. It
> does not create a log entry when the application is active which is arguably
> the most important case to audit.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
