Rohith Sharma K S created YARN-4882:
---------------------------------------
Summary: Change the log level to DEBUG for recovering completed
applications
Key: YARN-4882
URL: https://issues.apache.org/jira/browse/YARN-4882
Project: Hadoop YARN
Issue Type: Bug
Components: resourcemanager
Reporter: Rohith Sharma K S
I think for recovering completed applications no need to log as INFO, rather it
can be made it as DEBUG. The problem seen from large cluster is if any issue
happens during RM start up and continuously switching , then RM logs are
filled with most with recovering applications only.
There are 6 lines are logged for 1 applications as I shown in below logs, then
consider RM default value for max-completed applications is 10K. So for each
switch 10K*6=60K lines will be added which is not useful I feel.
{noformat}
2016-03-01 10:20:59,077 INFO
org.apache.hadoop.yarn.server.resourcemanager.RMAppManager: Default priority
level is set to application:application_1456298208485_21507
2016-03-01 10:20:59,094 INFO
org.apache.hadoop.yarn.server.resourcemanager.rmapp.RMAppImpl: Recovering app:
application_1456298208485_21507 with 1 attempts and final state = FINISHED
2016-03-01 10:20:59,100 INFO
org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.RMAppAttemptImpl:
Recovering attempt: appattempt_1456298208485_21507_000001 with final state:
FINISHED
2016-03-01 10:20:59,107 INFO
org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.RMAppAttemptImpl:
appattempt_1456298208485_21507_000001 State change from NEW to FINISHED
2016-03-01 10:20:59,111 INFO
org.apache.hadoop.yarn.server.resourcemanager.rmapp.RMAppImpl:
application_1456298208485_21507 State change from NEW to FINISHED
2016-03-01 10:20:59,112 INFO
org.apache.hadoop.yarn.server.resourcemanager.RMAuditLogger: USER=rohith
OPERATION=Application Finished - Succeeded TARGET=RMAppManager
RESULT=SUCCESS APPID=application_1456298208485_21507
{noformat}
The main problem is missing important information's from the logs before RM
unstable. Even though log roll back is 50 or 100, in a short period all these
logs will be rolled out and all the logs contains only RM switching information
that too recovering applications!!.
I suggest at least completed applications recovery should be logged as DEBUG.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
