YARN-6453. fairscheduler-statedump.log gets generated regardless of service (Contributed by Yufei Gu via Daniel Templeton)
Project: http://git-wip-us.apache.org/repos/asf/hadoop/repo Commit: http://git-wip-us.apache.org/repos/asf/hadoop/commit/f356f0f4 Tree: http://git-wip-us.apache.org/repos/asf/hadoop/tree/f356f0f4 Diff: http://git-wip-us.apache.org/repos/asf/hadoop/diff/f356f0f4 Branch: refs/heads/YARN-5355 Commit: f356f0f4cf22efdc920c2a4be2b4b6a5fc4c3e3c Parents: c0cf11e Author: Daniel Templeton <templ...@apache.org> Authored: Thu Apr 20 14:11:01 2017 -0700 Committer: Daniel Templeton <templ...@apache.org> Committed: Thu Apr 20 14:13:07 2017 -0700 ---------------------------------------------------------------------- .../src/main/conf/log4j.properties | 22 ++++++++++++-------- .../src/site/markdown/FairScheduler.md | 6 ++++++ 2 files changed, 19 insertions(+), 9 deletions(-) ---------------------------------------------------------------------- http://git-wip-us.apache.org/repos/asf/hadoop/blob/f356f0f4/hadoop-common-project/hadoop-common/src/main/conf/log4j.properties ---------------------------------------------------------------------- diff --git a/hadoop-common-project/hadoop-common/src/main/conf/log4j.properties b/hadoop-common-project/hadoop-common/src/main/conf/log4j.properties index b4658ae..22a2117 100644 --- a/hadoop-common-project/hadoop-common/src/main/conf/log4j.properties +++ b/hadoop-common-project/hadoop-common/src/main/conf/log4j.properties @@ -322,12 +322,16 @@ log4j.appender.EWMA.cleanupInterval=${yarn.ewma.cleanupInterval} log4j.appender.EWMA.messageAgeLimitSeconds=${yarn.ewma.messageAgeLimitSeconds} log4j.appender.EWMA.maxUniqueMessages=${yarn.ewma.maxUniqueMessages} -# Fair scheduler requests log on state dump -log4j.logger.org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FairScheduler.statedump=DEBUG,FSLOGGER -log4j.additivity.org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FairScheduler.statedump=false -log4j.appender.FSLOGGER=org.apache.log4j.RollingFileAppender -log4j.appender.FSLOGGER.File=${hadoop.log.dir}/fairscheduler-statedump.log -log4j.appender.FSLOGGER.layout=org.apache.log4j.PatternLayout -log4j.appender.FSLOGGER.layout.ConversionPattern=%d{ISO8601} %p %c: %m%n -log4j.appender.FSLOGGER.MaxFileSize=${hadoop.log.maxfilesize} -log4j.appender.FSLOGGER.MaxBackupIndex=${hadoop.log.maxbackupindex} \ No newline at end of file +# +# Fair scheduler state dump +# +# Use following logger to dump the state to a separate file + +#log4j.logger.org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FairScheduler.statedump=DEBUG,FSSTATEDUMP +#log4j.additivity.org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FairScheduler.statedump=false +#log4j.appender.FSSTATEDUMP=org.apache.log4j.RollingFileAppender +#log4j.appender.FSSTATEDUMP.File=${hadoop.log.dir}/fairscheduler-statedump.log +#log4j.appender.FSSTATEDUMP.layout=org.apache.log4j.PatternLayout +#log4j.appender.FSSTATEDUMP.layout.ConversionPattern=%d{ISO8601} %p %c: %m%n +#log4j.appender.FSSTATEDUMP.MaxFileSize=${hadoop.log.maxfilesize} +#log4j.appender.FSSTATEDUMP.MaxBackupIndex=${hadoop.log.maxbackupindex} \ No newline at end of file http://git-wip-us.apache.org/repos/asf/hadoop/blob/f356f0f4/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site/src/site/markdown/FairScheduler.md ---------------------------------------------------------------------- diff --git a/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site/src/site/markdown/FairScheduler.md b/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site/src/site/markdown/FairScheduler.md index 6661bc1..748fda9 100644 --- a/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site/src/site/markdown/FairScheduler.md +++ b/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site/src/site/markdown/FairScheduler.md @@ -234,3 +234,9 @@ The following fields can be seen for each queue on the web interface: The Fair Scheduler supports moving a running application to a different queue. This can be useful for moving an important application to a higher priority queue, or for moving an unimportant application to a lower priority queue. Apps can be moved by running `yarn application -movetoqueue appID -queue targetQueueName`. When an application is moved to a queue, its existing allocations become counted with the new queue's allocations instead of the old for purposes of determining fairness. An attempt to move an application to a queue will fail if the addition of the app's resources to that queue would violate the its maxRunningApps or maxResources constraints. + +###Dumping Fair Scheduler state + +Fair Scheduler is able to dump its state periodically. It is disabled by default. The administrator can enable it by setting org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FairScheduler logging level to DEBUG. + +Fair Scheduler logs go to the Resource Manager log file by default. Fair Scheduler state dumps can potentially generate large amount of log data. Uncomment the "Fair scheduler state dump" section in log4j.properties to dump the state into a separate file. --------------------------------------------------------------------- To unsubscribe, e-mail: common-commits-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-commits-h...@hadoop.apache.org
