Sandy Ryza created YARN-1291:
--------------------------------
Summary: RM INFO logs limit scheduling speed
Key: YARN-1291
URL: https://issues.apache.org/jira/browse/YARN-1291
Project: Hadoop YARN
Issue Type: Bug
Components: resourcemanager
Affects Versions: 2.2.0
Reporter: Sandy Ryza
Assignee: Sandy Ryza
I've been running some microbenchmarks to see how fast the Fair Scheduler can
fill up a cluster and found its performance is significantly hampered by
logging.
I tested with 500 (mock) nodes, and found that:
* Taking out fair scheduler INFO logs on the critical path brought down the
latency from 14000 ms to 6000 ms
* Taking out the INFO that RMContainerImpl logs when a container transitions
brought it down from 6000 ms to 4000 ms
* Taking out RMAuditLogger logs brought it down from 4000 ms to 1700 ms
--
This message was sent by Atlassian JIRA
(v6.1#6144)
