[
https://issues.apache.org/jira/browse/MAPREDUCE-4517?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jason Lowe updated MAPREDUCE-4517:
----------------------------------
Status: Open (was: Patch Available)
I agree the messages are too long and printing them every heartbeat is
wasteful. However the messages have proven multiple times to be critical in
debugging scheduling issues, and I fear having them turned off completely by
default will make debugging issues between the RM and AM very difficult.
I think we could log a lot less and still have the state info we need if we
only logged these messages when something in them changes. Logging the same
state every heartbeat isn't useful, but knowing when these states change is.
> Too many INFO messages written out during AM to RM heartbeat
> ------------------------------------------------------------
>
> Key: MAPREDUCE-4517
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4517
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: applicationmaster
> Reporter: James Kinley
> Priority: Minor
> Labels: patch
> Fix For: trunk
>
> Attachments: MAPREDUCE-4517.patch
>
>
> Too many INFO log messages written out during AM to RM heartbeat. Based on
> default frequency of 1000ms (scheduler.heartbeat.interval-ms) either 2 or 4
> INFO messages are written out per second:
> LOG.info("Before Scheduling: " + getStat());
> List<Container> allocatedContainers = getResources();
> LOG.info("After Scheduling: " + getStat());
> if (allocatedContainers.size() > 0) {
> LOG.info("Before Assign: " + getStat());
> scheduledRequests.assign(allocatedContainers);
> LOG.info("After Assign: " + getStat());
> }
> These should probably be changed to DEBUG message to save the log growing too
> quickly.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
