Varun Saxena created MAPREDUCE-5981:
---------------------------------------
Summary: Log levels of certain MR logs can be changed to DEBUG
Key: MAPREDUCE-5981
URL: https://issues.apache.org/jira/browse/MAPREDUCE-5981
Project: Hadoop Map/Reduce
Issue Type: Improvement
Reporter: Varun Saxena
Assignee: Varun Saxena
Following map reduce logs can be changed to DEBUG log level.
1. In org.apache.hadoop.mapreduce.task.reduce.Fetcher#copyFromHost(Fetcher.java
: 313), the second log is not required to be at info level. This can be moved
to debug as a warn log is anyways printed if verifyReply fails.
SecureShuffleUtils.verifyReply(replyHash, encHash, shuffleSecretKey);
LOG.info("for url="+msgToEncode+" sent hash and received reply");
2. Thread related info need not be printed in logs at INFO level. Below 2 logs
can be moved to DEBUG
a) In
org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl#getHost(ShuffleSchedulerImpl.java
: 381), below log can be changed to DEBUG
LOG.info("Assigning " + host + " with " + host.getNumKnownMapOutputs() +
" to " + Thread.currentThread().getName());
b) In
org.apache.hadoop.mapreduce.task.reduce.ShuffleScheduler.getMapsForHost(ShuffleSchedulerImpl.java
: 411), below log can be changed to DEBUG
LOG.info("assigned " + includedMaps + " of " + totalSize + " to " +
host + " to " + Thread.currentThread().getName());
--
This message was sent by Atlassian JIRA
(v6.2#6252)
