[
https://issues.apache.org/jira/browse/MAPREDUCE-3265?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13155656#comment-13155656
]
Arun C Murthy commented on MAPREDUCE-3265:
------------------------------------------
With the patch, the output looks so:
{noformat}
WARNING: org.apache.hadoop.metrics.jvm.EventCounter is deprecated. Please use
org.apache.hadoop.log.metrics.EventCounter in all the log4j.properties files.
11/11/23 03:05:12 WARN conf.Configuration: mapred.used.genericoptionsparser is
deprecated. Instead, use mapreduce.client.genericoptionsparser.used
Running on 5 nodes to sort from
hdfs://hrt12n01.cc1.ygridcore.net:8020/user/hrt_1/in into
hdfs://nn.foobar.com:8020/user/hrt_1/out with 9 reduces.
Job started: Wed Nov 23 03:05:12 UTC 2011
11/11/23 03:05:12 INFO hdfs.DFSClient: Created HDFS_DELEGATION_TOKEN token 379
for hrt_1 on xx.xx.xx.xx:8020
11/11/23 03:05:12 INFO security.TokenCache: Got dt for
hdfs://nn.foobar.com:8020:8020;uri=xx.xx.xx.xx:8020;t.service=xx.xx.xx.xx:8020
11/11/23 03:05:12 INFO hdfs.DFSClient: Created HDFS_DELEGATION_TOKEN token 380
for hrt_1 on xx.xx.xx.xx:8020
11/11/23 03:05:12 INFO security.TokenCache: Got dt for
hdfs://nn.foobar.com:8020:8020;uri=xx.xx.xx.xx:8020;t.service=xx.xx.xx.xx:8020
11/11/23 03:05:12 WARN conf.Configuration: fs.default.name is deprecated.
Instead, use fs.defaultFS
11/11/23 03:05:15 INFO input.FileInputFormat: Total input paths to process : 5
11/11/23 03:05:17 INFO mapreduce.JobSubmitter: number of splits:5
11/11/23 03:05:18 INFO mapred.ResourceMgrDelegate: Submitted application
application_1322016908667_0002 to ResourceManager at
rm.foobar.com/yy.yy.yy.yy:8040
11/11/23 03:05:18 INFO mapreduce.Job: Running job: job_1322016908667_0002
11/11/23 03:05:19 INFO mapreduce.Job: map 0% reduce 0%
11/11/23 03:05:29 INFO mapred.ClientServiceDelegate: Job tracking-url:
rm.foobar.com:8088/proxy/application_1322016908667_0002/
11/11/23 03:05:40 INFO mapreduce.Job: map 100% reduce 0%
{noformat}
> Reduce log level on MR2 IPC construction, etc
> ---------------------------------------------
>
> Key: MAPREDUCE-3265
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3265
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: mrv2
> Affects Versions: 0.23.0
> Reporter: Todd Lipcon
> Assignee: Arun C Murthy
> Priority: Blocker
> Fix For: 0.23.1
>
> Attachments: MAPREDUCE-3265.patch
>
>
> Currently MR's IPC logging is very verbose. For example, I see a lot of:
> 11/10/25 12:14:06 INFO ipc.YarnRPC: Creating YarnRPC for
> org.apache.hadoop.yarn.ipc.HadoopYarnProtoRPC
> 11/10/25 12:14:06 INFO mapred.ResourceMgrDelegate: Connecting to
> ResourceManager at c0309.hal.cloudera.com/172.29.81.91:40012
> 11/10/25 12:14:06 INFO ipc.HadoopYarnRPC: Creating a HadoopYarnProtoRpc proxy
> for protocol interface org.apache.hadoop.yarn.api.ClientRMProtocol
> 11/10/25 12:14:07 INFO mapred.ResourceMgrDelegate: Connected to
> ResourceManager at c0309.hal.cloudera.com/172.29.81.91:40012
> 11/10/25 12:14:08 INFO mapred.ClientCache: Connecting to HistoryServer at:
> c0309.hal.cloudera.com:10020
> 11/10/25 12:14:08 INFO ipc.YarnRPC: Creating YarnRPC for
> org.apache.hadoop.yarn.ipc.HadoopYarnProtoRPC
> 11/10/25 12:14:08 INFO mapred.ClientCache: Connected to HistoryServer at:
> c0309.hal.cloudera.com:10020
> 11/10/25 12:14:08 INFO ipc.HadoopYarnRPC: Creating a HadoopYarnProtoRpc proxy
> for protocol interface org.apache.hadoop.mapreduce.v2.api.MRClientProtocol
> ... when submitting a job. This should be DEBUG level.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
