[
https://issues.apache.org/jira/browse/MAPREDUCE-3307?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13138671#comment-13138671
]
Ramya Sunil commented on MAPREDUCE-3307:
----------------------------------------
The redundant information is highlighted below:
{color:red}
11/10/28 02:41:52 INFO ipc.YarnRPC: Creating YarnRPC for
org.apache.hadoop.yarn.ipc.HadoopYarnProtoRPC
{color}
11/10/28 02:41:52 INFO mapred.ResourceMgrDelegate: Connecting to
ResourceManager at <hostname>/<ip:port>
{color:red}
11/10/28 02:41:52 INFO ipc.HadoopYarnRPC: Creating a HadoopYarnProtoRpc proxy
for protocol interface org.apache.hadoop.yarn.api.ClientRMProtocol
{color}
11/10/28 02:41:52 INFO mapred.ResourceMgrDelegate: Connected to ResourceManager
at <hostname>/<ip:port>
Running 100 maps.
Job started: Fri Oct 28 02:41:52 UTC 2011
{color:red}
11/10/28 02:41:52 INFO ipc.YarnRPC: Creating YarnRPC for
org.apache.hadoop.yarn.ipc.HadoopYarnProtoRPC
11/10/28 02:41:52 INFO mapred.ResourceMgrDelegate: Connecting to
ResourceManager <hostname>/<ip:port>
11/10/28 02:41:52 INFO ipc.HadoopYarnRPC: Creating a HadoopYarnProtoRpc proxy
for protocol interface org.apache.hadoop.yarn.api.ClientRMProtocol
11/10/28 02:41:52 INFO mapred.ResourceMgrDelegate: Connected to ResourceManager
at <hostname>/<ip:port>
{color}
11/10/28 02:41:53 INFO hdfs.DFSClient: Created HDFS_DELEGATION_TOKEN token <id>
for <user> on <ip:port>
11/10/28 02:41:53 INFO security.TokenCache: Got dt for
hdfs://<host:port>;uri=<ip:port>;t.service=<ip:port>
{color:red}
11/10/28 02:41:53 INFO hdfs.DFSClient: Created HDFS_DELEGATION_TOKEN token <id>
for <user> on <ip:port>
11/10/28 02:41:53 INFO security.TokenCache: Got dt for
hdfs://<host:port>;uri=<ip:port>;t.service=<ip:port>
{color}
11/10/28 02:41:53 INFO mapreduce.JobSubmitter: number of splits:100
11/10/28 02:41:53 INFO mapred.YARNRunner: AppMaster capability = memory: 2048,
11/10/28 02:41:53 INFO mapred.YARNRunner: Command to launch container for
ApplicationMaster is : $JAVA_HOME/bin/java
-Dlog4j.configuration=container-log4j.properties
-Dyarn.app.mapreduce.container.log.dir=<LOG_DIR>
-Dyarn.app.mapreduce.container.log.filesize=0 -Dhadoop.root.logger=INFO,CLA
-Xmx1536m org.apache.hadoop.mapreduce.v2.app.MRAppMaster 1><LOG_DIR>/stdout
2><LOG_DIR>/stderr
11/10/28 02:41:53 INFO mapred.ResourceMgrDelegate: Submitted application
<appID> to ResourceManager
11/10/28 02:41:53 INFO mapred.ClientCache: Connecting to HistoryServer at:
<host:port>
{color:red}
11/10/28 02:41:53 INFO ipc.YarnRPC: Creating YarnRPC for
org.apache.hadoop.yarn.ipc.HadoopYarnProtoRPC
{color}
11/10/28 02:41:53 INFO mapred.ClientCache: Connected to HistoryServer at:
<host:port>
{color:red}
11/10/28 02:41:53 INFO ipc.HadoopYarnRPC: Creating a HadoopYarnProtoRpc proxy
for protocol interface org.apache.hadoop.mapreduce.v2.api.MRClientProtocol
{color}
11/10/28 02:41:53 INFO mapreduce.Job: Running job: job_ID
11/10/28 02:41:54 INFO mapreduce.Job: map 0% reduce 0%
11/10/28 02:42:05 INFO mapred.ClientServiceDelegate: Tracking Url of JOB is
<url>
11/10/28 02:42:05 INFO mapred.ClientServiceDelegate: Connecting to <host:port>
{color:red}
11/10/28 02:42:05 INFO ipc.YarnRPC: Creating YarnRPC for
org.apache.hadoop.yarn.ipc.HadoopYarnProtoRPC
11/10/28 02:42:05 INFO ipc.HadoopYarnRPC: Creating a HadoopYarnProtoRpc proxy
for protocol interface org.apache.hadoop.mapreduce.v2.api.MRClientProtocol
{color}
11/10/28 02:42:05 INFO security.ApplicationTokenSelector: Looking for a token
with service <ip:port>
11/10/28 02:42:05 INFO security.ApplicationTokenSelector: Token kind is
YARN_APPLICATION_TOKEN and the token's service name is <ip:port>
11/10/28 02:43:44 INFO mapreduce.Job: map 2% reduce 0%
11/10/28 02:43:51 INFO mapreduce.Job: map 3% reduce 0%
11/10/28 02:43:52 INFO mapreduce.Job: map 5% reduce 0%
11/10/28 02:43:56 INFO mapreduce.Job: map 10% reduce 0%
Also,
Printing, "Running job: job_ID" and then trying to connect to AM is misleading.
This should be fixed too.
> Improve logging on the console during job execution
> ---------------------------------------------------
>
> Key: MAPREDUCE-3307
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3307
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: mrv2
> Affects Versions: 0.23.0
> Reporter: Ramya Sunil
> Fix For: 0.23.0
>
>
> There is a lot of redundant information being printed on the console and a
> not so intuitive flow of events. We should improve the logging on console
> during job execution. More details in the next comment.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
