Warnings about attempt to override final parameter while getting delegation
token
---------------------------------------------------------------------------------
Key: HDFS-2417
URL: https://issues.apache.org/jira/browse/HDFS-2417
Project: Hadoop HDFS
Issue Type: Bug
Components: name-node
Affects Versions: 0.20.205.0
Reporter: Rajit Saha
I am seeing whenever I run any Mapreduce job and its trying to acquire
delegation from NN, In JT log following warnings coming about "a attempt to
override final parameter:"
The log snippet in JT log
2011-10-07 20:29:19,096 WARN org.apache.hadoop.conf.Configuration:
/tmp/mapred-local/jobTracker/job_201110072015_0005.xml:a attempt to override
final parameter:
mapred.job.reuse.jvm.num.tasks; Ignoring.
2011-10-07 20:29:19,096 WARN org.apache.hadoop.conf.Configuration:
/tmp/mapred-local/jobTracker/job_201110072015_0005.xml:a attempt to override
final parameter: mapred.system.dir;
Ignoring.
2011-10-07 20:29:19,096 WARN org.apache.hadoop.conf.Configuration:
/tmp/mapred-local/jobTracker/job_201110072015_0005.xml:a attempt to override
final parameter:
hadoop.job.history.user.location; Ignoring.
2011-10-07 20:29:19,096 WARN org.apache.hadoop.conf.Configuration:
/tmp/mapred-local/jobTracker/job_201110072015_0005.xml:a attempt to override
final parameter: mapred.local.dir;
Ignoring.
2011-10-07 20:29:19,096 WARN org.apache.hadoop.conf.Configuration:
/tmp/mapred-local/jobTracker/job_201110072015_0005.xml:a attempt to override
final parameter: m
apred.job.tracker.http.address; Ignoring.
2011-10-07 20:29:19,096 WARN org.apache.hadoop.conf.Configuration:
/tmp/mapred-local/jobTracker/job_201110072015_0005.xml:a attempt to override
final parameter: d
fs.data.dir; Ignoring.
2011-10-07 20:29:19,096 WARN org.apache.hadoop.conf.Configuration:
/tmp/mapred-local/jobTracker/job_201110072015_0005.xml:a attempt to override
final parameter: d
fs.http.address; Ignoring.
2011-10-07 20:29:19,096 WARN org.apache.hadoop.conf.Configuration:
/tmp/mapred-local/jobTracker/job_201110072015_0005.xml:a attempt to override
final parameter: m
apreduce.admin.map.child.java.opts; Ignoring.
2011-10-07 20:29:19,096 WARN org.apache.hadoop.conf.Configuration:
/tmp/mapred-local/jobTracker/job_201110072015_0005.xml:a attempt to override
final parameter:
mapreduce.history.server.http.address; Ignoring.
2011-10-07 20:29:19,096 WARN org.apache.hadoop.conf.Configuration:
/tmp/mapred-local/jobTracker/job_201110072015_0005.xml:a attempt to override
final parameter: m
apreduce.history.server.embedded; Ignoring.
2011-10-07 20:29:19,096 WARN org.apache.hadoop.conf.Configuration:
/tmp/mapred-local/jobTracker/job_201110072015_0005.xml:a attempt to override
final parameter: m
apreduce.jobtracker.split.metainfo.maxsize; Ignoring.2011-10-07 20:29:19,096
WARN
org.apache.hadoop.conf.Configuration:
/tmp/mapred-local/jobTracker/job_201110072015_0005.xml:a attempt to
override final parameter: m
apreduce.admin.reduce.child.java.opts; Ignoring.2011-10-07 20:29:19,096 WARN
org.apache.hadoop.conf.Configuration:
/tmp/mapred-local/jobTracker/job_201110072015_0005.xml:a attempt to override
final parameter: h
adoop.tmp.dir; Ignoring.
2011-10-07 20:29:19,096 WARN org.apache.hadoop.conf.Configuration:
/tmp/mapred-local/jobTracker/job_201110072015_0005.xml:a attempt to override
final parameter:
mapred.jobtracker.maxtasks.per.job; Ignoring.
2011-10-07 20:29:19,096 WARN org.apache.hadoop.conf.Configuration:
/tmp/mapred-local/jobTracker/job_201110072015_0005.xml:a attempt to override
final parameter:
mapred.job.tracker; Ignoring.
2011-10-07 20:29:19,096 WARN org.apache.hadoop.conf.Configuration:
/tmp/mapred-local/jobTracker/job_201110072015_0005.xml:a attempt to override
final parameter: dfs.name.dir;
Ignoring.
2011-10-07 20:29:19,096 WARN org.apache.hadoop.conf.Configuration:
/tmp/mapred-local/jobTracker/job_201110072015_0005.xml:a attempt to override
final parameter: m
apred.temp.dir; Ignoring.2011-10-07 20:29:19,103 INFO
org.apache.hadoop.mapreduce.security.token.DelegationTokenRenewal: registering
token for renewal for service
=<NN IP>:50470 and jobID
= job_201110072015_0005
2011-10-07 20:29:19,103 INFO
org.apache.hadoop.mapreduce.security.token.DelegationTokenRenewal: registering
token for
renewal for service =<NN IP>:8020 and jobID = job_201110072015_0005
The STDOUT of distcp job when these warnings logged into JT log
----------------------------------------------------------------
$hadoop distcp hftp://<NN>:50070/tmp/inp out
11/10/07 20:29:17 INFO tools.DistCp:
srcPaths=[hftp://<NN>:50070/tmp/inp]
11/10/07 20:29:17 INFO tools.DistCp: destPath=out
11/10/07 20:29:18 INFO security.TokenCache: Got dt for
hftp://<NN>:50070/tmp/inp;uri=<NN IP>:50470;t.service=<NN IP>:50470
11/10/07 20:29:18 INFO hdfs.DFSClient: Created HDFS_DELEGATION_TOKEN token 7
for <user> on <NN IP>:8020
11/10/07 20:29:18 INFO security.TokenCache: Got dt for
1318019341/as;uri=<NN IP>:8020;t.service=<NN IP>:8020
..
..
If this is not critical, it may be unnecessary noise in log
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
