[
https://issues.apache.org/jira/browse/HADOOP-7239?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13023313#comment-13023313
]
Venu Gopala Rao commented on HADOOP-7239:
-----------------------------------------
I looked at code for the root cause. It is as follows
While writing the job.xml, it is using the Configuration.writeXML(), which is
as follows
{code:title=Configuration.java|borderStyle=solid}
Properties properties = getProps();
try {
......
for (Enumeration e = properties.keys(); e.hasMoreElements();) {
String name = (String)e.nextElement();
Object object = properties.get(name);
String value = null;
if (object instanceof String) {
value = (String) object;
}else {
continue;
}
{code}
So here all properties in configuration are retrieved using getProps() which
returns a ConcurrentHashMap and written to XML from that. So the variables are
not substituted. instead of using *properties.get(name);*
in above snippet we should use *get(name)*, so that all variable substitution
happen correctly.
> The variables in the configuration files are not replaced with values, when
> the job is submitted by Job client
> --------------------------------------------------------------------------------------------------------------
>
> Key: HADOOP-7239
> URL: https://issues.apache.org/jira/browse/HADOOP-7239
> Project: Hadoop Common
> Issue Type: Bug
> Components: conf
> Affects Versions: 0.20.2, 0.21.0
> Environment: Linux + Sun JDK 1.6
> Reporter: Venu Gopala Rao
>
> We have a case where we wanted to create the Job names dynamically at run
> time.Since JobConf is an extension for the Configuration object, we thought
> we can make use of the variable substitution concept in configuration like
> below
> Job job = new Job(conf, "${mapred.user.name}" + "-job" + new
> Random().nextInt());
> job.setJarByClass(WordCount.class);
> job.setMapperClass(TokenizerMapper.class);
> job.setCombinerClass(IntSumReducer.class);
> job.setReducerClass(IntSumReducer.class);
> job.setOutputKeyClass(Text.class);
> job.setOutputValueClass(IntWritable.class);
> FileInputFormat.addInputPath(job, new Path(otherArgs[0]));
> FileOutputFormat.setOutputPath(job, new Path(otherArgs[1]));
> job.submit();
> We set the required run time variables(in this case mapred.user.name) before
> calling the job submit. But on the Job tracker side the variables are not
> replaced correctly.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
