Susheel actually brought up a good point.
once the client code connects to the cluster, is there way to get the real
cluster configuration variables/values instead of relying on the .xml files
on client side?
Demai
On Mon, Sep 8, 2014 at 10:12 PM, Susheel Kumar Gadalay skgada...@gmail.com
counts, mount names etc. When you ask configuration variables/values,
which one should be returned?
Yong
Date: Tue, 9 Sep 2014 10:01:14 -0700
Subject: Re: conf.get(dfs.data.dir) return null when hdfs-site.xml doesn't
set it explicitly
From: nid...@gmail.com
To: user@hadoop.apache.org
Susheel
from the JobContext.
Yong
Date: Tue, 9 Sep 2014 11:34:03 -0700
Subject: Re: conf.get(dfs.data.dir) return null when hdfs-site.xml doesn't
set it explicitly
From: nid...@gmail.com
To: user@hadoop.apache.org
Yong,
good point about each node of the cluster could have different values in the
.xml
: Re: conf.get(dfs.data.dir) return null when hdfs-site.xml
doesn't set it explicitly
From: nid...@gmail.com
To: user@hadoop.apache.org
Susheel actually brought up a good point.
once the client code connects to the cluster, is there way to get the real
cluster configuration variables/values
Hi Demai,
When you read a property from the conf object, it will only have a value if
the conf object contains that property.
In your case, you created the conf object as new Configuration() -- adds
core-default and core-site.xml.
Then you added site.xmls (hdfs-site.xml and core-site.xml) from
hi, Bhooshan,
thanks for your kind response. I run the code on one of the data node of
my cluster, with only one hadoop daemon running. I believe my java client
code connect to the cluster correctly as I am able to retrieve fileStatus,
and list files under a particular hdfs path, and similar
Hi Demai,
conf = new Configuration()
will create a new Configuration object and only add the properties from
core-default.xml and core-site.xml in the conf object.
This is basically a new configuration object, not the same that the daemons
in the hadoop cluster use.
I think what you are
Bhooshan,
Many thanks. I appreciate the help. I will also try out Cloudera mailing
list/community
Demai
On Mon, Sep 8, 2014 at 4:58 PM, Bhooshan Mogal bhooshan.mo...@gmail.com
wrote:
Hi Demai,
conf = new Configuration()
will create a new Configuration object and only add the properties
One doubt on building Configuration object.
I have a Hadoop remote client and Hadoop cluster.
When a client submitted a MR job, the Configuration object is built
from Hadoop cluster node xml files, basically the resource manager
node core-site.xml and mapred-site.xml and yarn-site.xml.
Am I
