Hello,
I'm still having problems configuring a cluster running federated
HDFS, in a single node. Here's what i've done so far:
1. Extracted hdfs 0.23 tarball at 4 different locations, one for
each of the 3 NameNode daemons and 1 for DataNode and client
2. I've added hdfs-site.xml and core-site.xml having the following
information to each of the NameNode:
core-site.xml ---- http://pastebin.com/nJaTq00s
hdfs-site.xml ---- http://pastebin.com/DkueDHDf
I've specified different port numbers, http port numbers and
nameservice ids for each NameNode
3. Finally, to the fourth copy (DataNode) i've added the following
configuration information:
core-site.xml -----
http://pastebin.com/DYfwfi44<http://pastebin.com/58FJfTnn>
hdfs-site.xml ----- http://pastebin.com/xnmNnCKq
I've specified fs.defaultFs here as I'll be using this as my
client too.
4. I then go to each bin folder and run the "hadoop namenode
-format" and "hadoop namenode"
I haven't set the HADOOP_HOME env variable yet. (all namenodes
run without any exception)
5. I then go to the DataNode bin folder and run "hadoop datanode".
(no exception here)
6. Then I set the HADOOP_HOME env variable to the hadoop
distribution from which the ran the DataNode.
7. Finally, I try to list out the contents of the viewfs file
system by running "*hadoop fs -ls /*"
The output i'm getting is only this : "*ls:
viewfs://localhost:33332/*"
Is this the right way to configure federated HDFS?
On Wed, Feb 8, 2012 at 12:28, Suresh Srinivas <sur...@hortonworks.com>wrote:
>
>
> On Tue, Feb 7, 2012 at 4:51 PM, Chandrasekar <chandruseka...@gmail.com>wrote:
>
>> In which file should i specify all this information about
>> nameservices and the list of namenodes?
>>
>
> hdfs-site.xml is the appropriate place, since it is hdfs-specific
> configuration.
>
> If there are multiple namenodes, then which one should i specify
>> in core-site.xml as fs.defaultFS?
>>
>
> core-site.xml is the right place for fs.defaultFS.
>
> Given you have multiple namespaces from in federation setup, fs.defaultFS
> should point to ViewFileSystem for a unified view of the namespaces to the
> clients. There is an open bug HDFS-2558 to track this. I will get to this
> as soon as I can.
>
> Regards,
> Suresh
>
