Antoine, In order to setup Federation, all the NameNodes have to be first formatted (or upgraded) with the same cluster ID. This is described here: http://hadoop.apache.org/common/docs/r0.23.0/hadoop-yarn/hadoop-yarn-site/Federation.html#Formatting_Namenodes .
-Vinithra On Wed, Jul 25, 2012 at 5:01 AM, Antoine Boudot <abou...@cyres.fr> wrote: > hello I'm beginner on technology hadoop and I encounter a problem ,**** > > I am testing to set 2 namenode of 2 separate host with HDFS federation > and a pool block is not to create to startup namenode**** > > log error here is the datanode**** > > ** ** > > 2012-07-25 11:20:39,818 INFO org.apache.hadoop.hdfs.server.common.Storage: > Storage directory /couloir1/tmp/dfs/data is not formatted.**** > > 2012-07-25 11:20:39,818 INFO org.apache.hadoop.hdfs.server.common.Storage: > Formatting ...**** > > 2012-07-25 11:20:39,858 INFO org.apache.hadoop.hdfs.server.common.Storage: > Locking is disabled**** > > 2012-07-25 11:20:39,858 INFO org.apache.hadoop.hdfs.server.common.Storage: > Storage directory > /couloir1/tmp/dfs/data/current/BP-459972778-172.31.6.11-1343208016681 is > not formatted.**** > > 2012-07-25 11:20:39,858 INFO org.apache.hadoop.hdfs.server.common.Storage: > Formatting ...**** > > 2012-07-25 11:20:39,858 INFO org.apache.hadoop.hdfs.server.common.Storage: > Formatting block pool BP-459972778-172.31.6.11-1343208016681 directory > /couloir1/tmp/dfs/data/current/BP-459972778-172.31.6.11-1343208016681/current > **** > > 2012-07-25 11:20:39,860 INFO > org.apache.hadoop.hdfs.server.datanode.DataNode: Setting up storage: > nsid=91302135;bpid=BP-459972778-172.31.6.11-1343208016681;lv=-40;nsInfo=lv=-40;cid=CID-345eacf9-d084-4d24-84f0-8294ca7174d8;nsid=91302135;c=0;bpid=BP-459972778-172.31.6.11-1343208016681 > **** > > 2012-07-25 11:20:39,870 INFO > org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl: Added > volume - /couloir1/tmp/dfs/data/current**** > > 2012-07-25 11:20:39,876 INFO > org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl: > Registered FSDatasetState MBean**** > > 2012-07-25 11:20:39,880 INFO > org.apache.hadoop.hdfs.server.datanode.DirectoryScanner: Periodic Directory > Tree Verification scan starting at 1343224800880 with interval 21600000*** > * > > 2012-07-25 11:20:39,881 INFO > org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl: Adding > block pool BP-459972778-172.31.6.11-1343208016681**** > > 2012-07-25 11:20:39,894 INFO > org.apache.hadoop.hdfs.server.datanode.DataNode: Block pool > BP-459972778-172.31.6.11-1343208016681 (storage id > DS-1163140142-172.31.6.10-50010-1343208039823) service to hadoop-n10/ > 172.31.6.11:33333 beginning handshake with NN**** > > 2012-07-25 11:20:39,899 FATAL > org.apache.hadoop.hdfs.server.datanode.DataNode: Initialization failed for > block pool Block pool BP-321278605-172.31.4.59-1343207919699 (storage id > DS-1163140142-172.31.6.10-50010-1343208039823) service to hadoop-vm03/ > 172.31.4.59:33333**** > > java.io.IOException: Cluster IDs not matched: dn > cid=CID-345eacf9-d084-4d24-84f0-8294ca7174d8 but ns > cid=CID-9f7751ec-615b-4f3b-8644-5583ba263a0e; > bpid=BP-321278605-172.31.4.59-1343207919699**** > > at > org.apache.hadoop.hdfs.server.datanode.DataNode.setClusterId(DataNode.java:307) > **** > > at > org.apache.hadoop.hdfs.server.datanode.DataNode.initBlockPool(DataNode.java:801) > **** > > at > org.apache.hadoop.hdfs.server.datanode.BPOfferService.verifyAndSetNamespaceInfo(BPOfferService.java:308) > **** > > at > org.apache.hadoop.hdfs.server.datanode.BPServiceActor.connectToNNAndHandshake(BPServiceActor.java:217) > **** > > at > org.apache.hadoop.hdfs.server.datanode.BPServiceActor.run(BPServiceActor.java:660) > **** > > at java.lang.Thread.run(Thread.java:662)**** > > 2012-07-25 11:20:39,899 WARN > org.apache.hadoop.hdfs.server.datanode.DataNode: Ending block pool service > for: Block pool BP-321278605-172.31.4.59-1343207919699 (storage id > DS-1163140142-172.31.6.10-50010-1343208039823) service to hadoop-vm03/ > 172.31.4.59:33333**** > > 2012-07-25 11:20:39,900 INFO > org.apache.hadoop.hdfs.server.datanode.DataNode: Removed Block pool > BP-321278605-172.31.4.59-1343207919699 (storage id > DS-1163140142-172.31.6.10-50010-1343208039823)**** > > ** ** > > Here config for core-site.xml**** > > ** ** > > <property>**** > > <name>hadoop.tmp.dir</name>**** > > <value>/couloir1/tmp</value>**** > > </property>**** > > ** ** > > Here config for hdfs-site.xml**** > > ** ** > > <property>**** > > <name>dfs.federation.nameservices</name>**** > > <value>ns1,ns2</value>**** > > </property>**** > > <property>**** > > <name>dfs.permissions</name>**** > > <value>false</value>**** > > </property>**** > > <property>**** > > <name>dfs.replication</name>**** > > <value>2</value>**** > > </property>**** > > <property>**** > > <name>dfs.namenode.rpc-address.ns1</name>**** > > <value>hadoop-vm03:33333</value>**** > > </property>**** > > <property>**** > > <name>dfs.namenode.rpc-address.ns2</name>**** > > <value>hadoop-n10:33333</value>**** > > </property>**** > > </configuration>**** > > ** ** > > Help me please **** > > Thank you**** > > ** ** > > ** ** >
