hi,
thanks for the reply,
I stopped all the processes,
moved the hadoop-data-node folders from all,
moved hadoop-name-node folder on the namenode machine
and formatted the namenode.
still some machines, the same as before has no storageID in
current/VERSION file.
these machine are'nt shown in the web-GUI.
and this is the last line from the datanode log file:
2012-08-30 10:05:29,430 INFO
org.apache.hadoop.hdfs.server.datanode.DataNode: dnRegistration =
DatanodeRegistration(datanode-machine-name.cs.tau.ac.il:50010,
storageID=, infoPort=50075, ipcPort=50020)
any ideas ?
--
Boaz Yarom
CS System Team
03-640-8961 / 7637
Quoting Arpit Gupta <[email protected]>:
2. how/where does the namenode stores the datanodes's storageids ?
When the datanode connects with the namenode for the first time it
will register with the namenode and during the registration the of
the datanode the storage id for the datanode is generated.
4. can I format/reset the namenode to enable the datanode to reconnect ?
If you do not care about the existing data you can certainly do
that. Make sure you stop all processes, format the namenode, clean
up the data directories on all your datanodes (dfs.data.dir prop in
hdfs-site.xml) and then start the namenode and the datanodes.
--
Arpit Gupta
Hortonworks Inc.
http://hortonworks.com/
On Aug 28, 2012, at 9:13 AM, [email protected] wrote:
Hi,
hope it's not a newby question...
I installed several versions of hadoop for testing,
(0.20.203, 0.21.0, and 1.0.3)
on various machines.
now I am using 1.0.3 on all the machines,
I face a problem that in some of the machhines, the datanode gets no
storageID from the namenode.
where it works, the datanode has the following lines in the log file:
(and current/VERSION has a storageID=< some ID > )
---------------------------------------------------------------
2012-08-28 19:04:31,415 INFO
org.apache.hadoop.hdfs.server.datanode.DataNode: dnRegistration =
DatanodeRegistration(datanode-works.cs.tau.ac.il:50010,
storageID=DS-996163017-<machines-ip>-50010-1342683478942,
infoPort=50075, ipcPort=50020)
2012-08-28 19:04:31,418 INFO
org.apache.hadoop.hdfs.server.datanode.DataNode: Starting
asynchronous block report scan
2012-08-28 19:04:31,418 INFO
org.apache.hadoop.hdfs.server.datanode.DataNode:
DatanodeRegistration(<machines-ip>:50010,
storageID=DS-996163017-<machines-ip>-50010-1342683478942,
infoPort=50075, ipcPort=50020)In DataNode.run, data =
FSDataset{dirpath='/var/cache/hdfs/hadoop-data-node/current'}
2012-08-28 19:04:31,419 INFO org.apache.hadoop.ipc.Server: IPC
Server Responder: starting
-------------------------------------------------------------------------------
where it doesn't work I have only the first line and it hangs.
(and current/VERSION has a 'storageID=' empty value line )
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
2012-08-28 18:42:01,297 INFO
org.apache.hadoop.hdfs.server.datanode.DataNode: dnRegistration =
DatanodeRegistration(machinename.cs.tau.ac.il:50010, storageID=,
infoPort=50075, ipcPort=50020)
2012-08-28 18:42:01,287 INFO org.apache.hadoop.ipc.Server: Starting
SocketReader
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
1. any Ideas?
2. how/where does the namenode stores the datanodes's storageids ?
3. how can I get a new storageid for a datanode or it's old ID ?
4. can I format/reset the namenode to enable the datanode to reconnect ?
thanks!
-
Boaz Yarom
CS System Team
03-640-8961 / 7637
