I was following the quickstart guide to run pseudo-distributed operations with
Hadoop 0.16.4. I got it to work successfully the first time. But I failed to
repeat the steps (I tried to re-do everything from re-formating the HDFS). Then
by looking at the log files of the daemons, I found out the datanode failed to
start because its namespaceID didn't match with the namenode's. I after that
found that the namespaceID is stored in the text file VERSION under
dfs/data/current and dfs/name/current for the datanode and the namenode,
respectively. The reformatting step does change namespaceID of the namenode,
but not for the datanode, and that's the cause for the inconsistency. So after
reformatting, if I manually update namespaceID for the datanode, things will
work totally fine again.
I guess there are probably others who had this same experience. Is it a bug in
Hadoop 0.16.4? If so, has it been taken care of in later versions?
Thanks,
David.
