I don't know the deep answer, but formatting your dfs creates a new namespaceId that needs to be consistent across all slaves. Any data directories containing old version ids will prevent the DataNode from starting on that node. Maybe somebody who really knows the machinery can elaborate to this.
Glad you are flying now, Jeff > -----Original Message----- > From: Colin Freas [mailto:[EMAIL PROTECTED] > Sent: Friday, March 21, 2008 1:51 PM > To: [email protected] > Subject: Re: Master as DataNode > > yup, got it working with that technique. > > pushed it out to 5 machines, things look good. appreciate the help. > > what is it that causes this? i know i formatted the dfs more than once. > is > that what does it? or just adding nodes, or... ? > > -colin > > > On Fri, Mar 21, 2008 at 2:30 PM, Jeff Eastman <[EMAIL PROTECTED]> > wrote: > > > I encountered this while I was starting out too, while moving from a > > single > > node cluster to more nodes. I suggest clearing your hadoop-datastore > > directory, reformatting the HDFS and restarting again. You are very > close > > :) > > Jeff > > > > > -----Original Message----- > > > From: Colin Freas [mailto:[EMAIL PROTECTED] > > > Sent: Friday, March 21, 2008 11:18 AM > > > To: [email protected] > > > Subject: Re: Master as DataNode > > > > > > ah: > > > > > > 2008-03-21 14:06:05,526 ERROR org.apache.hadoop.dfs.DataNode: > > > java.io.IOException: Incompatible namespaceIDs in > > > /var/tmp/hadoop-datastore/hadoop/dfs/data: namenode namespaceID = > > > 2121666262; datanode namespaceID = 2058961420 > > > > > > > > > looks like i'm hitting this "Incompatible namespaceID" bug: > > > http://issues.apache.org/jira/browse/HADOOP-1212 > > > > > > is there a work around for this? > > > > > > -colin > > > > > > > > > On Fri, Mar 21, 2008 at 1:50 PM, Jeff Eastman < > > [EMAIL PROTECTED]> > > > wrote: > > > > > > > Check your logs. That should work out of the box with the > > configuration > > > > steps you described. > > > > > > > > Jeff > > > > > > > > > -----Original Message----- > > > > > From: Colin Freas [mailto:[EMAIL PROTECTED] > > > > > Sent: Friday, March 21, 2008 10:40 AM > > > > > To: [email protected] > > > > > Subject: Master as DataNode > > > > > > > > > > setting up a simple hadoop cluster with two machines, i've gotten > to > > > the > > > > > point where the two machines can see each other, things seem fine, > > but > > > > i'm > > > > > trying to set up the master as both a master and a slave, just for > > > > testing > > > > > purposes. > > > > > > > > > > so, i've put the master into the conf/masters file and the > > conf/slaves > > > > > file. > > > > > > > > > > things seem to work, but there's no DataNode process listed with > jps > > > on > > > > > the > > > > > master. i'm wondering if there's a switch i need to flip to tell > > > hadoop > > > > > to > > > > > use the master as a datanode even if it's in the slaves file? > > > > > > > > > > thanks again. > > > > > > > > > > -colin > > > > > > > > > > > > > > > > > >
