Thanks, Steve. I'll see what I can find out about this virtual network adapter.
To make matters more interesting, I tried to install hadoop on another OS X box with parallel desktop installed, and installation proceeded normally. Datanodes started up fine on that machine and everything seemed to be in working order. This tells me that some internal state changed on the first machine in some way that hadoop does not like. I think my next step is reinstalling the 64-bit Java update for OS X, and seeing if that fixes it. I'll let the list know what fix works once I find one. Thanks again for all the responses! -SM On Tue, Feb 17, 2009 at 2:09 PM, zander1013 <zander1...@gmail.com> wrote: > > Workaroung 1 resolved the issue. > > -zanber > > Rasit OZDAS wrote: > > > > Sandy, I have no idea about your issue :( > > > > Zander, > > Your problem is probably about this JIRA issue: > > http://issues.apache.org/jira/browse/HADOOP-1212 > > > > Here is 2 workarounds explained: > > > http://www.michael-noll.com/wiki/Running_Hadoop_On_Ubuntu_Linux_(Multi-Node_Cluster)#java.io.IOException:_Incompatible_namespaceIDs<http://www.michael-noll.com/wiki/Running_Hadoop_On_Ubuntu_Linux_%28Multi-Node_Cluster%29#java.io.IOException:_Incompatible_namespaceIDs> > > > > I haven't tried it, hope it helps. > > Rasit > > > > 2009/2/17 zander1013 <zander1...@gmail.com>: > >> > >> hi, > >> > >> i am not seeing the DataNode run either. but i am seeing an extra > process > >> TaskTracker run. > >> > >> here is what hapens when i start the cluster run jps and stop the > >> cluster... > >> > >> had...@node0:/usr/local/hadoop$ bin/start-all.sh > >> starting namenode, logging to > >> /usr/local/hadoop/bin/../logs/hadoop-hadoop-namenode-node0.out > >> node0.local: starting datanode, logging to > >> /usr/local/hadoop/bin/../logs/hadoop-hadoop-datanode-node0.out > >> node1.local: starting datanode, logging to > >> /usr/local/hadoop/bin/../logs/hadoop-hadoop-datanode-node1.out > >> node0.local: starting secondarynamenode, logging to > >> /usr/local/hadoop/bin/../logs/hadoop-hadoop-secondarynamenode-node0.out > >> starting jobtracker, logging to > >> /usr/local/hadoop/bin/../logs/hadoop-hadoop-jobtracker-node0.out > >> node0.local: starting tasktracker, logging to > >> /usr/local/hadoop/bin/../logs/hadoop-hadoop-tasktracker-node0.out > >> node1.local: starting tasktracker, logging to > >> /usr/local/hadoop/bin/../logs/hadoop-hadoop-tasktracker-node1.out > >> had...@node0:/usr/local/hadoop$ jps > >> 13353 TaskTracker > >> 13126 SecondaryNameNode > >> 12846 NameNode > >> 13455 Jps > >> 13232 JobTracker > >> had...@node0:/usr/local/hadoop$ bin/stop-all.sh > >> stopping jobtracker > >> node0.local: stopping tasktracker > >> node1.local: stopping tasktracker > >> stopping namenode > >> node0.local: no datanode to stop > >> node1.local: no datanode to stop > >> node0.local: stopping secondarynamenode > >> had...@node0:/usr/local/hadoop$ > >> > >> here is the tail of the log file for the session above... > >> ************************************************************/ > >> 2009-02-16 19:35:13,999 INFO > >> org.apache.hadoop.hdfs.server.datanode.DataNode: STARTUP_MSG: > >> /************************************************************ > >> STARTUP_MSG: Starting DataNode > >> STARTUP_MSG: host = node1/127.0.1.1 > >> STARTUP_MSG: args = [] > >> STARTUP_MSG: version = 0.19.0 > >> STARTUP_MSG: build = > >> https://svn.apache.org/repos/asf/hadoop/core/branches/branch-0.19 -r > >> 713890; > >> compiled by 'ndaley' on Fri Nov 14 03:12:29 UTC 2008 > >> ************************************************************/ > >> 2009-02-16 19:35:18,999 ERROR > >> org.apache.hadoop.hdfs.server.datanode.DataNode: java.io.IOException: > >> Incompatible namespaceIDs in > >> /usr/local/hadoop-datastore/hadoop-hadoop/dfs/data: namenode namespaceID > >> = > >> 1050914495; datanode namespaceID = 722953254 > >> at > >> > org.apache.hadoop.hdfs.server.datanode.DataStorage.doTransition(DataStorage.java:233) > >> at > >> > org.apache.hadoop.hdfs.server.datanode.DataStorage.recoverTransitionRead(DataStorage.java:148) > >> at > >> > org.apache.hadoop.hdfs.server.datanode.DataNode.startDataNode(DataNode.java:287) > >> at > >> > org.apache.hadoop.hdfs.server.datanode.DataNode.<init>(DataNode.java:205) > >> at > >> > org.apache.hadoop.hdfs.server.datanode.DataNode.makeInstance(DataNode.java:1199) > >> at > >> > org.apache.hadoop.hdfs.server.datanode.DataNode.instantiateDataNode(DataNode.java:1154) > >> at > >> > org.apache.hadoop.hdfs.server.datanode.DataNode.createDataNode(DataNode.java:1162) > >> at > >> org.apache.hadoop.hdfs.server.datanode.DataNode.main(DataNode.java:1284) > >> > >> 2009-02-16 19:35:19,000 INFO > >> org.apache.hadoop.hdfs.server.datanode.DataNode: SHUTDOWN_MSG: > >> /************************************************************ > >> SHUTDOWN_MSG: Shutting down DataNode at node1/127.0.1.1 > >> ************************************************************/ > >> > >> i have not seen DataNode run yet. i have only started and stopped the > >> cluster a couple of times. > >> > >> i tried to reformat datanode and namenode with bin/hadoop datanode > >> -format > >> and bin/hadoop namenode -format from /usr/local/hadoop dir. > >> > >> please advise > >> > >> zander > >> > >> > >> > >> Mithila Nagendra wrote: > >>> > >>> Hey Sandy > >>> I had a similar problem with Hadoop. All I did was I stopped all the > >>> daemons > >>> using stop-all.sh. Then formatted the namenode again using hadoop > >>> namenode > >>> -format. After this I went on to restarting everything by using > >>> start-all.sh > >>> > >>> I hope you dont have much data on the datanode, reformatting it would > >>> erase > >>> everything out. > >>> > >>> Hope this helps! > >>> Mithila > >>> > >>> > >>> > >>> On Sat, Feb 14, 2009 at 2:39 AM, james warren <ja...@rockyou.com> > wrote: > >>> > >>>> Sandy - > >>>> > >>>> I suggest you take a look into your NameNode and DataNode logs. From > >>>> the > >>>> information posted, these likely would be at > >>>> > >>>> > >>>> > /Users/hadoop/hadoop-0.18.2/bin/../logs/hadoop-hadoop-namenode-loteria.cs.tamu.edu.log > >>>> > >>>> > /Users/hadoop/hadoop-0.18.2/bin/../logs/hadoop-hadoop-jobtracker-loteria.cs.tamu.edu.log > >>>> > >>>> If the cause isn't obvious from what you see there, could you please > >>>> post > >>>> the last few lines from each log? > >>>> > >>>> -jw > >>>> > >>>> On Fri, Feb 13, 2009 at 3:28 PM, Sandy <snickerdoodl...@gmail.com> > >>>> wrote: > >>>> > >>>> > Hello, > >>>> > > >>>> > I would really appreciate any help I can get on this! I've suddenly > >>>> ran > >>>> > into > >>>> > a very strange error. > >>>> > > >>>> > when I do: > >>>> > bin/start-all > >>>> > I get: > >>>> > hadoop$ bin/start-all.sh > >>>> > starting namenode, logging to > >>>> > > >>>> > > >>>> > /Users/hadoop/hadoop-0.18.2/bin/../logs/hadoop-hadoop-namenode-loteria.cs.tamu.edu.out > >>>> > starting jobtracker, logging to > >>>> > > >>>> > > >>>> > /Users/hadoop/hadoop-0.18.2/bin/../logs/hadoop-hadoop-jobtracker-loteria.cs.tamu.edu.out > >>>> > > >>>> > No datanode, secondary namenode or jobtracker are being started. > >>>> > > >>>> > When I try to upload anything on the dfs, I get a "node in safemode" > >>>> error > >>>> > (even after waiting 5 minutes), presumably because it's trying to > >>>> reach > >>>> a > >>>> > datanode that does not exist. The same "safemode" error occurs when > >>>> I > >>>> try > >>>> > to run jobs. > >>>> > > >>>> > I have tried bin/stop-all and then bin/start-all again. I get the > >>>> same > >>>> > problem! > >>>> > > >>>> > This is incredibly strange, since I was previously able to start and > >>>> run > >>>> > jobs without any issue using this version on this machine. I am > >>>> running > >>>> > jobs > >>>> > on a single Mac Pro running OS X 10.5 > >>>> > > >>>> > I have tried updating to hadoop-0.19.0, and I get the same problem. > I > >>>> have > >>>> > even tried this using previous versions, and I'm getting the same > >>>> problem! > >>>> > > >>>> > Anyone have any idea why this suddenly could be happening? What am I > >>>> doing > >>>> > wrong? > >>>> > > >>>> > For convenience, I'm including portions of both conf/hadoop-env.sh > >>>> and > >>>> > conf/hadoop-site.xml: > >>>> > > >>>> > --- hadoop-env.sh --- > >>>> > # Set Hadoop-specific environment variables here. > >>>> > > >>>> > # The only required environment variable is JAVA_HOME. All others > >>>> are > >>>> > # optional. When running a distributed configuration it is best to > >>>> > # set JAVA_HOME in this file, so that it is correctly defined on > >>>> > # remote nodes. > >>>> > > >>>> > # The java implementation to use. Required. > >>>> > export > >>>> > > >>>> > JAVA_HOME=/System/Library/Frameworks/JavaVM.framework/Versions/1.6.0/Home > >>>> > > >>>> > # Extra Java CLASSPATH elements. Optional. > >>>> > # export HADOOP_CLASSPATH= > >>>> > > >>>> > # The maximum amount of heap to use, in MB. Default is 1000. > >>>> > export HADOOP_HEAPSIZE=3000 > >>>> > ... > >>>> > --- hadoop-site.xml --- > >>>> > <configuration> > >>>> > > >>>> > <property> > >>>> > <name>hadoop.tmp.dir</name> > >>>> > <value>/Users/hadoop/hadoop-0.18.2/hadoop-${user.name}</value> > >>>> > <description>A base for other temporary directories.</description> > >>>> > </property> > >>>> > > >>>> > <property> > >>>> > <name>fs.default.name</name> > >>>> > <value>hdfs://localhost:9000</value> > >>>> > <description>The name of the default file system. A URI whose > >>>> > scheme and authority determine the FileSystem implementation. The > >>>> > uri's scheme determines the config property (fs.SCHEME.impl) naming > >>>> > the FileSystem implementation class. The uri's authority is used > to > >>>> > determine the host, port, etc. for a filesystem.</description> > >>>> > </property> > >>>> > > >>>> > <property> > >>>> > <name>mapred.job.tracker</name> > >>>> > <value>localhost:9001</value> > >>>> > <description>The host and port that the MapReduce job tracker runs > >>>> > at. If "local", then jobs are run in-process as a single map > >>>> > and reduce task. > >>>> > </description> > >>>> > </property> > >>>> > > >>>> > <property> > >>>> > <name>mapred.tasktracker.tasks.maximum</name> > >>>> > <value>1</value> > >>>> > <description>The maximum number of tasks that will be run > >>>> simultaneously > >>>> by > >>>> > a > >>>> > a task tracker > >>>> > </description> > >>>> > </property> > >>>> > ... > >>>> > > >>>> > >>> > >>> > >> > >> -- > >> View this message in context: > >> > http://www.nabble.com/datanode-not-being-started-tp22006929p22049288.html > >> Sent from the Hadoop core-user mailing list archive at Nabble.com. > >> > >> > > > > > > > > -- > > M. Raşit ÖZDAŞ > > > > > > -- > View this message in context: > http://www.nabble.com/datanode-not-being-started-tp22006929p22065291.html > Sent from the Hadoop core-user mailing list archive at Nabble.com. > >
