Is it the directory that Hadoop uses? /tmp/hadoop-oracle /tmp/hadoop-oracle/dfs/ /tmp/hadoop-oracle/mapred/
If yes, how can I change the directory to anywhere else? I do not want it to be kept in /tmp folder. > From: [email protected] > Date: Mon, 3 Aug 2009 17:02:50 -0700 > Subject: Re: Problem with starting Hadoop in Pseudo Distributed Mode > To: [email protected] > > I'm assuming that you have no data in HDFS since it never came up... So, go > ahead and clean up the directory where you are storing the datanode's data > and the namenode's metadata. After that format the namenode and restart > hadoop. > > > 2009/8/3 Onur AKTAS <[email protected]> > > > > > Hi, > > > > I'm having troubles with running Hadoop in RHEL 5, I did everything as > > documented in: > > http://hadoop.apache.org/common/docs/r0.20.0/quickstart.html > > > > And configured: > > conf/core-site.xml, conf/hdfs-site.xml, > > conf/mapred-site.xml. > > > > Connected to "localhost" with ssh (did passphrase stuff etc.), then I did > > the following: > > > > $ bin/hadoop namenode -format > > $ bin/start-all.sh > > starting namenode, logging to > > /hda3/ps/hadoop-0.20.0/bin/../logs/hadoop-oracle-namenode-localhost.localdomain.out > > localhost: starting datanode, logging to > > /hda3/ps/hadoop-0.20.0/bin/../logs/hadoop-oracle-datanode-localhost.localdomain.out > > localhost: starting secondarynamenode, logging to > > /hda3/ps/hadoop-0.20.0/bin/../logs/hadoop-oracle-secondarynamenode-localhost.localdomain.out > > starting jobtracker, logging to > > /hda3/ps/hadoop-0.20.0/bin/../logs/hadoop-oracle-jobtracker-localhost.localdomain.out > > localhost: starting tasktracker, logging to > > /hda3/ps/hadoop-0.20.0/bin/../logs/hadoop-oracle-tasktracker-localhost.localdomain.out > > > > Everything seems ok, but when I check the Hadoop Logs I see many errors. > > (and they all cause HBase connection problems.) > > How can I solve this problem? Here are the Logs > > > > hadoop-oracle-datanode-localhost.localdomain.log: > > 2009-08-04 02:54:28,971 INFO > > org.apache.hadoop.hdfs.server.datanode.DataNode: STARTUP_MSG: > > /************************************************************ > > STARTUP_MSG: Starting DataNode > > STARTUP_MSG: host = localhost.localdomain/127.0.0.1 > > STARTUP_MSG: args = [] > > STARTUP_MSG: version = 0.20.0 > > STARTUP_MSG: build = > > https://svn.apache.org/repos/asf/hadoop/core/branches/branch-0.20 -r > > 763504; compiled by 'ndaley' on Thu Apr 9 05:18:40 UTC 2009 > > ************************************************************/ > > 2009-08-04 02:54:29,562 ERROR > > org.apache.hadoop.hdfs.server.datanode.DataNode: java.io.IOException: > > Incompatible namespaceIDs in /tmp/hadoop-oracle/dfs/data: namenode > > namespaceID = 36527197; datanode namespaceID = 2138759529 > > at > > org.apache.hadoop.hdfs.server.datanode.DataStorage.doTransition(DataStorage.java:233) > > at > > org.apache.hadoop.hdfs.server.datanode.DataStorage.recoverTransitionRead(DataStorage.java:148) > > at > > org.apache.hadoop.hdfs.server.datanode.DataNode.startDataNode(DataNode.java:298) > > at > > org.apache.hadoop.hdfs.server.datanode.DataNode.<init>(DataNode.java:216) > > at > > org.apache.hadoop.hdfs.server.datanode.DataNode.makeInstance(DataNode.java:1283) > > at > > org.apache.hadoop.hdfs.server.datanode.DataNode.instantiateDataNode(DataNode.java:1238) > > at > > org.apache.hadoop.hdfs.server.datanode.DataNode.createDataNode(DataNode.java:1246) > > at > > org.apache.hadoop.hdfs.server.datanode.DataNode.main(DataNode.java:1368) > > > > 2009-08-04 02:54:29,563 INFO > > org.apache.hadoop.hdfs.server.datanode.DataNode: SHUTDOWN_MSG: > > /************************************************************ > > SHUTDOWN_MSG: Shutting down DataNode at localhost.localdomain/127.0.0.1 > > ************************************************************/ > > > > ------------------------------------------------------------------------------------------ > > hadoop-oracle-namenode-localhost.localdomain.log > > 2009-08-04 02:54:26,987 INFO > > org.apache.hadoop.hdfs.server.namenode.NameNode: STARTUP_MSG: > > /************************************************************ > > STARTUP_MSG: Starting NameNode > > STARTUP_MSG: host = localhost.localdomain/127.0.0.1 > > STARTUP_MSG: args = [] > > STARTUP_MSG: version = 0.20.0 > > STARTUP_MSG: build = > > https://svn.apache.org/repos/asf/hadoop/core/branches/branch-0.20 -r > > 763504; compiled by 'ndaley' on Thu Apr 9 05:18:40 UTC 2009 > > ************************************************************/ > > 2009-08-04 02:54:27,116 INFO org.apache.hadoop.ipc.metrics.RpcMetrics: > > Initializing RPC Metrics with hostName=NameNode, port=9000 > > 2009-08-04 02:54:27,174 INFO > > org.apache.hadoop.hdfs.server.namenode.NameNode: Namenode up at: > > localhost.localdomain/127.0.0.1:9000 > > 2009-08-04 02:54:27,179 INFO org.apache.hadoop.metrics.jvm.JvmMetrics: > > Initializing JVM Metrics with processName=NameNode, sessionId=null > > 2009-08-04 02:54:27,180 INFO > > org.apache.hadoop.hdfs.server.namenode.metrics.NameNodeMetrics: Initializing > > NameNodeMeterics using context > > object:org.apache.hadoop.metrics.spi.NullContext > > 2009-08-04 02:54:27,278 INFO > > org.apache.hadoop.hdfs.server.namenode.FSNamesystem: > > fsOwner=oracle,oinstall,root,dba,oper,asmadmin > > 2009-08-04 02:54:27,278 INFO > > org.apache.hadoop.hdfs.server.namenode.FSNamesystem: supergroup=supergroup > > 2009-08-04 02:54:27,278 INFO > > org.apache.hadoop.hdfs.server.namenode.FSNamesystem: > > isPermissionEnabled=true > > 2009-08-04 02:54:27,294 INFO > > org.apache.hadoop.hdfs.server.namenode.metrics.FSNamesystemMetrics: > > Initializing FSNamesystemMetrics using context > > object:org.apache.hadoop.metrics.spi.NullContext > > 2009-08-04 02:54:27,297 INFO > > org.apache.hadoop.hdfs.server.namenode.FSNamesystem: Registered > > FSNamesystemStatusMBean > > 2009-08-04 02:54:27,341 INFO org.apache.hadoop.hdfs.server.common.Storage: > > Number of files = 8 > > 2009-08-04 02:54:27,348 INFO org.apache.hadoop.hdfs.server.common.Storage: > > Number of files under construction = 2 > > 2009-08-04 02:54:27,351 INFO org.apache.hadoop.hdfs.server.common.Storage: > > Image file of size 923 loaded in 0 seconds. > > 2009-08-04 02:54:27,351 INFO org.apache.hadoop.hdfs.server.common.Storage: > > Edits file /tmp/hadoop-oracle/dfs/name/current/edits of size 4 edits # 0 > > loaded in 0 seconds. > > 2009-08-04 02:54:27,435 INFO org.apache.hadoop.hdfs.server.common.Storage: > > Image file of size 923 saved in 0 seconds. > > 2009-08-04 02:54:27,495 INFO > > org.apache.hadoop.hdfs.server.namenode.FSNamesystem: Finished loading > > FSImage in 262 msecs > > 2009-08-04 02:54:27,496 INFO > > org.apache.hadoop.hdfs.server.namenode.FSNamesystem: Total number of blocks > > = 0 > > 2009-08-04 02:54:27,496 INFO > > org.apache.hadoop.hdfs.server.namenode.FSNamesystem: Number of invalid > > blocks = 0 > > 2009-08-04 02:54:27,497 INFO > > org.apache.hadoop.hdfs.server.namenode.FSNamesystem: Number of > > under-replicated blocks = 0 > > 2009-08-04 02:54:27,497 INFO > > org.apache.hadoop.hdfs.server.namenode.FSNamesystem: Number of > > over-replicated blocks = 0 > > 2009-08-04 02:54:27,497 INFO org.apache.hadoop.hdfs.StateChange: STATE* > > Leaving safe mode after 0 secs. > > 2009-08-04 02:54:27,497 INFO org.apache.hadoop.hdfs.StateChange: STATE* > > Network topology has 0 racks and 0 datanodes > > 2009-08-04 02:54:27,497 INFO org.apache.hadoop.hdfs.StateChange: STATE* > > UnderReplicatedBlocks has 0 blocks > > 2009-08-04 02:54:27,696 INFO org.mortbay.log: Logging to > > org.slf4j.impl.Log4jLoggerAdapter(org.mortbay.log) via > > org.mortbay.log.Slf4jLog > > 2009-08-04 02:54:27,775 INFO org.apache.hadoop.http.HttpServer: Jetty bound > > to port 50070 > > 2009-08-04 02:54:27,775 INFO org.mortbay.log: jetty-6.1.14 > > 2009-08-04 02:54:28,277 INFO org.mortbay.log: Started > > [email protected]:50070 > > 2009-08-04 02:54:28,278 INFO > > org.apache.hadoop.hdfs.server.namenode.NameNode: Web-server up at: > > 0.0.0.0:50070 > > 2009-08-04 02:54:28,278 INFO org.apache.hadoop.ipc.Server: IPC Server > > Responder: starting > > 2009-08-04 02:54:28,279 INFO org.apache.hadoop.ipc.Server: IPC Server > > listener on 9000: starting > > 2009-08-04 02:54:28,280 INFO org.apache.hadoop.ipc.Server: IPC Server > > handler 0 on 9000: starting > > 2009-08-04 02:54:28,280 INFO org.apache.hadoop.ipc.Server: IPC Server > > handler 1 on 9000: starting > > 2009-08-04 02:54:28,316 INFO org.apache.hadoop.ipc.Server: IPC Server > > handler 2 on 9000: starting > > 2009-08-04 02:54:28,316 INFO org.apache.hadoop.ipc.Server: IPC Server > > handler 3 on 9000: starting > > 2009-08-04 02:54:28,321 INFO org.apache.hadoop.ipc.Server: IPC Server > > handler 4 on 9000: starting > > 2009-08-04 02:54:28,321 INFO org.apache.hadoop.ipc.Server: IPC Server > > handler 5 on 9000: starting > > 2009-08-04 02:54:28,328 INFO org.apache.hadoop.ipc.Server: IPC Server > > handler 6 on 9000: starting > > 2009-08-04 02:54:28,361 INFO org.apache.hadoop.ipc.Server: IPC Server > > handler 7 on 9000: starting > > 2009-08-04 02:54:28,362 INFO org.apache.hadoop.ipc.Server: IPC Server > > handler 8 on 9000: starting > > 2009-08-04 02:54:28,366 INFO org.apache.hadoop.ipc.Server: IPC Server > > handler 9 on 9000: starting > > 2009-08-04 02:54:38,433 INFO > > org.apache.hadoop.hdfs.server.namenode.FSNamesystem.audit: > > ugi=oracle,oinstall,root,dba,oper,asmadmin ip=/127.0.0.1 > > cmd=listStatus src=/tmp/hadoop-oracle/mapred/system dst=null > > perm=null > > 2009-08-04 02:54:38,755 INFO > > org.apache.hadoop.hdfs.server.namenode.FSNamesystem.audit: > > ugi=oracle,oinstall,root,dba,oper,asmadmin ip=/127.0.0.1 cmd=delete > > src=/tmp/hadoop-oracle/mapred/system dst=null perm=null > > 2009-08-04 02:54:38,773 INFO > > org.apache.hadoop.hdfs.server.namenode.FSNamesystem.audit: > > ugi=oracle,oinstall,root,dba,oper,asmadmin ip=/127.0.0.1 cmd=mkdirs > > src=/tmp/hadoop-oracle/mapred/system dst=null > > perm=oracle:supergroup:rwxr-xr-x > > 2009-08-04 02:54:38,785 INFO > > org.apache.hadoop.hdfs.server.namenode.FSNamesystem.audit: > > ugi=oracle,oinstall,root,dba,oper,asmadmin ip=/127.0.0.1 > > cmd=setPermission src=/tmp/hadoop-oracle/mapred/system dst=null > > perm=oracle:supergroup:rwx-wx-wx > > 2009-08-04 02:54:38,862 INFO > > org.apache.hadoop.hdfs.server.namenode.FSNamesystem.audit: > > ugi=oracle,oinstall,root,dba,oper,asmadmin ip=/127.0.0.1 cmd=create > > src=/tmp/hadoop-oracle/mapred/system/jobtracker.info dst=null > > perm=oracle:supergroup:rw-r--r-- > > 2009-08-04 02:54:38,900 INFO > > org.apache.hadoop.hdfs.server.namenode.FSNamesystem.audit: > > ugi=oracle,oinstall,root,dba,oper,asmadmin ip=/127.0.0.1 > > cmd=setPermission src=/tmp/hadoop-oracle/mapred/system/jobtracker.info > > dst=null perm=oracle:supergroup:rw------- > > 2009-08-04 02:54:38,955 INFO org.apache.hadoop.ipc.Server: IPC Server > > handler 4 on 9000, call addBlock(/tmp/hadoop-oracle/mapred/system/ > > jobtracker.info, DFSClient_-603868025) from 127.0.0.1:51803: error: > > java.io.IOException: File > > /tmp/hadoop-oracle/mapred/system/jobtracker.infocould only be replicated to > > 0 nodes, instead of 1 > > java.io.IOException: File > > /tmp/hadoop-oracle/mapred/system/jobtracker.infocould only be replicated to > > 0 nodes, instead of 1 > > at > > org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getAdditionalBlock(FSNamesystem.java:1256) > > at > > org.apache.hadoop.hdfs.server.namenode.NameNode.addBlock(NameNode.java:422) > > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > > at > > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) > > at > > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) > > at java.lang.reflect.Method.invoke(Method.java:597) > > at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:508) > > at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:959) > > at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:955) > > at java.security.AccessController.doPrivileged(Native Method) > > at javax.security.auth.Subject.doAs(Subject.java:396) > > at org.apache.hadoop.ipc.Server$Handler.run(Server.java:953) > > 2009-08-04 02:54:39,548 INFO org.apache.hadoop.ipc.Server: IPC Server > > handler 5 on 9000, call addBlock(/tmp/hadoop-oracle/mapred/system/ > > jobtracker.info, DFSClient_-603868025) from 127.0.0.1:51803: error: > > java.io.IOException: File > > /tmp/hadoop-oracle/mapred/system/jobtracker.infocould only be replicated to > > 0 nodes, instead of 1 > > java.io.IOException: File > > /tmp/hadoop-oracle/mapred/system/jobtracker.infocould only be replicated to > > 0 nodes, instead of 1 > > at > > org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getAdditionalBlock(FSNamesystem.java:1256) > > at > > org.apache.hadoop.hdfs.server.namenode.NameNode.addBlock(NameNode.java:422) > > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > > at > > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) > > at > > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) > > at java.lang.reflect.Method.invoke(Method.java:597) > > at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:508) > > at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:959) > > at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:955) > > at java.security.AccessController.doPrivileged(Native Method) > > at javax.security.auth.Subject.doAs(Subject.java:396) > > at org.apache.hadoop.ipc.Server$Handler.run(Server.java:953) > > 2009-08-04 02:54:40,359 INFO org.apache.hadoop.ipc.Server: IPC Server > > handler 6 on 9000, call addBlock(/tmp/hadoop-oracle/mapred/system/ > > jobtracker.info, DFSClient_-603868025) from 127.0.0.1:51803: error: > > java.io.IOException: File > > /tmp/hadoop-oracle/mapred/system/jobtracker.infocould only be replicated to > > 0 nodes, instead of 1 > > java.io.IOException: File > > /tmp/hadoop-oracle/mapred/system/jobtracker.infocould only be replicated to > > 0 nodes, instead of 1 > > at > > org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getAdditionalBlock(FSNamesystem.java:1256) > > at > > org.apache.hadoop.hdfs.server.namenode.NameNode.addBlock(NameNode.java:422) > > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > > at > > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) > > at > > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) > > at java.lang.reflect.Method.invoke(Method.java:597) > > at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:508) > > at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:959) > > at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:955) > > at java.security.AccessController.doPrivileged(Native Method) > > at javax.security.auth.Subject.doAs(Subject.java:396) > > at org.apache.hadoop.ipc.Server$Handler.run(Server.java:953) > > 2009-08-04 02:54:41,969 INFO org.apache.hadoop.ipc.Server: IPC Server > > handler 7 on 9000, call addBlock(/tmp/hadoop-oracle/mapred/system/ > > jobtracker.info, DFSClient_-603868025) from 127.0.0.1:51803: error: > > java.io.IOException: File > > /tmp/hadoop-oracle/mapred/system/jobtracker.infocould only be replicated to > > 0 nodes, instead of 1 > > java.io.IOException: File > > /tmp/hadoop-oracle/mapred/system/jobtracker.infocould only be replicated to > > 0 nodes, instead of 1 > > at > > org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getAdditionalBlock(FSNamesystem.java:1256) > > at > > org.apache.hadoop.hdfs.server.namenode.NameNode.addBlock(NameNode.java:422) > > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > > at > > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) > > at > > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) > > at java.lang.reflect.Method.invoke(Method.java:597) > > at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:508) > > at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:959) > > at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:955) > > at java.security.AccessController.doPrivileged(Native Method) > > at javax.security.auth.Subject.doAs(Subject.java:396) > > at org.apache.hadoop.ipc.Server$Handler.run(Server.java:953) > > 2009-08-04 02:54:45,180 INFO org.apache.hadoop.ipc.Server: IPC Server > > handler 8 on 9000, call addBlock(/tmp/hadoop-oracle/mapred/system/ > > jobtracker.info, DFSClient_-603868025) from 127.0.0.1:51803: error: > > java.io.IOException: File > > /tmp/hadoop-oracle/mapred/system/jobtracker.infocould only be replicated to > > 0 nodes, instead of 1 > > java.io.IOException: File > > /tmp/hadoop-oracle/mapred/system/jobtracker.infocould only be replicated to > > 0 nodes, instead of 1 > > at > > org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getAdditionalBlock(FSNamesystem.java:1256) > > at > > org.apache.hadoop.hdfs.server.namenode.NameNode.addBlock(NameNode.java:422) > > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > > at > > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) > > at > > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) > > at java.lang.reflect.Method.invoke(Method.java:597) > > at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:508) > > at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:959) > > at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:955) > > at java.security.AccessController.doPrivileged(Native Method) > > at javax.security.auth.Subject.doAs(Subject.java:396) > > at org.apache.hadoop.ipc.Server$Handler.run(Server.java:953) > > > > > > > > _________________________________________________________________ > > Windows Live ile fotoğraflarınızı organize edebilir, düzenleyebilir ve > > paylaşabilirsiniz. > > > > http://www.microsoft.com/turkiye/windows/windowslive/products/photo-gallery-edit.aspx _________________________________________________________________ Windows Live tüm arkadaşlarınızla tek bir yerden iletişim kurmanıza yardımcı olur. http://www.microsoft.com/turkiye/windows/windowslive/products/social-network-connector.aspx
