Here is what I do.
Pseudo-Distributed Operation in:
http://hadoop.apache.org/common/docs/current/quickstart.html
I edit
conf/core-site.xml,
conf/hdfs-site.xm,
conf/mapred-site.xml:
$ bin/hadoop namenode -format
$ bin/start-all.shstarting namenode, logging to
/hda3/ps/hadoop-0.20.0/bin/../logs/hadoop-oracle-namenode-localhost.localdomain.out
localhost: starting datanode, logging to
/hda3/ps/hadoop-0.20.0/bin/../logs/hadoop-oracle-datanode-localhost.localdomain.out
localhost: starting secondarynamenode, logging to
/hda3/ps/hadoop-0.20.0/bin/../logs/hadoop-oracle-secondarynamenode-localhost.localdomain.out
starting jobtracker, logging to
/hda3/ps/hadoop-0.20.0/bin/../logs/hadoop-oracle-jobtracker-localhost.localdomain.out
localhost: starting tasktracker, logging to
/hda3/ps/hadoop-0.20.0/bin/../logs/hadoop-oracle-tasktracker-localhost.localdomain.out
When I check the logs in hadoop-oracle-namenode-localhost.localdomain.log
I see something like
2009-08-03 21:26:20,757 INFO org.apache.hadoop.ipc.Server: IPC Server handler 3
on 9000, call addBlock(/tmp/hadoop-oracle/mapred/system/jobtracker.info,
DFSClient_-1600979110) from 127.0.0.1:22460: error: java.io.IOException: File
/tmp/hadoop-oracle/mapred/system/jobtracker.info could only be replicated to 0
nodes, instead of 1
java.io.IOException: File /tmp/hadoop-oracle/mapred/system/jobtracker.info
could only be replicated to 0 nodes, instead of 1
at
org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getAdditionalBlock(FSNamesystem.java:1256)
at
org.apache.hadoop.hdfs.server.namenode.NameNode.addBlock(NameNode.java:422)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:508)
at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:959)
at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:955)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:396)
at org.apache.hadoop.ipc.Server$Handler.run(Server.java:953)
2009-08-03 21:26:21,177 INFO org.apache.hadoop.ipc.Server: IPC Server handler 5
on 9000, call addBlock(/tmp/hadoop-oracle/mapred/system/jobtracker.info,
DFSClient_-1600979110) from 127.0.0.1:22460: error: java.io.IOException: File
/tmp/hadoop-oracle/mapred/system/jobtracker.info could only be replicated to 0
nodes, instead of 1
java.io.IOException: File /tmp/hadoop-oracle/mapred/system/jobtracker.info
could only be replicated to 0 nodes, instead of 1
at
org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getAdditionalBlock(FSNamesystem.java:1256)
at
org.apache.hadoop.hdfs.server.namenode.NameNode.addBlock(NameNode.java:422)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:508)
at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:959)
at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:955)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:396)
at org.apache.hadoop.ipc.Server$Handler.run(Server.java:953)
2009-08-03 21:26:21,982 INFO org.apache.hadoop.ipc.Server: IPC Server handler 6
on 9000, call addBlock(/tmp/hadoop-oracle/mapred/system/jobtracker.info,
DFSClient_-1600979110) from 127.0.0.1:22460: error: java.io.IOException: File
/tmp/hadoop-oracle/mapred/system/jobtracker.info could only be replicated to 0
nodes, instead of 1
java.io.IOException: File /tmp/hadoop-oracle/mapred/system/jobtracker.info
could only be replicated to 0 nodes, instead of 1
at
org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getAdditionalBlock(FSNamesystem.java:1256)
at
org.apache.hadoop.hdfs.server.namenode.NameNode.addBlock(NameNode.java:422)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:508)
at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:959)
at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:955)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:396)
at org.apache.hadoop.ipc.Server$Handler.run(Server.java:953)
2009-08-03 21:26:23,612 INFO org.apache.hadoop.ipc.Server: IPC Server handler 7
on 9000, call addBlock(/tmp/hadoop-oracle/mapred/system/jobtracker.info,
DFSClient_-1600979110) from 127.0.0.1:22460: error: java.io.IOException: File
/tmp/hadoop-oracle/mapred/system/jobtracker.info could only be replicated to 0
nodes, instead of 1
java.io.IOException: File /tmp/hadoop-oracle/mapred/system/jobtracker.info
could only be replicated to 0 nodes, instead of 1
at
org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getAdditionalBlock(FSNamesystem.java:1256)
at
org.apache.hadoop.hdfs.server.namenode.NameNode.addBlock(NameNode.java:422)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:508)
at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:959)
at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:955)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:396)
at org.apache.hadoop.ipc.Server$Handler.run(Server.java:953)
Whats the problem? And why does Hadoop choose /tmp directory for its file
system? Because of pseudo distributed mode?
Thanks.
> Date: Mon, 3 Aug 2009 09:33:20 -0400
> Subject: Re: Connection failure to HBase
> From: [email protected]
> To: [email protected]
>
> If the client is not able to talk to the Master, it means that
> something wrong happened there that prevents it to start. Look in the
> master's log you should see an exception.
>
> J-D
>
> 2009/8/3 Onur AKTAS <[email protected]>:
> >
> > No, this is what after I changed.
> >
> > I was using like below, but it was not working. It was giving an exception
> > like "INFO: Retrying connect to server: localhost/127.0.0.1:60000. Already
> > tried"
> > <property>
> > <name>hbase.rootdir</name>
> > <value>hdfs://localhost:9000/hbase</value>
> > <description>The directory shared by region servers.
> > </description>
> > </property>
> >
> >
> >> Date: Mon, 3 Aug 2009 08:36:09 -0400
> >> Subject: Re: Connection failure to HBase
> >> From: [email protected]
> >> To: [email protected]
> >>
> >> If this is all of your hbase-site.xml, you're not using Hadoop at all.
> >> Please review the Pseudo-distributed documentation for HBase.
> >>
> >> J-D
> >>
> >> 2009/8/3 Onur AKTAS <[email protected]>:
> >> >
> >> > I have changed hbase-site.xml as below, and it now works (in Local
> >> > mode). Its something about Hadoop maybe?
> >> >
> >> > <configuration>
> >> > <property>
> >> > <name>hbase.master</name>
> >> > <value>localhost:60000</value>
> >> > <description>The directory shared by region servers.
> >> > </description>
> >> > </property>
> >> > <property>
> >> > <name>hbase.regionserver</name>
> >> > <value>localhost:60020</value>
> >> > </property>
> >> > </configuration>
> >> >
> >> >
> >> >> Date: Sun, 2 Aug 2009 19:25:16 -0700
> >> >> Subject: Re: Connection failure to HBase
> >> >> From: [email protected]
> >> >> To: [email protected]
> >> >>
> >> >> You can set hbase.master property on the configuration object:
> >> >>
> >> >> config.set("hbase.master", "localhost:9000");
> >> >>
> >> >> Regards,
> >> >> Vaibhav
> >> >>
> >> >> 2009/8/2 Onur AKTAS <[email protected]>
> >> >>
> >> >> >
> >> >> > Hi,
> >> >> >
> >> >> > I have just installed Hadoop 19.3 (pseudo distributed mode) and Hbase
> >> >> > 19.2
> >> >> > by following the instructions.
> >> >> > Both of them starts fine.
> >> >> >
> >> >> > Hadoop Log:
> >> >> > $ bin/start-all.sh
> >> >> > starting namenode, logging to
> >> >> > /hda3/ps/hadoop-0.19.2/bin/../logs/hadoop-oracle-namenode-localhost.localdomain.out
> >> >> > localhost: starting datanode, logging to
> >> >> > /hda3/ps/hadoop-0.19.2/bin/../logs/hadoop-oracle-datanode-localhost.localdomain.out
> >> >> > ....
> >> >> >
> >> >> > HBase Log:
> >> >> > $ bin/start-hbase.sh
> >> >> > starting master, logging to
> >> >> > /hda3/ps/hbase-0.19.3/bin/../logs/hbase-oracle-master-localhost.localdomain.out
> >> >> > localhost:
> >> >> > starting regionserver, logging to
> >> >> >
> >> >> > /hda3/ps/hbase-0.19.3/bin/../logs/hbase-oracle-regionserver-localhost.localdomain.out
> >> >> >
> >> >> > When I try to connect HBase from a client, it gives an error as:
> >> >> >
> >> >> > Aug 3, 2009 3:35:04 AM
> >> >> > org.apache.hadoop.hbase.ipc.HBaseClient$Connection
> >> >> > handleConnectionFailure
> >> >> > INFO: Retrying connect to server: localhost/127.0.0.1:60000. Already
> >> >> > tried
> >> >> > 0 time
> >> >> > (s).
> >> >> > Aug 3, 2009 3:35:05 AM
> >> >> > org.apache.hadoop.hbase.ipc.HBaseClient$Connection
> >> >> > handleConnectionFailure
> >> >> > INFO: Retrying connect to server: localhost/127.0.0.1:60000. Already
> >> >> > tried
> >> >> > 1 time(s).
> >> >> >
> >> >> > I
> >> >> > have configured sites.xml etc as "localhost:9000", How can I change
> >> >> > that 60000 port in client? I use like below in my Java class.
> >> >> > HBaseConfiguration config = new HBaseConfiguration();
> >> >> >
> >> >> > Thanks.
> >> >> >
> >> >> > _________________________________________________________________
> >> >> > Teker teker mi, yoksa hepsi birden mi? Arkadaşlarınızla ilgili güncel
> >> >> > bilgileri tek bir yerden edinin.
> >> >> >
> >> >> > http://www.microsoft.com/turkiye/windows/windowslive/products/social-network-connector.aspx
> >> >
> >> > _________________________________________________________________
> >> > Windows Live ile fotoğraflarınızı organize edebilir, düzenleyebilir ve
> >> > paylaşabilirsiniz.
> >> > http://www.microsoft.com/turkiye/windows/windowslive/products/photo-gallery-edit.aspx
> >
> > _________________________________________________________________
> > Windows Live tüm arkadaşlarınızla tek bir yerden iletişim kurmanıza
> > yardımcı olur.
> > http://www.microsoft.com/turkiye/windows/windowslive/products/social-network-connector.aspx
_________________________________________________________________
Windows Live ile fotoğraflarınızı organize edebilir, düzenleyebilir ve
paylaşabilirsiniz.
http://www.microsoft.com/turkiye/windows/windowslive/products/photo-gallery-edit.aspx
