Re: Hadoop/HBase setup

Thu, 01 Nov 2007 12:45:38 -0800

Hey Jonathan. From the below, the regionserver looks to have reported into the master fine and even gotten instruction that it should deploy the -ROOT- region but then when master tried to talk back later, it couldn't. I have seen this previous when hosts were confused on how to reach each other -- is there a bogus entry in an /etc/hosts?
But it looks like you are trying the hbase from the hadoop 0.14.x branch. IMO, you'll get further if you try the first hbase release bundled with the hadoop 0.15.0 candidate -- http://people.apache.org/~cutting/hadoop-0.15.0-candidate-1/ -- or go get a recent nightly build from here: http://lucene.zones.apache.org:8080/hudson/job/Hadoop-Nightly/. 

St.Ack


jonathan doklovic wrote:

Hi,

I'm trying to evaluate hadoop/hbase for a project I'm on that requires
filtering massive amounts of RSS data.

I've been trying to follow the simple tutorials, but I can't seem to get
anything to work.

So far, I've formatted hadoop storage, that went fine.
Then I start hadoop:


[EMAIL PROTECTED]:~/apps/hadoop-0.14.3/bin> ./start-dfs.sh 
starting namenode, logging

to 
/home/jdoklovic/apps/hadoop-0.14.3/bin/../logs/hadoop-jdoklovic-namenode-msp-na214982.out
localhost: starting datanode, logging
to 
/home/jdoklovic/apps/hadoop-0.14.3/bin/../logs/hadoop-jdoklovic-datanode-msp-na214982.out
localhost: starting secondarynamenode, logging
to 
/home/jdoklovic/apps/hadoop-0.14.3/bin/../logs/hadoop-jdoklovic-secondarynamenode-msp-na214982.out

next I start hbase:


[EMAIL PROTECTED]:~/apps/hadoop-0.14.3/src/contrib/hbase/bin> ./start-hbase.sh 
starting master, logging

to 
/home/jdoklovic/apps/hadoop-0.14.3/src/contrib/hbase/bin/../../../..//logs/hbase-jdoklovic-master-msp-na214982.out
localhost: starting regionserver, logging
to 
/home/jdoklovic/apps/hadoop-0.14.3/src/contrib/hbase/bin/../../../..//logs/hbase-jdoklovic-regionserver-msp-na214982.out

right after this, I tail the master and region server logs.
In the region server log, everything seems fine....

[EMAIL PROTECTED]:~/apps/hadoop-0.14.3/logs> tail

-f ./hbase-jdoklovic-regionserver-msp-na214982.log 
2007-11-01 14:48:34,613 INFO org.apache.hadoop.ipc.Server: IPC Server

handler 3 on 60010: starting
2007-11-01 14:48:34,613 INFO org.apache.hadoop.ipc.Server: IPC Server
handler 4 on 60010: starting
2007-11-01 14:48:34,613 INFO org.apache.hadoop.ipc.Server: IPC Server
handler 5 on 60010: starting
2007-11-01 14:48:34,613 INFO org.apache.hadoop.ipc.Server: IPC Server
handler 6 on 60010: starting
2007-11-01 14:48:34,613 INFO org.apache.hadoop.ipc.Server: IPC Server
handler 7 on 60010: starting
2007-11-01 14:48:34,617 INFO org.apache.hadoop.ipc.Server: IPC Server
handler 8 on 60010: starting
2007-11-01 14:48:34,617 INFO org.apache.hadoop.ipc.Server: IPC Server
handler 9 on 60010: starting
2007-11-01 14:48:34,617 INFO org.apache.hadoop.hbase.HRegionServer:
HRegionServer started at: 10.0.4.29:60010
2007-11-01 14:48:34,808 INFO org.apache.hadoop.hbase.HStore: HStore
online for --ROOT--,,0/info
2007-11-01 14:48:34,811 INFO org.apache.hadoop.hbase.HRegion: region
--ROOT--,,0 available

However, the master server goes crazy and can't seem to connect to the
region server:

[EMAIL PROTECTED]:~/apps/hadoop-0.14.3/logs> tail -f

hbase-jdoklovic-master-msp-na214982.log 
2007-11-01 14:48:33,246 INFO org.apache.hadoop.ipc.Server: IPC Server

handler 1 on 60000: starting
2007-11-01 14:48:33,246 INFO org.apache.hadoop.ipc.Server: IPC Server
handler 2 on 60000: starting
2007-11-01 14:48:33,246 INFO org.apache.hadoop.ipc.Server: IPC Server
handler 3 on 60000: starting
2007-11-01 14:48:33,247 INFO org.apache.hadoop.ipc.Server: IPC Server
handler 4 on 60000: starting
2007-11-01 14:48:33,247 INFO org.apache.hadoop.ipc.Server: IPC Server
handler 5 on 60000: starting
2007-11-01 14:48:33,247 INFO org.apache.hadoop.ipc.Server: IPC Server
handler 6 on 60000: starting
2007-11-01 14:48:33,247 INFO org.apache.hadoop.ipc.Server: IPC Server
handler 7 on 60000: starting
2007-11-01 14:48:33,247 INFO org.apache.hadoop.ipc.Server: IPC Server
handler 8 on 60000: starting
2007-11-01 14:48:33,247 INFO org.apache.hadoop.ipc.Server: IPC Server
handler 9 on 60000: starting
2007-11-01 14:48:34,618 INFO org.apache.hadoop.hbase.HMaster: received
start message from: 10.0.4.29:60010
2007-11-01 14:48:49,667 INFO org.apache.hadoop.ipc.Client: Retrying
connect to server: /10.0.4.29:60010. Already tried 1 time(s).
2007-11-01 14:48:50,669 INFO org.apache.hadoop.ipc.Client: Retrying
connect to server: /10.0.4.29:60010. Already tried 2 time(s).
2007-11-01 14:48:51,673 INFO org.apache.hadoop.ipc.Client: Retrying
connect to server: /10.0.4.29:60010. Already tried 3 time(s).
2007-11-01 14:48:52,678 INFO org.apache.hadoop.ipc.Client: Retrying
connect to server: /10.0.4.29:60010. Already tried 4 time(s).
2007-11-01 14:48:53,680 INFO org.apache.hadoop.ipc.Client: Retrying
connect to server: /10.0.4.29:60010. Already tried 5 time(s).
2007-11-01 14:48:54,681 INFO org.apache.hadoop.ipc.Client: Retrying
connect to server: /10.0.4.29:60010. Already tried 6 time(s).
2007-11-01 14:48:55,686 INFO org.apache.hadoop.ipc.Client: Retrying
connect to server: /10.0.4.29:60010. Already tried 7 time(s).
2007-11-01 14:48:56,690 INFO org.apache.hadoop.ipc.Client: Retrying
connect to server: /10.0.4.29:60010. Already tried 8 time(s).
2007-11-01 14:48:57,692 INFO org.apache.hadoop.ipc.Client: Retrying
connect to server: /10.0.4.29:60010. Already tried 9 time(s).
2007-11-01 14:48:58,696 INFO org.apache.hadoop.ipc.Client: Retrying
connect to server: /10.0.4.29:60010. Already tried 10 time(s).
2007-11-01 14:48:59,698 INFO org.apache.hadoop.ipc.RPC: Server
at /10.0.4.29:60010 not available yet, Zzzzz...

this repeats forever.

Can anyone help me get this running properly?

My config files are really simple:

hadoop-site.xml
---------------
<configuration>
<property>
     <name>hadoop.tmp.dir</name>
     <value>/tmp/hadoop</value>
</property>
<property>
     <name>fs.default.name</name>
     <value>localhost:8585</value>
</property>
<property>
     <name>mapred.job.tracker</name>
     <value>localhost:8686</value>
</property>
<property>
  <name>mapred.child.java.opts</name>
  <value>-Xmx512m</value>
</property>
</configuration>

hbase-site.xml
---------------
<configuration>
<property>
     <name>hbase.master</name>
     <value>localhost:60000</value>
   </property>
<property>
     <name>hbase.regionserver</name>
     <value>localhost:60010</value>
   </property>

</configuration>


Thanks,

- Jonathan



  
























					Reply via email to