St.Ack,
It's weird. I did Hbase upgrade from Hbase 0.20.3 to Hbase 0.90.2 by following
"replace the Hadoop JAR file from Hadoop 0.20.2 release with the
sync-supporting Hadoop JAR file found in HBase 0.90.2". It was suceesfully
before, but now we are facing error from Hbase shell: NativeException:
org.apache.hadoop.hbase.MasterNotRunningException:
hofvmecln15.ecom.sears.com:60000, even though the processes of Hmaster and
Region are running, and 60010 can be accessible via 60010.
Could you please help to find out the possible root cause? Below is the logging
messages from the hbase.log from Master Node:
2011-07-14 22:54:55,903 INFO zookeeper.ZooKeeper - Client
environment:user.dir=/appl/hbase/share/hbase-0.90.2
2011-07-14 22:54:55,904 INFO zookeeper.ZooKeeper - Initiating client
connection, connectString=hofvmecln19:2181,hofvmecln18:2181,hofvmecln17:2181
sessionTimeout=60000
watcher=org.apache.hadoop.hbase.regionserver.HRegionServer@51b4a74b
2011-07-14 22:54:55,905 INFO zookeeper.ClientCnxn -
zookeeper.disableAutoWatchReset is false
2011-07-14 22:54:55,932 INFO zookeeper.ClientCnxn - Attempting connection to
server hofvmecln18/151.149.116.46:2181
2011-07-14 22:54:55,938 INFO zookeeper.ClientCnxn - Priming connection to
java.nio.channels.SocketChannel[connected local=/151.149.119.93:52156
remote=hofvmecln18/151.149.116.46:2181]
2011-07-14 22:54:55,939 INFO zookeeper.ClientCnxn - Server connection
successful
2011-07-14 22:54:56,030 INFO regionserver.HRegionServer - Got ZooKeeper event,
state: SyncConnected, type: None, path: null
2011-07-14 22:54:56,202 INFO regionserver.HRegionServer - Telling master at
hofvmecln15.ecom.sears.com:60000 that we are up
2011-07-14 22:54:56,407 WARN ipc.HBaseServer - IPC Server listener on 60000:
readAndProcess threw exception java.io.EOFException. Count of bytes read: 0
java.io.EOFException
at java.io.DataInputStream.readFully(DataInputStream.java:180)
at java.io.DataInputStream.readUTF(DataInputStream.java:592)
at java.io.DataInputStream.readUTF(DataInputStream.java:547)
at
org.apache.hadoop.hbase.ipc.HBaseRPC$Invocation.readFields(HBaseRPC.java:122)
at
org.apache.hadoop.hbase.ipc.HBaseServer$Connection.processData(HBaseServer.java:978)
at
org.apache.hadoop.hbase.ipc.HBaseServer$Connection.readAndProcess(HBaseServer.java:946)
at
org.apache.hadoop.hbase.ipc.HBaseServer$Listener.doRead(HBaseServer.java:522)
at
org.apache.hadoop.hbase.ipc.HBaseServer$Listener$Reader.run(HBaseServer.java:316)
at
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:619)
2011-07-14 22:54:56,412 WARN regionserver.HRegionServer - Unable to connect to
master. Retrying. Error was:
java.io.IOException: Call to hofvmecln15.ecom.sears.com/151.149.119.93:60000
failed on local exception: java.io.EOFException
at
org.apache.hadoop.hbase.ipc.HBaseClient.wrapException(HBaseClient.java:757)
at org.apache.hadoop.hbase.ipc.HBaseClient.call(HBaseClient.java:727)
at
org.apache.hadoop.hbase.ipc.HBaseRPC$Invoker.invoke(HBaseRPC.java:333)
at $Proxy0.getProtocolVersion(Unknown Source)
at org.apache.hadoop.hbase.ipc.HBaseRPC.getProxy(HBaseRPC.java:489)
at org.apache.hadoop.hbase.ipc.HBaseRPC.getProxy(HBaseRPC.java:465)
at org.apache.hadoop.hbase.ipc.HBaseRPC.getProxy(HBaseRPC.java:512)
at org.apache.hadoop.hbase.ipc.HBaseRPC.waitForProxy(HBaseRPC.java:423)
at
org.apache.hadoop.hbase.regionserver.HRegionServer.getMaster(HRegionServer.java:1296)
at
org.apache.hadoop.hbase.regionserver.HRegionServer.reportForDuty(HRegionServer.java:1314)
at
org.apache.hadoop.hbase.regionserver.HRegionServer.run(HRegionServer.java:430)
at java.lang.Thread.run(Thread.java:619)
Caused by: java.io.EOFException
at java.io.DataInputStream.readInt(DataInputStream.java:375)
at
org.apache.hadoop.hbase.ipc.HBaseClient$Connection.receiveResponse(HBaseClient.java:504)
at
org.apache.hadoop.hbase.ipc.HBaseClient$Connection.run(HBaseClient.java:448)
thanks,
Andy Zhong
-----Original Message-----
From: [email protected] [mailto:[email protected]] On Behalf Of Stack
Sent: Wednesday, June 08, 2011 11:53 AM
To: [email protected]
Subject: Re: Hadoop/HBase Upgrade Suggestion
On Wed, Jun 8, 2011 at 9:25 AM, Zhong, Sheng <[email protected]> wrote:
> Could anyone give me suggestion for Hadoop/HBase upgrade? We're
> currently using apache hadoop 0.20.2 + hbase 0.20.3 + zookeeper-3.2.2.
> Has anyone done with latest stable version of hadoop-0.20.203.0rc1 +
> Hbase 0.90.2, and will Hbase 0.90.2 have compatible issue with
> hadoop-0.20.203.0rc1? I am very appreciated by any help and suggestion.
>
hadoop-0.20.203 does not have append support so its not recommended that you
run hbase on it. Here is our (a little stale at this point
-- see recent thread) story regards hadoop versions
http://hbase.apache.org/book/notsoquick.html#hadoop
St.Ack
This message, including any attachments, is the property of Sears Holdings
Corporation and/or one of its subsidiaries. It is confidential and may contain
proprietary or legally privileged information. If you are not the intended
recipient, please delete it without reading the contents. Thank you.
