Hi All,
I did some more fact finding around this issue, and it seems to me that the
Master is not getting initialized and hence I am not able to create table. I
tried connecting via the Java client as well, which gave me the following
exception. I am not sure what is causing Master to not initialize at all. I am
not able to figure out, where am I going wrong, and would request help. Thanks.
java.lang.RuntimeException: java.io.IOException: HRegionInfo was null or empty
in -ROOT-, row=keyvalues={.META.,,1/info:server/1350496120102/Put/vlen=15/ts=0,
.META.,,1/info:serverstartcode/1350496120102/Put/vlen=8/ts=0}
at
com.benchmark.uds.hbase.NativeJavaHBaseLoader.createSchema(NativeJavaHBaseLoader.java:56)
at
com.benchmark.uds.hbase.RequestGenerator.main(RequestGenerator.java:72)
Caused by: java.io.IOException: HRegionInfo was null or empty in -ROOT-,
row=keyvalues={.META.,,1/info:server/1350496120102/Put/vlen=15/ts=0,
.META.,,1/info:serverstartcode/1350496120102/Put/vlen=8/ts=0}
at
org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegionInMeta(HConnectionManager.java:985)
at
org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:841)
at
org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:810)
at org.apache.hadoop.hbase.client.HTable.finishSetup(HTable.java:232)
at org.apache.hadoop.hbase.client.HTable.<init>(HTable.java:172)
at
org.apache.hadoop.hbase.catalog.MetaReader.getHTable(MetaReader.java:200)
at
org.apache.hadoop.hbase.catalog.MetaReader.getMetaHTable(MetaReader.java:226)
at
org.apache.hadoop.hbase.catalog.MetaReader.fullScan(MetaReader.java:705)
at
org.apache.hadoop.hbase.catalog.MetaReader.fullScan(MetaReader.java:183)
at
org.apache.hadoop.hbase.catalog.MetaReader.tableExists(MetaReader.java:448)
at
org.apache.hadoop.hbase.client.HBaseAdmin.tableExists(HBaseAdmin.java:233)
at
org.apache.hadoop.hbase.client.HBaseAdmin.tableExists(HBaseAdmin.java:247)
at
com.benchmark.uds.hbase.NativeJavaHBaseLoader.createSchema(NativeJavaHBaseLoader.java:44)
Also, the error as seen in zookeeper log for the above session is
2012-10-17 23:32:04,274 INFO org.apache.zookeeper.server.NIOServerCnxn: Closed
socket connection for client /127.0.0.1:51602 which had sessionid
0x13a6fd74b50000c
2012-10-17 23:32:05,105 WARN org.apache.zookeeper.server.NIOServerCnxn: caught
end of stream exception
EndOfStreamException: Unable to read additional data from client sessionid
0x13a6fd74b500009, likely client has closed socket
at
org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:220)
at
org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:224)
at java.lang.Thread.run(Thread.java:662)
2012-10-17 23:32:05,105 INFO org.apache.zookeeper.server.NIOServerCnxn: Closed
socket connection for client /127.0.0.1:51515 which had sessionid
0x13a6fd74b500009
Thanks and Regards
Pankaj Misra
________________________________________
From: Pankaj Misra
Sent: Wednesday, October 17, 2012 10:30 PM
To: [email protected]
Subject: RE: HBase Issue after Namenode Formating
Hi All,
I have been using HBase 0.94.1 with Hadoop 0.23.1. I had the complete setup in
a pseudo-distributed mode, and was running fine.
Today I had to reset and reformat the namenode and clear all the data as well
to have a fresh run, but since then Hbase is not working as expected. For
instance I am able to list the tables but cannot create new one, it reports
that Master is initializing, as seen from the exception below
hbase(main):004:0> create 'testrun','samplecf'
ERROR: org.apache.hadoop.hbase.PleaseHoldException:
org.apache.hadoop.hbase.PleaseHoldException: Master is initializing
at
org.apache.hadoop.hbase.master.HMaster.checkInitialized(HMaster.java:1626)
at org.apache.hadoop.hbase.master.HMaster.createTable(HMaster.java:1154)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at
org.apache.hadoop.hbase.ipc.WritableRpcEngine$Server.call(WritableRpcEngine.java:364)
at
org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:1400)
In order to resolve this I have even tried to delete the root so that hbase can
recreate it fresh. HBase is able to recreate the ROOT but this issue persists.
I am also not able to connect via native java client, as it does not get
reference of the Master.
Additionally I see the following error in the HBase Master logs
2012-10-17 21:15:33,043 ERROR
org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper: Node /hbase/master
already exists and this is not a retry
2012-10-17 21:15:33,045 INFO
org.apache.hadoop.hbase.master.ActiveMasterManager: Adding ZNode for
/hbase/backup-masters/localhost,60000,1350488732582 in backup master directory
2012-10-17 21:15:33,083 INFO
org.apache.hadoop.hbase.master.ActiveMasterManager: Current master has this
master's address, localhost,60000,1350487401859; master was restarted? Deleting
node.
2012-10-17 21:15:33,084 DEBUG
org.apache.hadoop.hbase.master.ActiveMasterManager: No master available.
Notifying waiting threads
2012-10-17 21:15:33,084 INFO org.apache.hadoop.hbase.master.HMaster: Cluster
went down before this master became active
2012-10-17 21:15:33,084 DEBUG org.apache.hadoop.hbase.master.HMaster: Stopping
service threads
Calling for help, as I think I am missing something here.
Thanks and Regards
Pankaj Misra
________________________________
Impetus Ranked in the Top 50 India's Best Companies to Work For 2012.
Impetus webcast 'Designing a Test Automation Framework for Multi-vendor
Interoperable Systems' available at http://lf1.me/0E/.
NOTE: This message may contain information that is confidential, proprietary,
privileged or otherwise protected by law. The message is intended solely for
the named addressee. If received in error, please destroy and notify the
sender. Any use of this email is prohibited when received in error. Impetus
does not represent, warrant and/or guarantee, that the integrity of this
communication has been maintained nor that the communication is free of errors,
virus, interception or interference.
________________________________
Impetus Ranked in the Top 50 India’s Best Companies to Work For 2012.
Impetus webcast ‘Designing a Test Automation Framework for Multi-vendor
Interoperable Systems’ available at http://lf1.me/0E/.
NOTE: This message may contain information that is confidential, proprietary,
privileged or otherwise protected by law. The message is intended solely for
the named addressee. If received in error, please destroy and notify the
sender. Any use of this email is prohibited when received in error. Impetus
does not represent, warrant and/or guarantee, that the integrity of this
communication has been maintained nor that the communication is free of errors,
virus, interception or interference.
