Hi Mohith First of all thanks to Tariq for his replies. Just to add on, Basically HBase uses the Zookeeper to know the status of the cluster like the no of tables enabled, disabled and deleted. Enabled and deleted states are handled bit different in the 0.94 version.
ZK is used for various region assignments. Also the ZK is used to track the Active master and standby master. As you understand correctly that the master is responsible for the overall maintenance of the no of tables and their respective states, it seeks the help of ZK to do it and that is where the states are persisted. Also there are few cases where the enable and disable table are having some issues due to some race conditions in the 0.92 versions, In the latest version we are trying to resolve them. You can attach the master and RS logs to identify exactly what caused this problem in your case which will be really help ful so that I can be fixed in the kernel. Regards Ram > -----Original Message----- > From: Mohit Anchlia [mailto:[email protected]] > Sent: Thursday, September 27, 2012 5:09 AM > To: [email protected] > Subject: Re: disable table > > I did /hbase/table/SESSIONID_TIMELINE and that seem to work. I'll > restart > hbase and see if it works. > > One thing I don't understand is why is zookeeper holding information > about > this table if it is enabled or disabled? Wouldn't this information be > with > master? > > On Wed, Sep 26, 2012 at 4:27 PM, Mohit Anchlia > <[email protected]>wrote: > > > I don't see path like /hbase/SESSIONID_TIMELINE > > This is what I see > > > > [zk: pprfdaaha303:5181(CONNECTED) 5] ls /hbase/table > > [SESSIONID_TIMELINE] > > [zk: pprfdaaha303:5181(CONNECTED) 6] get /hbase/table > > > > cZxid = 0x1000000fe > > ctime = Mon Sep 10 15:31:45 PDT 2012 > > mZxid = 0x1000000fe > > mtime = Mon Sep 10 15:31:45 PDT 2012 > > pZxid = 0x5000008f1 > > cversion = 3 > > dataVersion = 0 > > aclVersion = 0 > > ephemeralOwner = 0x0 > > dataLength = 0 > > numChildren = 1 > > > > On Wed, Sep 26, 2012 at 3:57 PM, Mohammad Tariq > <[email protected]>wrote: > > > >> In order to delete a znode you have to go to the ZK shell and issue > the > >> "delete" command along with the required path. For example : > >> delete /hbase/SESSIONID_TIMELINE. For detailed info you can visit > the ZK > >> homepage at : zookeeper.apache.org > >> > >> Actually when we try to fetch data from an Hbase table, the client > or app > >> first contacts the ZK to get the location of server holding the > >> -ROOT- table. From this we come to know about the server hosting the > >> .META. > >> table. This tells us the location of the server which actually holds > the > >> rows of interest. Because of some reasons the znode which was > holding this > >> info has either faced some catastrophe or lost the info associated > with > >> this particular table. Or sometimes the znode remains unable to keep > >> itself > >> updated with the latest changes. That could also be a probable > reason. We > >> should always keep in mind that ZK is the centralized service that > >> actually > >> coordinating everything behind the scene. As a result, any problem > to the > >> ZK quorum means problem with Hbase custer. > >> > >> Regards, > >> Mohammad Tariq > >> > >> > >> > >> On Thu, Sep 27, 2012 at 3:39 AM, Mohit Anchlia > <[email protected] > >> >wrote: > >> > >> > Thanks! I do see Inconsistency. How do I remove the znode. And > also > >> could > >> > you please help me understand how this might have happened? > >> > > >> > > >> > ERROR: Region > >> > > SESSIONID_TIMELINE,,1348689726526.0e200aace5e81cead8d8714ed8076050. not > >> > deployed on any region server. > >> > > >> > > >> > On Wed, Sep 26, 2012 at 2:36 PM, Mohammad Tariq > <[email protected]> > >> > wrote: > >> > > >> > > A possible reason could be that the znode associated with this > >> particular > >> > > table is not behaving properly. In such case, you can try the > >> following: > >> > > > >> > > Stop Hbase > >> > > Stop ZK > >> > > Take a backup of ZK data > >> > > Restart ZK > >> > > Remove the znode > >> > > Start Hbase again > >> > > > >> > > After this hopefully your table would be enabled. > >> > > > >> > > Regards, > >> > > Mohammad Tariq > >> > > > >> > > > >> > > > >> > > On Thu, Sep 27, 2012 at 2:59 AM, Mohammad Tariq > <[email protected]> > >> > > wrote: > >> > > > >> > > > Yes. Also have a look at the logs of the problematic region if > hbck > >> > shows > >> > > > any inconsistency. > >> > > > > >> > > > Regards, > >> > > > Mohammad Tariq > >> > > > > >> > > > > >> > > > > >> > > > On Thu, Sep 27, 2012 at 2:55 AM, Mohit Anchlia < > >> [email protected] > >> > > >wrote: > >> > > > > >> > > >> Which node should I look at for logs? Is this the master > node? I'll > >> > try > >> > > >> hbck. > >> > > >> > >> > > >> On Wed, Sep 26, 2012 at 2:19 PM, Mohammad Tariq < > >> [email protected]> > >> > > >> wrote: > >> > > >> > >> > > >> > Hello Mohit, > >> > > >> > > >> > > >> > Try hbck once and see if it shows any inconsistency. > Also, > >> you > >> > can > >> > > >> try > >> > > >> > restarting your cluster and deleting the table again. > Having a > >> look > >> > at > >> > > >> the > >> > > >> > logs could also be useful. > >> > > >> > > >> > > >> > Regards, > >> > > >> > Mohammad Tariq > >> > > >> > > >> > > >> > > >> > > >> > > >> > > >> > On Thu, Sep 27, 2012 at 2:24 AM, Mohit Anchlia < > >> > > [email protected] > >> > > >> > >wrote: > >> > > >> > > >> > > >> > > When I try to disable table I get: > >> > > >> > > > >> > > >> > > hbase(main):011:0> disable 'SESSIONID_TIMELINE' > >> > > >> > > ERROR: org.apache.hadoop.hbase.TableNotEnabledException: > >> > > >> > > org.apache.hadoop.hbase.TableNotEnabledException: > >> > SESSIONID_TIMELINE > >> > > >> > > Here is some help for this command: > >> > > >> > > Start disable of named table: e.g. "hbase> disable 't1'" > >> > > >> > > > >> > > >> > > But then I try to enable I get: > >> > > >> > > > >> > > >> > > hbase(main):012:0> enable 'SESSIONID_TIMELINE' > >> > > >> > > ERROR: org.apache.hadoop.hbase.TableNotDisabledException: > >> > > >> > > org.apache.hadoop.hbase.TableNotDisabledException: > >> > > SESSIONID_TIMELINE > >> > > >> > > Here is some help for this command: > >> > > >> > > Start enable of named table: e.g. "hbase> enable 't1'" > >> > > >> > > > >> > > >> > > I've tried flush, major_compaction also. I tseems it's > stuck in > >> > > >> > > inconsistent state. Could someone point me to correct > >> direction? I > >> > > am > >> > > >> > using > >> > > >> > > 92.1 > >> > > >> > > > >> > > >> > > >> > > >> > >> > > > > >> > > > > >> > > > >> > > >> > > > >
