Thanks everyone for the input, it's helpful. I did remove the znode from /hbase/table/SESSIONID_TIMELINE and after that I was able to list the table. At that point I tried to do a "put" but when I did a put I got a message NoRegionServer online. I looked in the logs and it says the Failed to open region server at nodexxx. When I went to nodexxx it complains something about unable to run testcompression.
I setup SNAPPY compression on my table and I also ran SNAPPY compression test which was successful. Not sure what's going on in the cluster. On Thu, Sep 27, 2012 at 1:10 AM, Mohammad Tariq <[email protected]> wrote: > Hello Mohit, > > It should be /hbase/hbase/table/SESSIONID_TIMELINE..Apologies for the > typo. For rest of the things, I feel Ramkrishna sir has provided a good and > proper explanation. Please let us know if you still have any doubt or > question. > > Ramkrishna.S.Vasudevan : You are welcome sir. It's my pleasure to share > space with you people. > > Regards, > Mohammad Tariq > > > > On Thu, Sep 27, 2012 at 9:59 AM, Ramkrishna.S.Vasudevan < > [email protected]> wrote: > > > Hi Mohith > > First of all thanks to Tariq for his replies. > > > > Just to add on, > > Basically HBase uses the Zookeeper to know the status of the cluster > like > > the no of tables enabled, disabled and deleted. > > Enabled and deleted states are handled bit different in the 0.94 version. > > > > ZK is used for various region assignments. > > > > Also the ZK is used to track the Active master and standby master. > > > > As you understand correctly that the master is responsible for the > overall > > maintenance of the no of tables and their respective states, it seeks the > > help of ZK to do it and that is where the states are persisted. > > > > Also there are few cases where the enable and disable table are having > some > > issues due to some race conditions in the 0.92 versions, In the latest > > version we are trying to resolve them. > > You can attach the master and RS logs to identify exactly what caused > this > > problem in your case which will be really help ful so that I can be fixed > > in > > the kernel. > > > > Regards > > Ram > > > > > -----Original Message----- > > > From: Mohit Anchlia [mailto:[email protected]] > > > Sent: Thursday, September 27, 2012 5:09 AM > > > To: [email protected] > > > Subject: Re: disable table > > > > > > I did /hbase/table/SESSIONID_TIMELINE and that seem to work. I'll > > > restart > > > hbase and see if it works. > > > > > > One thing I don't understand is why is zookeeper holding information > > > about > > > this table if it is enabled or disabled? Wouldn't this information be > > > with > > > master? > > > > > > On Wed, Sep 26, 2012 at 4:27 PM, Mohit Anchlia > > > <[email protected]>wrote: > > > > > > > I don't see path like /hbase/SESSIONID_TIMELINE > > > > This is what I see > > > > > > > > [zk: pprfdaaha303:5181(CONNECTED) 5] ls /hbase/table > > > > [SESSIONID_TIMELINE] > > > > [zk: pprfdaaha303:5181(CONNECTED) 6] get /hbase/table > > > > > > > > cZxid = 0x1000000fe > > > > ctime = Mon Sep 10 15:31:45 PDT 2012 > > > > mZxid = 0x1000000fe > > > > mtime = Mon Sep 10 15:31:45 PDT 2012 > > > > pZxid = 0x5000008f1 > > > > cversion = 3 > > > > dataVersion = 0 > > > > aclVersion = 0 > > > > ephemeralOwner = 0x0 > > > > dataLength = 0 > > > > numChildren = 1 > > > > > > > > On Wed, Sep 26, 2012 at 3:57 PM, Mohammad Tariq > > > <[email protected]>wrote: > > > > > > > >> In order to delete a znode you have to go to the ZK shell and issue > > > the > > > >> "delete" command along with the required path. For example : > > > >> delete /hbase/SESSIONID_TIMELINE. For detailed info you can visit > > > the ZK > > > >> homepage at : zookeeper.apache.org > > > >> > > > >> Actually when we try to fetch data from an Hbase table, the client > > > or app > > > >> first contacts the ZK to get the location of server holding the > > > >> -ROOT- table. From this we come to know about the server hosting the > > > >> .META. > > > >> table. This tells us the location of the server which actually holds > > > the > > > >> rows of interest. Because of some reasons the znode which was > > > holding this > > > >> info has either faced some catastrophe or lost the info associated > > > with > > > >> this particular table. Or sometimes the znode remains unable to keep > > > >> itself > > > >> updated with the latest changes. That could also be a probable > > > reason. We > > > >> should always keep in mind that ZK is the centralized service that > > > >> actually > > > >> coordinating everything behind the scene. As a result, any problem > > > to the > > > >> ZK quorum means problem with Hbase custer. > > > >> > > > >> Regards, > > > >> Mohammad Tariq > > > >> > > > >> > > > >> > > > >> On Thu, Sep 27, 2012 at 3:39 AM, Mohit Anchlia > > > <[email protected] > > > >> >wrote: > > > >> > > > >> > Thanks! I do see Inconsistency. How do I remove the znode. And > > > also > > > >> could > > > >> > you please help me understand how this might have happened? > > > >> > > > > >> > > > > >> > ERROR: Region > > > >> > > > > SESSIONID_TIMELINE,,1348689726526.0e200aace5e81cead8d8714ed8076050. not > > > >> > deployed on any region server. > > > >> > > > > >> > > > > >> > On Wed, Sep 26, 2012 at 2:36 PM, Mohammad Tariq > > > <[email protected]> > > > >> > wrote: > > > >> > > > > >> > > A possible reason could be that the znode associated with this > > > >> particular > > > >> > > table is not behaving properly. In such case, you can try the > > > >> following: > > > >> > > > > > >> > > Stop Hbase > > > >> > > Stop ZK > > > >> > > Take a backup of ZK data > > > >> > > Restart ZK > > > >> > > Remove the znode > > > >> > > Start Hbase again > > > >> > > > > > >> > > After this hopefully your table would be enabled. > > > >> > > > > > >> > > Regards, > > > >> > > Mohammad Tariq > > > >> > > > > > >> > > > > > >> > > > > > >> > > On Thu, Sep 27, 2012 at 2:59 AM, Mohammad Tariq > > > <[email protected]> > > > >> > > wrote: > > > >> > > > > > >> > > > Yes. Also have a look at the logs of the problematic region if > > > hbck > > > >> > shows > > > >> > > > any inconsistency. > > > >> > > > > > > >> > > > Regards, > > > >> > > > Mohammad Tariq > > > >> > > > > > > >> > > > > > > >> > > > > > > >> > > > On Thu, Sep 27, 2012 at 2:55 AM, Mohit Anchlia < > > > >> [email protected] > > > >> > > >wrote: > > > >> > > > > > > >> > > >> Which node should I look at for logs? Is this the master > > > node? I'll > > > >> > try > > > >> > > >> hbck. > > > >> > > >> > > > >> > > >> On Wed, Sep 26, 2012 at 2:19 PM, Mohammad Tariq < > > > >> [email protected]> > > > >> > > >> wrote: > > > >> > > >> > > > >> > > >> > Hello Mohit, > > > >> > > >> > > > > >> > > >> > Try hbck once and see if it shows any inconsistency. > > > Also, > > > >> you > > > >> > can > > > >> > > >> try > > > >> > > >> > restarting your cluster and deleting the table again. > > > Having a > > > >> look > > > >> > at > > > >> > > >> the > > > >> > > >> > logs could also be useful. > > > >> > > >> > > > > >> > > >> > Regards, > > > >> > > >> > Mohammad Tariq > > > >> > > >> > > > > >> > > >> > > > > >> > > >> > > > > >> > > >> > On Thu, Sep 27, 2012 at 2:24 AM, Mohit Anchlia < > > > >> > > [email protected] > > > >> > > >> > >wrote: > > > >> > > >> > > > > >> > > >> > > When I try to disable table I get: > > > >> > > >> > > > > > >> > > >> > > hbase(main):011:0> disable 'SESSIONID_TIMELINE' > > > >> > > >> > > ERROR: org.apache.hadoop.hbase.TableNotEnabledException: > > > >> > > >> > > org.apache.hadoop.hbase.TableNotEnabledException: > > > >> > SESSIONID_TIMELINE > > > >> > > >> > > Here is some help for this command: > > > >> > > >> > > Start disable of named table: e.g. "hbase> disable 't1'" > > > >> > > >> > > > > > >> > > >> > > But then I try to enable I get: > > > >> > > >> > > > > > >> > > >> > > hbase(main):012:0> enable 'SESSIONID_TIMELINE' > > > >> > > >> > > ERROR: org.apache.hadoop.hbase.TableNotDisabledException: > > > >> > > >> > > org.apache.hadoop.hbase.TableNotDisabledException: > > > >> > > SESSIONID_TIMELINE > > > >> > > >> > > Here is some help for this command: > > > >> > > >> > > Start enable of named table: e.g. "hbase> enable 't1'" > > > >> > > >> > > > > > >> > > >> > > I've tried flush, major_compaction also. I tseems it's > > > stuck in > > > >> > > >> > > inconsistent state. Could someone point me to correct > > > >> direction? I > > > >> > > am > > > >> > > >> > using > > > >> > > >> > > 92.1 > > > >> > > >> > > > > > >> > > >> > > > > >> > > >> > > > >> > > > > > > >> > > > > > > >> > > > > > >> > > > > >> > > > > > > > > > > > > >
