Fetching HBase Region info for a particular table on a particular region server

Hi, I'm kinda new to HBase API, so pardon me if the question seems very basic. I wanted to know if I can fetch the region info of a particular table on a particular region server. Basically I want to fetch the region size of a table (say 't1') on a region server (say 'rs1'). Any help is

Re: Fetching HBase Region info for a particular table on a particular region server

I believe what you are looking for is here: http://stackoverflow.com/questions/14573466/get-hbase-region-size-via-api Thanks Divye Sheth On Thu, Mar 6, 2014 at 2:01 PM, drunkenfist sudharshan@gmail.comwrote: Hi, I'm kinda new to HBase API, so pardon me if the question seems very basic.

connecting to hbase remotely

Hi i am using HBase 0.94.6-cdh4.5.0 i conected to hbase by setting config explicitly in my code through config.set zookeeper quorum property. i am able to read the hbase table data properly,it is connecting to the host specified in config log: Creating new Groups object Group mapping

Bunch of errros/exceptions in 2 node hbase cluster setup

Hi, I am configuring 2 node hbase cluster atop 2 node Hadoop cluster for production. I am getting below warning/error/info/fail/exceptions in master,region,zookeeper logs. Please help in debugging these.What each means ? Master logs 1) WARN org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper:

Re: Bunch of errros/exceptions in 2 node hbase cluster setup

Comments below. I will skip most of the warnings as they are only warnings and due to your setup. Will also skip duplicate messages. From what I can see, there is nothing to be really worried about in what you sent. JMS 2014-03-06 6:10 GMT-05:00 Vimal Jain vkj...@gmail.com: Hi, I am

Re: connecting to hbase remotely

Can you share a sample of you code to see where it can come from? 2014-03-06 6:05 GMT-05:00 yeshwanth kumar yeshwant...@gmail.com: Hi i am using HBase 0.94.6-cdh4.5.0 i conected to hbase by setting config explicitly in my code through config.set zookeeper quorum property. i am able to

Region splits even when max Hstorefile is less then default value

Hi, I am running 2 node hbase cluster atop HDFS. I was simulating a heavy write process on Hbase for performance analysis. I found that my HstoreFile has not reached hbase.hregion.max.filesize property ( which is by default 10G ) and yet my region is split into 2 regions. *du -sh HadoopData/*

Re: Region splits even when max Hstorefile is less then default value

Hi Vimal, This is used only with ConstantSizeRegionSplitPolicy. Which split policy do you have configured in your setup? JM 2014-03-06 7:55 GMT-05:00 Vimal Jain vkj...@gmail.com: Hi, I am running 2 node hbase cluster atop HDFS. I was simulating a heavy write process on Hbase for

Re: Region splits even when max Hstorefile is less then default value

Hi Jean, I am not sure about this. Whats the default policy ? On Thu, Mar 6, 2014 at 6:30 PM, Jean-Marc Spaggiari jean-m...@spaggiari.org wrote: Hi Vimal, This is used only with ConstantSizeRegionSplitPolicy. Which split policy do you have configured in your setup? JM 2014-03-06 7:55

Re: Region splits even when max Hstorefile is less then default value

Hi Vimal, Which version of HBase do you use? Can you also share your config file? JM 2014-03-06 8:02 GMT-05:00 Vimal Jain vkj...@gmail.com: Hi Jean, I am not sure about this. Whats the default policy ? On Thu, Mar 6, 2014 at 6:30 PM, Jean-Marc Spaggiari jean-m...@spaggiari.org

Re: Region splits even when max Hstorefile is less then default value

Hi, I am using 0.94.17 and my config file is property namehbase.rootdir/name valuehdfs://10.14.24.19:9000/hbase/value /property property namehbase.cluster.distributed/name

Is FuzzyRowFilter also a PrefixFilter

Hi all, I have the following row structure: MMdd_Country_CategoryId1_CategoryId2_CategoryId3 Any field, except for the date may be empty (each day there is one row that looks like this: MMdd) I use FuzzyRowFilter like this: Filter filter = new FuzzyRowFilter(

Re: Region splits even when max Hstorefile is less then default value

Thanks Jean. On Thu, Mar 6, 2014 at 6:41 PM, Jean-Marc Spaggiari jean-m...@spaggiari.org wrote: Default split policy for 0.94.17 is IncreasingToUpperBoundRegionSplitPolicy. Since you did not set a specific policy in your config file, that's the one used. Comments from the class: * Split

Re: Fetching HBase Region info for a particular table on a particular region server

Thanks Divye, but wouldn't that fetch the storefilesize of all tables? I want the region size of a specific table, given the tablename and regionserver name. And are storefilesize and region size the same? Thanks. -- View this message in context:

Re: Fetching HBase Region info for a particular table on a particular region server

Hey, There is no dependency between region size and regionserver. Though regions map to regionserver, they actually lie on HDFS. RS is just a virtual entity that manages a bunch of regions. So the easiest way to get the size of a region is to fetch it from HDFS? - Bharath On Thu, Mar 6,

Re: Is FuzzyRowFilter also a PrefixFilter

Take a look at this method of FuzzyRowFilter: private static SatisfiesCode satisfies(byte[] row, int offset, int length, byte[] fuzzyKeyBytes, byte[] fuzzyKeyMeta) { The for loop is controlled by fuzzyKeyMeta.length Meaning it compares as many bytes

Re: Fetching HBase Region info for a particular table on a particular region server

So there is no way to get the region size of a table from a region server? Or is it possible to get the HDFS containing the region server name and then get the region size of a particular table from that HDFS? Thanks. -- View this message in context:

Re: Fetching HBase Region info for a particular table on a particular region server

Take a look at Hannibal... Will help you a lot. https://github.com/sentric/hannibal JM 2014-03-06 11:11 GMT-05:00 drunkenfist sudharshan@gmail.com: So there is no way to get the region size of a table from a region server? Or is it possible to get the HDFS containing the region server

Re: Is FuzzyRowFilter also a PrefixFilter

So, correct me if I'm wrong, fuzzy acts as prefix as well, right ? for key: MMdd with mask: both 20140201 and 2014020113 will pass the filter. Is there a way to use FuzzyRowFilter as exact fuzzy match so that prefix matches will not pass ? On Thu,

Re: Is FuzzyRowFilter also a PrefixFilter

If you're only interested in the pattern given below, you can use PrefixFilter directly. Otherwise enhancement to FuzzyRowFilter is needed. On Thu, Mar 6, 2014 at 8:45 AM, Amit Sela am...@infolinks.com wrote: So, correct me if I'm wrong, fuzzy acts as prefix as well, right ? for key:

Re: Fetching HBase Region info for a particular table on a particular region server

Take a look at HBASE-10413 Tablesplit.getLength returns 0 RegionSizeCalculator.java should give you what you need. On Thu, Mar 6, 2014 at 8:11 AM, drunkenfist sudharshan@gmail.comwrote: So there is no way to get the region size of a table from a region server? Or is it possible to get

Re: connecting to hbase remotely

Looks like you were serving requests using web server. Is hbase-site.xml in the classpath ? Thanks On Thu, Mar 6, 2014 at 3:05 AM, yeshwanth kumar yeshwant...@gmail.comwrote: Hi i am using HBase 0.94.6-cdh4.5.0 i conected to hbase by setting config explicitly in my code through config.set

turn off size-based Major Compaction?

hi, folks, Is there a way to turn off Major Compaction, for sure? We are using 96.0, and set hbase.hregion.majorcompaction is 0. The intention was to only run major compaction manually. It turns out after heavy load, major compaction still occur as shown in WebUI. After a bit search, I read

Re: turn off size-based Major Compaction?

The problem is not major compactions, but compactions that compact a lot of files. HBase will always compact in the background (minor compactions). If a minor compaction needs to compact all files anyway it is promoted to a major compaction. There is no performance penalty to that, just that

Re: turn off size-based Major Compaction?

Demai: You can find description of compaction-related parameters in: http://hbase.apache.org/book.html#compaction Cheers On Thu, Mar 6, 2014 at 10:44 AM, lars hofhansl la...@apache.org wrote: The problem is not major compactions, but compactions that compact a lot of files. HBase will

Distributed log splitting failing after cluster outage.

Hello, Our HBase cluster had an unexpected shut-down and while trying to bring it back up we the Master gets stuck with the following message: Failed splitting of [ list of host_name,port,tmst ] java.io.IOException: error or interrupted while splitting logs in [ list of host_name,port,tmst ]

Re: Distributed log splitting failing after cluster outage.

bq. error or interrupted while splitting logs in The list following the above message should reveal which log directories had problem. You can go to corresponding region server(s) to see what caused the issue. BTW which hbase release are you using ? Cheers On Thu, Mar 6, 2014 at 11:13 AM,

Re: Distributed log splitting failing after cluster outage.

We ran into this a few weeks ago when while adding new nodes into an existing cluster. Due to a misconfiguration, the new nodes were assigned a wrong zookeeper quorum, and ended up forming a new cluster. We saw a similar error in our logs: 2014-01-30 16:47:19,196 ERROR

[ANNOUNCE] Next releases of HBase will drop Hadoop-1.x support

Hi, In the dev thread [1], HBase developers are considering dropping support for Hadoop-1 in future releases 0.99, and HBase-1.0. This is a heads up, so that you can plan ahead if you are choosing to go with 0.96.x and 0.98.x releases. Hadoop-2.2 was released last October, and it is superior in

HBase looks for hadoop-common.jar in Hdfs

I am trying to run a hadoop(version:2.2.0) job on a single machine which writes the reducer output into a Hbase(version 0.98) table. When I run the job, HBase looks for hadoop-common.jar file in the HDFS: 2014-03-07 10:52:44,499 ERROR [main] security.UserGroupInformation

Re: HBase looks for hadoop-common.jar in Hdfs

With 2.2.0, default mr engine is yarn, but I believe your job is being submitted to local mrv1. Just add the following to the mapred-site.xml and see if that works, property namemapreduce.framework.name/name valueyarn/value /property On Fri, Mar 7, 2014 at 7:38

Need suggestion, jpa kind of package for hbase

Hi all, I'm new to hbase in search of jpa kind of package for hbase to push the data into hbase system. Started trying with stargate where i found very strict json notation, base 64 encoding and other issue finding it difficult to use. Then i looked at kundera then i realized we need to have

Re: Need suggestion, jpa kind of package for hbase

Kundera also has support for HBase as far as I'm aware. On Thu, Mar 6, 2014 at 8:13 PM, jeevi tesh jeevitesh...@gmail.com wrote: Hi all, I'm new to hbase in search of jpa kind of package for hbase to push the data into hbase system. Started trying with stargate where i found very strict

Upgrading from pseudo distributed mode to fully distributed mode

Hi, I have hbase setup in pseudo distributed atop HDFS ( again pseudo distributed mode ) in production. This Hbase version is 0.94.7 and Hadoop version is 1.1.2. I was planning to have a fully distributed 2 node cluster on 2 new machines with latest stable Hbase and Hadoop versions and then copy

Re: Upgrading from pseudo distributed mode to fully distributed mode

Please note i am planning to have hbase 0.94.17 and Hadoop 1.2.1 in new cluster. On Fri, Mar 7, 2014 at 10:52 AM, Vimal Jain vkj...@gmail.com wrote: Hi, I have hbase setup in pseudo distributed atop HDFS ( again pseudo distributed mode ) in production. This Hbase version is 0.94.7 and

Re: connecting to hbase remotely

thanks Jean and Ted, issue was resolved, i was using multiple configurations for two different tables, for one table i specified config property but for another i missed it, that caused the issue. in my application i made Rest service on hbase data, for every call i am creating a connection to

Re: Fetching HBase Region info for a particular table on a particular region server

Thanks Ted. Works just fine. -- View this message in context: http://apache-hbase.679495.n3.nabble.com/Fetching-HBase-Region-info-for-a-particular-table-on-a-particular-region-server-tp4056742p4056796.html Sent from the HBase User mailing list archive at Nabble.com.