Re: HBase resgionServer crashed with no gc detected

Thanks Ted.I uploaded another log https://github.com/eswidy/waterspider/tree/master/rscase/rs-more.log Followed you advice i increased the tickTime and works well at present. Maybe the problem caused by he bad I/O,I found the CPU I/O idle always more than 70% during the heavy load. But that

Re: [Query :] hbase rebalancing the data after adding new nodes in cluster

I have deleted my table but when I am going to re -create it it said Family 'C1' already exists, the old one will be replaced i used below command echo -e "disable_all '.*'\ny" | hbase shell -n echo -e "drop_all '.*'\ny" | hbase shell -n Thanks Manjeet On Thu, Oct 20, 2016 at 4:19 PM, Manjeet

Re: [Query :] hbase rebalancing the data after adding new nodes in cluster

Hi Jean, I recreated the table as same I create befor and I disabled it droped it and run Major compaction and after that i again re create the table at this time I got above message below is my table create command create 'TEST_TABLE','C1',{ NAME => 'C1', COMPRESSION => 'SNAPPY' }

Re: [Query :] hbase rebalancing the data after adding new nodes in cluster

Hi, in my dev enviorment my one data node get crashed and we removed that data node from cluster and by mistak I run HDFS Balancer insted of HBase Blancer and my HBase is down can any one suggest me how can I up the Hbase cluster On Fri, Oct 7, 2016 at 1:09 PM, Ted Yu

Re: [Query :] hbase rebalancing the data after adding new nodes in cluster

Hi Manjeet, Probably because your table is not really deleted. Can you "list" the tables to confirm? For the balacing, just run a major compaction of your table and locallity will come back. JMS 2016-10-20 6:58 GMT-04:00 Manjeet Singh : > I have deleted my table but

Re: [Query :] hbase rebalancing the data after adding new nodes in cluster

Manjeet: 'C1' appeared twice in the first command below. Remove the first occurrence and try again. Cheers > On Oct 20, 2016, at 4:38 AM, Manjeet Singh wrote: > > Hi Jean, > > I recreated the table as same I create befor and I disabled it droped it > and run

Scan a region in parallel

HI, I am loading hbase table into an in-memory db to support filter, ordering and pagination. I am scanning region and inserting data into in-memory db. each region scan is done in single thread so each region is scanned in parallel. Is there any way to scan a region in parallel ? any pointers

Re: HBase resgionServer crashed with no gc detected

I haven't found more clue from the latest log. I noticed DEBUG log was not turned on. Please keep monitoring and get back if you encounter region server crash in the future. On Thu, Oct 20, 2016 at 3:07 AM, who.cat wrote: > Thanks Ted.I uploaded another log

Parallelizing TableSnapshotScanner

Hi, I have attempted to parallelize the TableSnapshotScanner by taking the ClientSideRegionScanner’s it generates and sending them each to their own thread to read the results. I am finding that I do not get all the rows per region when I do this. It seems the more parallelism the less data

Re: Parallelizing TableSnapshotScanner

I can work on that. Since it is reading the restore data from a snapshot it is definitely not changing. I wanted to make sure there was not some inherent thread model under the scanner that prohibited this from working. On 10/20/16, 8:30 AM, "Ted Yu" wrote: >Can you

Doing map-reduce with Hive external table on Hbase throws error

Hive 2.0.1 Hbase 0.98 hive> select max(price) from test.marketdatahbase; Throws: Caused by: java.lang.NoSuchMethodError: org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ I have both hbase-protocol-0.98.21-hadoop2.jar and protobuf-java-2.5.0.jar in $HBASE_HOME/lib ditectory Full error

Re: Doing map-reduce with Hive external table on Hbase throws error

I downloaded hive 2.0.1 source tar ball. In their pom.xml : 1.1.1 Can you run against 1.1.1 or newer hbase release ? On Thu, Oct 20, 2016 at 8:58 AM, Mich Talebzadeh wrote: > Hive 2.0.1 > Hbase 0.98 > > hive> select max(price) from test.marketdatahbase; > >

Re: Parallelizing TableSnapshotScanner

Can you come up with unit test that shows what you observed ? You may have seen the javadoc for ClientSideRegionScanner : * A client scanner for a region opened for read-only on the client side. Assumes region data * is not changing. Cheers On Thu, Oct 20, 2016 at 8:26 AM, Randy Fox

Re: Doing map-reduce with Hive external table on Hbase throws error

Thanks Ted hbase-1.2.3 worked! Dr Mich Talebzadeh LinkedIn * https://www.linkedin.com/profile/view?id=AAEWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw * http://talebzadehmich.wordpress.com *Disclaimer:* Use it at

Re: Tips on Writing Custom HBase Filters

master branch is compiled with Java 8 where removeIf() is supported. For master branch, there is no need to use Guava, right ? On Thu, Oct 20, 2016 at 12:08 PM, Robert Yokota wrote: > Here are some tips on writing HBase filters in case anyone is interested: > >

Endpoint coprocessor performance issue

I had written to an end point coprocess which take in an Request object and builds an Multi Range filter scan. This scan is executed and results are aggregated and sent back. Performance of the co processor is very slow. I am able to get the result back.. but it takes 2 minutes. I am able to find

Tips on Writing Custom HBase Filters

Here are some tips on writing HBase filters in case anyone is interested: https://rayokota.wordpress.com/2016/10/20/tips-on-writing-custom-hbase-filters/

Re: Tips on Writing Custom HBase Filters

Yes, if you are using Java 8, Collection.removeIf() should work as well. Thanks, I'll mention that in the blog On Thu, Oct 20, 2016 at 12:43 PM, Ted Yu wrote: > master branch is compiled with Java 8 where removeIf() is supported. > For master branch, there is no need to use

setup two hbase instances on Mac?

hi, folks, I am trying to setup a simple development environment on my Mac Book. And like to have multiple instances of HBases, for some testing of replication, backup. etc. And wondering there is any instruction to setup for multiple instances(not the VM/container way). Here is what I did so

Re: setup two hbase instances on Mac?

Actually I don't have a good reason of 'not use container', except that I already have homebrew install hadoop and hbase on my laptop, hence like to just keep using it. Thanks for the instruction through the blog. A quick question to clarify: the blog is for multi-node cluster, instead of

Re: setup two hbase instances on Mac?

So what you could essentially do is use the Apache HBase topology for clusterdock, running the clusterdock_run ./bin/start_cluster command twice, once for each cluster you want to start. I can provide specific command line arguments if you let me know which version of HBase you're hoping to use.

Re: setup two hbase instances on Mac?

Any reason to not use the container way via clusterdock [1]? I do replication testing on my Mac for this using it and have had pretty good results. 1. http://blog.cloudera.com/blog/2016/08/multi-node-clusters-with-cloudera-quickstart-for-docker/ -Dima On Thu, Oct 20, 2016 at 2:51 PM, Demai Ni

Re: Hbase cluster not getting UP one Region server get down

It can be lots of things, Manjeet. You've gotta do a bit of troubleshooting yourself first; a long dump of your machine specs doesn't change that. Can you describe what happened before/after the node went down? The log just says server isn't running, so we can't tell much from that alone. -Dima

Re: setup two hbase instances on Mac?

Dima, thanks. After a couple hours trying, I got it works now. So will try the docking method another time. Just in case someone else plan to do this kind of exercise in the future, and to keep a closure for myself. Here are my changes: /** *first HBase*, is installed by homebrew and using HDFS

Re: Hbase native API Utility to test hbase table stats

Is your method of getting stats for unique records similar to what RowCounter does ? Can you describe your requirement in bit more detail (I don't fully understand from your initial email) ? Thanks On Thu, Oct 20, 2016 at 5:48 PM, Manjeet Singh wrote: > Hi All > >

Hbase native API Utility to test hbase table stats

Hi All I am using native API to test hbase table stats like unique records how many column qualifier size of hbase table etc. It woking fine with 1 lack or 2 but when record get increased its hang. Is their any reanimate utility? Or any buddy get chance to create it? Thanks Manjeet

Re: HBase resgionServer crashed with no gc detected

Thanks for your help .I'll keep on tuning it . I turn on debug yesterday but nothing found. Thanks again. -- Original -- From: "Ted Yu";; Send time: Thursday, Oct 20, 2016 9:43 PM To: "user@hbase.apache.org";

RE: Hbase Coprocessor postPut not triggered for

We found the issue in our code itself. These tests work fine too. Thanks, Veena. -Original Message- From: Ted Yu [mailto:yuzhih...@gmail.com] Sent: Monday, October 17, 2016 9:55 AM To: user@hbase.apache.org Subject: Re: Hbase Coprocessor postPut not triggered for Your scenario should

Re: Scan a region in parallel

Any pointers ? On 20 October 2016 at 18:15, Anil wrote: > HI, > > I am loading hbase table into an in-memory db to support filter, ordering > and pagination. > > I am scanning region and inserting data into in-memory db. each region > scan is done in single thread so each

Re: Hbase cluster not getting UP one Region server get down

Hi Dima My one node get crashed due to hdd crashed and as I have only 4 node I configure zk on 3 node as it should be in odd figure and my zk was on this crashed node. Due to this my whole cluster goes down and by mistake after removing my crashed node I run hdfs balancing which was again

Re: Hbase Coprocessor postPut not triggered for

Do you mind describing in a bit more detail (so that other people developing coprocessors can watch out) ? Thanks On Thu, Oct 20, 2016 at 7:12 PM, Begar, Veena wrote: > We found the issue in our code itself. These tests work fine too. > > Thanks, > Veena. > > -Original

Re: Hbase native API Utility to test hbase table stats

For example I have mobile subscribe data with their plan code So my rowkey designed in a way which is distrusted on cluster with pre split I want . Unique subscriber, (its not row count) How many plan they have as my row key like subscriber_planId What is the size of table/data On 21 Oct 2016

Re: Scan a region in parallel

Hi Anil So now you are spawning those many scan threads equal to the number of regions. bq.Is there any way to scan a region in parallel ? You mean with in a region you want to scan parallely? Which means that a single query you want to split up into N number of small scans and read and