Is it possible to execute co-processors like cron job?

2015-06-19 Thread Chandrashekhar Kotekar
Hi, Can you please help to see if there is any feature in HBase which will satisfy following? 1) I need something which will be native to HBase and 2) Which will be able to execute some code at certain interval like daily or weekly. Is there any HBase feature which satisfies these two

Re: Is it possible to execute co-processors like cron job?

2015-06-19 Thread Anoop John
CPs are core extensions which act as pre/post hooks for events happening in the system. (like put, flush, compaction etc). This is not like cron jobs which can be scheduled. What action you want to run as cron job? Depending on that we can say whether HBase already allow some

Re: Is it possible to execute co-processors like cron job?

2015-06-19 Thread Jean-Marc Spaggiari
But you can still do call end points with a cronjob from a gateway node maybe? like echo blablabla | hbase shell from a script run daily? 2015-06-19 8:14 GMT-04:00 Anoop John anoop.hb...@gmail.com: CPs are core extensions which act as pre/post hooks for events happening in the system. (like

Re: Stochastic Balancer by tables

2015-06-19 Thread Dejan Menges
Just have to say that hbase.master.loadbalance.bytable saved us after we discovered it. In our case we had to set it manually to true, and then it was easy to catch hot spotting on unusually large regions and handle it. Btw +1 for HBASE-13013, had to say it, something that makes me starting

Re: HBase still try to start a zookeeper instance even with HBASE_MANAGES_ZK set to false in standalone mode.

2015-06-19 Thread Jean-Marc Spaggiari
Yep, I think I figured the same. But I think it does'nt start it at the end even if it tries, right? 2015-06-18 23:04 GMT-04:00 guxiaobo1982 guxiaobo1...@qq.com: Hi My Hbase version is 1.0 running on mac, it seems a bug .

Re: Stochastic Balancer by tables

2015-06-19 Thread Nasron Cheong
Hi, So it was the case that the region sizes were very skewed, partly due to the IncreasingToUpperBoundRegionSplitPolicy, which favours making lots of smaller regions early on. I couldn't find a tool to show regions and their sizes, for a specific table, so ended up writing one. I used that

Re: Stochastic Balancer by tables

2015-06-19 Thread Nick Dimiduk
On Fri, Jun 19, 2015 at 7:45 AM, Nasron Cheong nas...@gmail.com wrote: I couldn't find a tool to show regions and their sizes, for a specific table, so ended up writing one. Nasron, Would you mind having a look at the patch/RB on HBASE-13103? Does the API pair RegionNormalizer/Normalization

Re: Stochastic Balancer by tables

2015-06-19 Thread Nasron Cheong
Hi Mikhail, Something like N * number of region servers, or something different? It's pretty much this, but additionally also trying to ensure that in best case we can fit the whole column in memory per region server. Another restriction is that since # MR map tasks == # of regions, keeping

Re: Stochastic Balancer by tables

2015-06-19 Thread Mikhail Antonov
Nasron, Yeah, looks like you pretty much implemented pieces of logic being discussed in HBASE-13103 :) So that's interesting, thanks for telling us. Wondering, how did you estimate the number of desired regions? Something like N * number of region servers, or something different? I couldn't find