Re: shedding regions when one region server dies

By trunk, you mean 0.89 or 0.20.6? -Jack On Mon, Oct 4, 2010 at 10:59 PM, Jack Levin magn...@gmail.com wrote: Full stop of all region servers, restart of master, is what brings it all back: Please attached.  Lots of data there, search for 'Shedding'. -Jack On Mon, Oct 4, 2010 at 9:42

Re: shedding regions when one region server dies

I mean TRUNK. 0.89s have been cut from TRUNK every 3 or 4 weeks or so. J-D is about to put up our next 0.89. It does not have new loadbalancer. The next release we hope will be 0.90.0RC1. That'll have the new balancer. Feature freeze is this weds. Hopefully it'll be up not too long after

Re: shedding regions when one region server dies

I probably will wait... the 'Shedding' seems like a race condition bug, if you grep for it in the log I've sent you will see what I mean. -Jack On Mon, Oct 4, 2010 at 11:15 PM, Stack st...@duboce.net wrote: I mean TRUNK. 0.89s have been cut from TRUNK every 3 or 4 weeks or so. J-D is about

RE: Paid OSS task for performing manual major compactions

HBASE-917 looks relevant too. -Original Message- From: Andrew Purtell [mailto:apurt...@apache.org] Sent: Tuesday, October 05, 2010 11:50 AM To: user@hbase.apache.org Subject: Re: Paid OSS task for performing manual major compactions From: Daniel Einspanjer Mozilla recently

HBase map reduce job timing

I've a mapreduce job that is taking too long..over an hour..Trying to see what can a tune to to bring it down..One thing I noticed, the job is kicking off - 500+ map tasks : 490 of them do not process any records..where as 10 of them process all the records (200 K each..)..Any idea why

Re: HBase map reduce job timing

Sorry..yeah..i've to do some digging to provide some data.. What sort of data would be helpful? Would stats reported by jobtracker.jsp suffice? I've pasted that in this email.. I can gather more jvm stats..thanks Status: Succeeded Started at: Tue Oct 05 21:39:58 EDT 2010 Finished at: Tue

Can I pick which region server to store my row?

Hi folks, I have a general question about Hbase. Can we pick which region server we want to save a particular row? The reason I am asking this is because sometimes we want to manually balance region servers' load. If we could assign particular rows to particular region servers, we can have that

Re: Can I pick which region server to store my row?

In 0.90, you can turn off the balancer and then distribute the regions any way you wish with new 'move' command. St.Ack On Tue, Oct 5, 2010 at 8:13 PM, Jean-Daniel Cryans jdcry...@apache.org wrote: A row is always served a single region, so the question is more about region distribution. In

Re: HBase map reduce job timing

Sure..Both input output are HBase tables Input (mapper phase) - scanning a HBase table for all records within time range (using hbase timestamps) Output (reduce phase) - doing a Put to 3 different HBase tables -Original Message- From: Jean-Daniel Cryans jdcry...@apache.org To:

Parallel computing on HBase

Hi guys, Is there any project going on co-processing on region servers? Right now, we have to transfer all data from region servers to region client after query, is that right? This can be slow. Furthermore, the cpus on the region servers are not fully used. If we could distribute the computation

Re: HBase map reduce job timing

Ah ok, then using the write buffer should get you the speed you need (providing that you have the hardware capacity and that you use HTable in a efficient way). In setup() set this to false on all 3 htables:

Re: Can I pick which region server to store my row?

So, I can use the 'move' command to manually balance the load? Is this available to 0.20.6? Was there any automatic balance mechanism in hbase before if the replicated block is not for load distribution purpose? Thanks. William On Tue, Oct 5, 2010 at 11:19 PM, Stack st...@duboce.net wrote: In

Re: Parallel computing on HBase

You can incorporate map reduce with hbase for parallel computing. On Wed, Oct 6, 2010 at 11:24 AM, William Kang weliam.cl...@gmail.com wrote: Hi guys, Is there any project going on co-processing on region servers? Right now, we have to transfer all data from region servers to region client

Re: Parallel computing on HBase

Can you tell me a little about how HBase works with MR? If the MR source/sink has to go through just ONE region client, then it is not I am looking for. But if MR can plug directly with the region server containing specific rows, then it might work. Furthermore, MR is a heavy weight process with

stopping namenode and regionservers

Hi, is it possible to tell regionservers not to die/stop when namenode is restarted? Every onces in a while I need to restart namenode, and it causes all regionservers pretty much to shutdown, and in many cases not cleanly, which causes long start up delays (hlog, etc). Thanks. -Jack