We have test a cluster which have more than 30,000 regions, max size of a region is 512MB. At this situation, data no more growing, but remove some old data and insert new, and regions will be more and more. This occupies too much heapsize, and will be more if regions cannot be merged. And it takes too long to make the table offline.
Zhou Shuaifeng(Frank) HUAWEI TECHNOLOGIES CO.,LTD. ------------------------------------------------------------------------------------------------------------------------------------- This e-mail and its attachments contain confidential information from HUAWEI, which is intended only for the person or entity whose address is listed above. Any use of the information contained herein in any way (including, but not limited to, total or partial disclosure, reproduction, or dissemination) by persons other than the intended recipient(s) is prohibited. If you receive this e-mail in error, please notify the sender by phone or email immediately and delete it! -----邮件原件----- 发件人: Ryan Rawson [mailto:[email protected]] 发送时间: 2011年2月9日 14:41 收件人: [email protected] 主题: Re: Can region be merged with others automatically when all data in the region has expired and removed ? I'm curious, if you are expiring a lot of data, does your table grow? If not, could you fit it in to a mysql instance instead? As for pre-splitting tables, if you have a really large data set, how would you manage this? One of our tables has 700 regions, and we didnt pre-split. I didnt really know the distribution of keys before I started inserting data, and I'd rather just let HBase do the right thing. -ryan On Tue, Feb 8, 2011 at 10:21 PM, Ted Dunning <[email protected]> wrote: > Online merge is a bit dangerous. Lots of applications require that the > table be set up pre-split. This is probably more common than the need for > merging. > > Having such a pre-split table collapse before it is full would be a > disaster. > > It should be pretty easy to script taking a few regions off-line and then > nuking them. > > 2011/2/8 Jean-Daniel Cryans <[email protected]> > >> An automatic online merge feature would be a nice contribution too. >> >
