On Fri, Sep 3, 2010 at 7:57 AM, Michael Segel <[email protected]>wrote:
> > > > > Date: Fri, 3 Sep 2010 18:00:42 +0530 > > From: [email protected] > > Subject: Re: HBase secondary index performance > > To: [email protected] > > > > Thanks Andrey, > > > > * Setting the autoflush to false and increasing the writeBuffer > size to 12MB > > improved the writes to 100/s > > * custom indexing is good, but our data keeps changing every day. > So, probably > > indextable is the best option for us > > * Just added one more regionserver and it did not help. Actually it > went back > > to 60/s for some strange reason(with one client). The requests in the > hbase ui > > is not uniform across 2 region servers. One server is doing around 2000 > and the > > other 500. Probably once the region gets split and when we have lots of > data, > > writes will improve ? (Now it is just writing to one region for the main > table) > > * Is there some way to do bulk load the indexedtable? Earlier I > have used the > > bulk loader tool (mapreduce job which creates the regions offline) but > not sure > > whether it works with indexed table. > > Just a small suggestion... > > If you have a table that is populated and you add a new region server, your > data isn't going to balance itself out. > If you want to balance your existing data, you'll need to bring down hbase, > then run hadoop's balancer app. When its completed, you'll see that your > data is now spread more evenly across the cloud. Please remember that you > need to have HBase down when you run the balancer app. > > > The above is all incorrect. The data *will* balance itself out on HDFS after major compactions have taken place, and even before that, the regions *will* balance themselves across region servers. Running the balancer while HBase is running is also perfectly safe, though it is not necessary for performance reasons. -Todd > -- Todd Lipcon Software Engineer, Cloudera
