Please note that you will get a prompt response about HBase questions if you ask them on the HBase user list ( [EMAIL PROTECTED] )
> -----Original Message----- > From: Mork0075 [mailto:[EMAIL PROTECTED] > Sent: Sunday, August 17, 2008 11:51 PM > To: [email protected] > Subject: Re: Why is scaling HBase much simpler then scaling a relational db? > > I've read some papers and tutorials this week and now got some conrete > questions: > > (1) Sharding is also available in common relational systems. Often it is > discribed that you need an application layer for the (shards) > federation. I unterstand HBase like this layer, which implements the > whole sharding thing. HBase distributes the shards (regions) over the > region servers if a certain size increases. > > Wouldn't it be more practicabel to distribute the regions by load and > not by size? So if a region gets more requests (load), it would be > splittet and ditributed? Bigtable uses both. First it splits row ranges based on size. It also has the ability to detect hot row ranges and will split a region if it becomes too hot. This is tricky because you don't want to have a hot range split off and then have it drop below the too hot threshold forcing a range merge. If you get into this situation, you could end up with oscillation of split - merge - split - etc. > (2) What happens if the ressources of one machine are not able to handle > the load of the most used region (which is still splitted to the > maximum) - then you need replication, so that one region is served by > multiple region servers. Is this feature planed? HBase 0.2.0 provides simple load balancing. Currently, it only balances the number of regions per server. More sophisticated balancing is planned for the future (either 0.3.0 or 0.4.0). The advanced balancing will consider the load on regions as well. Allowing multiple servers to serve the same region is only practical if the region is read-only. Otherwise you run into distributed cache coherency problems. > (3) I don't understand at the moment, how restoring after crash works. > On each commit, a snippet of data is written into the commit log which > is distributed/replicated over all region servers? So how many servers > may die whithout loosing any data? No. Each region server keeps its own redo log. When a region server dies, the master splits its log into one log for each region the server was serving, before the master reassigns the regions. When the regions are reassigned, the first thing the region server does is apply the log for that region, before announcing that the region is back on-line. > (4) How is backup of the data realised? In a perhaps MySQL environment > you run the Admin console and perform a backup. Is this also available > in an HBase enviroment? We currently do not support backup (other than taking HBase down and copying the files to some other location). As each data block is replicated three times (the default for HDFS), the probability of losing data is low.
