Hi,

I am in the middle of performance testing on my service using HBase as
the backend, and I have difficulty to explain the write and read
performance difference, which bring up the following questions.

(1) write-ahead log

In the default mode of HTable “put”, does the write-ahead log get turn
on? If not, what is the API that allows me to turn on the write-ahead
log?  Without the write-ahead logging turned on, is it true that when
the region server dies, all the data that has been written by the
client but not flushed to the persistent HDFS will get lost?

So far I have not seen any documentation on the HBase configuration
parameters to specify where the write-ahead log is store. Does it get
stored in a reliable shared file system that the system administrator
needs to provide, for example, a SAN? Or HBase uses the HDFS as the
reliable shared file system? Previously I read some articles saying
that HBase is not using (or did not use) HDFS for write-ahead logging,
as the operation of “append” provided from HDFS is not robust enough.

(2) Failure recovery

Related to the write-ahead log problem that I brought up, could you
explain to me the current mechanism adopted in HBase, when a region
server fails and the client is making a request to fetch a row to this
particular region server? My understanding is that the master node
will detect that the region server fails and then starts to migrate
the data (including the write-ahead log) that is served by that region
to the other region and update the meta data of the master node after
the migration finishes.

The concern that I have is that such migration takes quite some time,
depending on the data size of the failed region server. For large
size, it will take, say couple of minutes. During this time window,
will it be the case that all the clients that try to access data rows
on that failed region server will all fail? As a result, the HBase
client will need to have a retry window that matches the region data’s
migration time.

Thank you very much for your information in advance!

Jun

Reply via email to