The performance isn't horrible, hundreds of rows/sec and if you use the
write buffer it will speed things up. There is also deferred flush for less
durability but vastly increased performance.
On Oct 13, 2010 6:47 PM, "Stack" <[email protected]> wrote:
> On Mon, Oct 11, 2010 at 5:12 PM, Sean Bigdatafun
> <[email protected]> wrote:
>> Can someone give me a detailed look at the HLog mechanism for 0.90
>> durablity?
>> I recall that HBase committers claim that data will be truly durable in
0.90
>> after the client gets 'ok' acknowledgement from server, while it was not
>> true in 0.20 (i.e., HBase may have the chance to lose the data even it
says
>> 'ok, I got it' to clients). Can someone give me some detailed info?
>>
>
> The issue was that HDFS did not support appends. It would only retain
> data that was in a file that had been properly closed. If a
> regionserver crash while we were appending to the WAL/HLog -- i.e. the
> crash would make it so the writer did not properly close the WAL/HLog
> -- we'd lose effectively all edits that had been applied to the
> WAL/HLog up to the time of the crash.
>
>> First, HLog can't seem to be flushed onto disk for each Put operation for
>> performance reason (i.e., there exists time window when the log of a
certain
>> amout of hlog reside in the RegionServer's memory). If this understanding
is
>> correct, how are we going to assure total durablity?
>>
>
> You can have total durability if you want. You will pay in slower
> performance -- as it is for any database.
>
> St.Ack
