[
https://issues.apache.org/jira/browse/HBASE-2957?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12906923#action_12906923
]
ryan rawson commented on HBASE-2957:
------------------------------------
would you use the delayed sync option? It has the similar semantics, where we
don't wait for the HDFS acks and continue with the memstore applications and
then return to the client. Data is only persisted when EITHER a different put
without this option comes in, OR the time-based flush kicks in. Our tests on a
prior implementation of this indicated amazing speedups.
Also todd's idea yesterday won't work - you can't put a long pole event (hlog
sync) inside a memstore transaction sequence. We already had something like
that and It Just Did Not Work.
I think another option would be:
Get latest Value for ICV
create next value
Write HLog & sync
Obtain row lock
begin memstore insert
insert memstore values
commit memstore
release row lock
return to client
But this wouldn't work for ICVs either! Multiple clients would create multiple
next values, and that would be bad.
If we had this potential implementation:
Write HLog diff of ICV (ie: +1 to so and so)
Obtain row lock
get existing value
create new value
begin memstore insert
insert memstore values
commit memstore
release row lock
return to client
I think you should seriously look into delayed sync. You can run your sync
thread as often as 100ms and it still improves performance. 100ms of lost data
in the worst case isnt so bad... right?
> Release row lock when waiting for wal-sync
> ------------------------------------------
>
> Key: HBASE-2957
> URL: https://issues.apache.org/jira/browse/HBASE-2957
> Project: HBase
> Issue Type: Improvement
> Components: regionserver, wal
> Affects Versions: 0.20.0
> Reporter: Prakash Khemani
>
> Is there a reason to hold on to the row-lock while waiting for the WAL-sync
> to be completed by the logSyncer thread?
> I think data consistency will be guaranteed even if the following happens (a)
> the row lock is held while the row is updated in memory (b) the row lock is
> released after queuing the KV record for WAL-syncing (c) the log-sync system
> guarantees that the log records for any given row are synced in order (d) the
> HBase client only receives a success notification after the sync completes
> (no change from the current state)
> I think this should be a huge win. For my use case, and I am sure for others,
> the handler thread spends the bulk of its row-lock critical section time
> waiting for sync to complete.
> Even if the log-sync system cannot guarantee the orderly completion of sync
> records, the "Don't hold row lock while waiting for sync" option should be
> available to HBase clients on a per request basis.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
