[
https://issues.apache.org/jira/browse/HBASE-1213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12707088#action_12707088
]
stack commented on HBASE-1213:
------------------------------
This is about carrying batches deeper down into the Region. We'd then get lock
on HLog, run a batch, let the HLog lock go and then apply the commit logged
edits across Stores.
This is going to become even more critical as HLog becomes more expensive when
flush/sync actually starts working.
Currently, edits go in a row at a time. The commit log and the commit to
memcache are both done within the row lock. We'd break this batching HLog
writes. Would make it so commit log had edits not yet added to memcache. This
is probably OK.
Of note, if a row has many edits to apply -- the BatchUpdate touches many
columns -- these are applied as a batch to HLog and then to the memcache.
If we could presume the array of incoming edits already sorted by region, then
we could pass sub-arrays down to Regions -- no need to sort first or repackage
in a new array or List.
We'd need a new batchUpdate method in HRegion, one that took sub-arrays of
Region edits. It'd pass the edits to the HLog first, then one by one, call
current batchUpdate only with writeToWAL set to false.
> Investigate Locking & Contention in the Write Path
> --------------------------------------------------
>
> Key: HBASE-1213
> URL: https://issues.apache.org/jira/browse/HBASE-1213
> Project: Hadoop HBase
> Issue Type: Improvement
> Affects Versions: 0.19.0
> Reporter: Ben Maurer
> Assignee: stack
> Fix For: 0.20.0
>
>
> When doing a large number of bulk updates from different clients, I noticed
> that there was a high level of lock contention for stuff like locking the
> HLog. It seems that each thread acquires the lock for a single BatchUpdate,
> releases the lock then another thread owns the lock before the initial writer
> gets to the next update. Having the threads bounce around may lead to
> suboptimal performance.
> Should be benchmarked & maybe changed to have less context switching.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
