[
https://issues.apache.org/jira/browse/HBASE-4487?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13117889#comment-13117889
]
dhruba borthakur commented on HBASE-4487:
-----------------------------------------
> This is like a group commit in HLog. We have a group committing going on
> inside in sync down in dfsclient.
The DFSClient write code path is pretty heavyweight (as it stands now), as
shown in the code DFSClient.writeChunk(). It acquires a synchronized section,
creates packets, stuffs crcs in it, acquires locks for the outgoing queue and
queues the packet. And all this is done while holding the HLog.updateLock,
which means it blocks out any other writes to the HLog for this time. Instead,
this patch queues the write-data completely inside the hbase code and the
HLog.updateLock is released much much earlier.
> If I read the patch correctly it adds new sync calls that are performed with
> the updateLock held
There are no additional code paths that this patch introduced where a HLog.sync
occurs within the HLog.updateLock. The only pre-existing place where a sync
calls occurs within the HLog.updateLock is in HLog.cleanupCurrentWriter. This
code path is invoked only during a logroll; and during a logroll all currently
executing transactions are anyways quiesced, so I see no impact to concurrency
in this code path. Please let me know if you disagree.
> if (this.closed) : Is this check not needed here then?
I think this code is still needed, this check was not introduced by this patch.
Any reasons why you think it might not be needed?
> Can we pass new config parameter through conf which enables this new feature ?
I can surely make this configurable. But I am doubtful if anybody would care to
run with this feature switched off (since it is a performance fix with
potentially no downsides for any usecase). But if you feel strongly about it, I
will make it configurable. However, most of the common code in HLog will not be
inside this configurable parameter, only the change in HRegion.increment() can
be switched on/off via this new config. Please let me know.
> The increment operation can release the rowlock before sync-ing the Hlog
> ------------------------------------------------------------------------
>
> Key: HBASE-4487
> URL: https://issues.apache.org/jira/browse/HBASE-4487
> Project: HBase
> Issue Type: Improvement
> Reporter: dhruba borthakur
> Assignee: dhruba borthakur
> Attachments: appendNoSync4.txt, appendNoSync5.txt, appendNoSync6.txt
>
>
> This allows for better throughput when there are hot rows.I have seen this
> change make a single row update improve from 400 increments/sec/server to
> 4000 increments/sec/server.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
