[
https://issues.apache.org/jira/browse/HBASE-21721?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16742805#comment-16742805
]
Bo Cui commented on HBASE-21721:
--------------------------------
when testing with hbase pe on 3 rs：
if regions num is 10, Requests Per Second = 30000+（ hbase pe --table=petest04
--autoFlush=True --columns=10 --valueSize=100 --presplit=20 --bloomFilter=NONE
--rows=100000000 randomWrite 100）
if regions num is 200, Requests Per Second = 3000-4000，（ hbase pe
--table=petest04 --autoFlush=True --columns=10 --valueSize=100 --presplit=200
--bloomFilter=NONE --rows=100000000 randomWrite 100），and running-log frequent
prints FSHLog info
2019-01-15 09:34:40,379 | INFO | sync.3 | Slow sync cost: 470 ms, current
pipeline:
[DatanodeInfoWithStorage[192.168.203.11:25009,DS-45187cf0-e521-42ed-8fa9-cb554faffdd2,DISK],
DatanodeInfoWithStorage[192.168.206.2:25009,DS-fa58557a-1e30-40f2-a890-3cd4d0ac74be,DISK],
DatanodeInfoWithStorage[192.168.203.13:25009,DS-e9dc57e9-d702-4d33-9245-865d7bf2d0bd,DISK]]
| org.apache.hadoop.hbase.regionserver.wal.FSHLog.postSync(FSHLog.java:1454)
2019-01-15 09:34:40,479 | INFO | sync.4 | Slow sync cost: 495 ms,
2019-01-15 09:34:40,590 | INFO | sync.0 | Slow sync cost: 522 ms,
2019-01-15 09:34:40,680 | INFO | sync.1 | Slow sync cost: 512 ms,
2019-01-15 09:34:40,771 | INFO | sync.2 | Slow sync cost: 492 ms,
analysis code, i found that FSHLog can be optimized to reduce the times of
write#sync()
> reduce write#syncs() times
> --------------------------
>
> Key: HBASE-21721
> URL: https://issues.apache.org/jira/browse/HBASE-21721
> Project: HBase
> Issue Type: Improvement
> Affects Versions: 1.3.1, 2.1.1
> Reporter: Bo Cui
> Priority: Major
>
> the number of write#syncs can be reduced by updating the
> highestUnsyncedSequence：
> before write#sync(), get the current highestUnsyncedSequence
> after write#sync, highestSyncedSequence=highestUnsyncedSequence
>
> {code:title=FSHLog.java|borderStyle=solid}
> // Some comments here
> public void run()
> {
> long currentSequence;
> while (!isInterrupted()) {
> int syncCount = 0;
> try {
> while (true) {
> ...
> try {
> Trace.addTimelineAnnotation("syncing writer");
> long unSyncedFlushSeq = highestUnsyncedSequence;
> writer.sync();
> Trace.addTimelineAnnotation("writer synced");
> if( unSyncedFlushSeq > currentSequence ) currentSequence =
> unSyncedFlushSeq;
> currentSequence = updateHighestSyncedSequence(currentSequence);
> } catch (IOException e) {
> LOG.error("Error syncing, request close of WAL", e);
> lastException = e;
> } catch (Exception e) {
> ...
> }
> }
> {code}
> Add code
> long unSyncedFlushSeq = highestUnsyncedSequence;
> if( unSyncedFlushSeq > currentSequence ) currentSequence = unSyncedFlushSeq;
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
