[
https://issues.apache.org/jira/browse/HBASE-21738?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16747427#comment-16747427
]
Zheng Hu edited comment on HBASE-21738 at 1/20/19 10:31 AM:
------------------------------------------------------------
Made a performance test again after applied patch.v3, we can see the patch has
eliminated the request whose p999 > 100ms. While there's still some spike in
p999, will try to dig this in future issue. there's a possible reason: all
of our regions are flushed at the same time because of our balanced test data,
so the ssd disk will be very busy at the flushing time, finally latency go up,
If so, I will try to use the PressureAwareFlushThroughputController to balance
this.
Anyway, I'll commit patch.v3 to branch-2.x and master firstly.
was (Author: openinx):
Made a performance test again after applied patch.v3, we can see the patch has
eliminated the request whose p999 > 100ms. While there still be some spike in
p999, will try to dig this in future issue. there's a possible reason: all
of our regions are flushed at the same time because of our balanced test data,
so the ssd disk will be very busy at the flushing time, finally latency go up,
If so, I will try to use the PressureAwareFlushThroughputController to balance
this.
Anyway, I'll commit patch.v3 to branch-2.x and master firstly.
> Remove all the CLSM#size operation in our memstore because it's an quite time
> consuming.
> ----------------------------------------------------------------------------------------
>
> Key: HBASE-21738
> URL: https://issues.apache.org/jira/browse/HBASE-21738
> Project: HBase
> Issue Type: Sub-task
> Components: Performance
> Reporter: Zheng Hu
> Assignee: Zheng Hu
> Priority: Critical
> Fix For: 3.0.0, 2.2.0, 2.1.3, 2.0.5
>
> Attachments: HBASE-21738.v1.patch, HBASE-21738.v2.patch,
> HBASE-21738.v3.patch, add-some-log.patch, image-2019-01-18-14-03-28-662.png,
> log.txt, performance-after-the-patch.v3.png
>
>
> Made some performance test for 100% put case in branch-2 before.
> We can see that there are many latency peak in p999 latency curve , and the
> peak time are almost the point time which our region is flushing.
> See the [hbase20-ssd-put-10000000000-rows-latencys-and-qps
> |https://issues.apache.org/jira/secure/attachment/12955341/12955341_image-2019-01-18-14-03-28-662.png]
> And, I used the
> [add-some-log.patch|https://issues.apache.org/jira/secure/attachment/12955342/add-some-log.patch]
> to log some time consuming when we grab the update.writeLock() to make a
> memstore snapshot. Tested again, I found those logs in [log.txt.
> |https://issues.apache.org/jira/secure/attachment/12955343/log.txt]
> Seems most of the time was consumed when taking memstore snapshot.. Let me
> dig into this.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
