[
https://issues.apache.org/jira/browse/HBASE-20188?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16403904#comment-16403904
]
Eshcar Hillel commented on HBASE-20188:
---------------------------------------
Another reason flushes are too slow could be the disk cannot maintain the load.
Do you pre-split the regions before the test? If not then maybe the overhead of
multiple concurrent splits is too high. Can you estimate the rate at which the
test is writing, what is the throughput? what is the data size written at each
operation?
{quote} I was wondering what your thoughts were regards our doing MORE
aggressive in-memory compaction moving Cells from CSLM to your flat structures,
could it save on the number of overall compares (and hence CPU) or, if not on
compares, overhead from the CSLM itself shows up as a pretty big CPU user too?
What you reckon?
{quote}
All written cells are added first to the CSLM and there is no way around it.
With in-memory compaction the CSLM is smaller and hence overall compares should
be less than say in hbase1.4. To further reduce these compares you can decrease
the size of the CSLM by decreasing
{{hbase.memstore.inmemoryflush.threshold.factor}} . The current default is 0.1
but you can try 0.02 it worked well in the past.
{quote}But now we have to read from multiple segments in a heap way and so more
compares there.
{quote}
To check if this is indeed the problem you can decrease
{{hbase.hregion.compacting.pipeline.segments.limit}}. The default is currently
4, you can try it with 2.
> [TESTING] Performance
> ---------------------
>
> Key: HBASE-20188
> URL: https://issues.apache.org/jira/browse/HBASE-20188
> Project: HBase
> Issue Type: Umbrella
> Components: Performance
> Reporter: stack
> Priority: Critical
> Fix For: 2.0.0
>
> Attachments: flamegraph-1072.1.svg, flamegraph-1072.2.svg, tree.txt
>
>
> How does 2.0.0 compare to old versions? Is it faster, slower? There is rumor
> that it is much slower, that the problem is the asyncwal writing. Does
> in-memory compaction slow us down or speed us up? What happens when you
> enable offheaping?
> Keep notes here in this umbrella issue. Need to be able to say something
> about perf when 2.0.0 ships.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
