[
https://issues.apache.org/jira/browse/HBASE-16417?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15615299#comment-15615299
]
Edward Bortnikov commented on HBASE-16417:
------------------------------------------
Just to give a sense of what we've been thinking as possible auto-tuning policy
(smile). It's a "war driving" approach that is actually similar to
opportunistic scans we had once but is a bit smarter. Suppose we do full
(data) compaction once in a while; a by-product is the compaction factor - how
much space we saved. If the latter is small - schedule the next compaction
further away, using some exponential backoff scheme. For workloads with very
few duplicates - compactions will never happen, de-facto. For skewed workloads,
compactions will consistently prove valuable, and will run at a constant pace.
Note that the above is unrelated to whether we flush just one or all segments
in the pipeline once the disk flush time comes. Personal opinion - no problem
with flushing everything if this shows value. Let's wait for more benchmark
results, they're just around the corner. One more personal opinion - we should
strive to a generic policy, as much independent as possible on whether we use
MSLAB's or not, run on-heap or off-heap, etc.; let's see if we can get there.
Actually it's the most fun stage now - we have all the building blocks, the
goal is connecting them right :) Stay tuned, we'll keep sharing the results and
the ideas.
> In-Memory MemStore Policy for Flattening and Compactions
> --------------------------------------------------------
>
> Key: HBASE-16417
> URL: https://issues.apache.org/jira/browse/HBASE-16417
> Project: HBase
> Issue Type: Sub-task
> Reporter: Anastasia Braginsky
> Assignee: Eshcar Hillel
> Fix For: 2.0.0
>
>
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
