[
https://issues.apache.org/jira/browse/HBASE-16417?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15612232#comment-15612232
]
Edward Bortnikov commented on HBASE-16417:
------------------------------------------
[~anoop.hbase], [~eshcar] - the end state you both want to reach is the same,
just the ways of going there are different. Before going into any detail - the
holy grail is a self-tuning policy that does the right thing in EVERY use case
of interest. We'd like to achieve it without any additional configuration or
private solutions for specific cases. Reason is - what ends up as non-default
option will never be used.
Anoop - you want two things actually: (1) no compaction of any kind happen when
there is no redundancy and (2) flushing everything to disk when the memstore
overflows. (Note that these two can be decoupled.) Hopefully you don't mind if
there's a policy that magically figures out that we're in that use case, at
practically zero cost, and does exactly (1) and (2). We just disagree on
marking the opposite case (many duplicates) as special and going down a
different code path there - because if we leave it to the admin as non-default
we know what'll happen.
So we are after that magic policy. The quest won't take long but it has to be
data-driven. At the moment, we've just reproduced one microbenchmark (uniform
writes, no reads), but there are many other cases that should be looked at. We
have the env to run them, and we'll be producing those results over the next
couple of weeks. We'll be very much transparent in the process, publishing the
results frequently. Once we have the data let's decide collectively. If nothing
universal we'll work we can always back off to configs but I'd consider that
undesirable.
> In-Memory MemStore Policy for Flattening and Compactions
> --------------------------------------------------------
>
> Key: HBASE-16417
> URL: https://issues.apache.org/jira/browse/HBASE-16417
> Project: HBase
> Issue Type: Sub-task
> Reporter: Anastasia Braginsky
> Assignee: Anastasia Braginsky
> Fix For: 2.0.0
>
>
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
