[
https://issues.apache.org/jira/browse/HBASE-16608?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15579719#comment-15579719
]
Edward Bortnikov commented on HBASE-16608:
------------------------------------------
[~devaraj], you have raised multiple issues. Let me summarize the answers.
(Some have been answered in prior communication by different folks).
1. Performance degradation of mainstream HBase. There is none - because
Compacting Memstore makes no changes to the Default Memstore. We'd love to see
it become the new default overtime, but at the moment it is intentionally built
as a side feature.
2. Stability of mainstream HBase. No issue here either. Compacting Memstore
does not impact the regression at all, on the contrary, it extends it with
multiple UT's.
3. Stability of Compacting Memstore implementation. Specifically, in this
subtask, a fickle bug was introduced that showed up consistently under huge
stress in PE. It has been fixed in the most recent patch, and does not
reproduce anymore. In fact, the current subtask is a moderate-sized one; much
larger modifications have been committed to trunk before for the reasons
indicated by [~stack].
4. External documentation. You are right, it is missing very much, especially
for those want to try the feature out. The reason we delayed the publication is
that we wanted to reduce manual configuration as much as possible, and we're
still playing with this automation. Following [~stack]'s advice, we created a
new subtask (HBASE-16851) focusing on documentation. Please follow the
configuration instructions in the doc shared thru that jira; they are subject
to change.
5. Compacting Memstore performance. We have very good results demonstrated at
HBaseCon '16, see the presentation attached to HBASE-16851. In a nutshell, the
more skewed your write workload is, the more value you get from the algorithm.
We are working on reproducing the results with a much larger dataset, through
both the YCSB and PE tools; these results will be published on the same public
blog post. Expected completion Nov '16.
6. Remaining dev work. There are two issues on the table: (1) automating the
in-memory compaction policy for minimum configuration (HBASE-16417), and (2)
off-heap implementation for the flat cell index (CellChunkMap). The latter is a
standalone project that has no jira yet; in-memory compaction has value
regardless of off-heaping.
Long laundry list - but hopefully answers the questions posted before. Please
keep asking if something is not clear, we'd love to take the questions.
[~anastas] and I will be in Bay Area during the week of Dec 5, it's possible to
set up a mini-meetup if this makes sense.
> Introducing the ability to merge ImmutableSegments without copy-compaction or
> SQM usage
> ---------------------------------------------------------------------------------------
>
> Key: HBASE-16608
> URL: https://issues.apache.org/jira/browse/HBASE-16608
> Project: HBase
> Issue Type: Sub-task
> Reporter: Anastasia Braginsky
> Assignee: Anastasia Braginsky
> Attachments: HBASE-16417-V02.patch, HBASE-16417-V04.patch,
> HBASE-16417-V06.patch, HBASE-16417-V07.patch, HBASE-16417-V08.patch,
> HBASE-16417-V10.patch, HBASE-16608-V01.patch, HBASE-16608-V03.patch,
> HBASE-16608-V04.patch, HBASE-16608-V08.patch
>
>
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
