Edward Bortnikov commented on HBASE-16608:

[~devaraj], you have raised multiple issues. Let me summarize the answers. 
(Some have been answered in prior communication by different folks).

1. Performance degradation of mainstream HBase. There is none - because 
Compacting Memstore makes no changes to the Default Memstore. We'd love to see 
it become the new default overtime, but at the moment it is intentionally built 
as a side feature. 
2. Stability of mainstream HBase. No issue here either. Compacting Memstore 
does not impact the regression at all, on the contrary, it extends it with 
multiple UT's. 
3. Stability of Compacting Memstore implementation. Specifically, in this 
subtask, a fickle bug was introduced that showed up consistently under huge 
stress in PE. It has been fixed in the most recent patch, and does not 
reproduce anymore. In fact, the current subtask is a moderate-sized one; much 
larger modifications have been committed to trunk before for the reasons 
indicated by [~stack].
4. External documentation. You are right, it is missing very much, especially 
for those want to try the feature out. The reason we delayed the publication is 
that we wanted to reduce manual configuration as much as possible, and we're 
still playing with this automation. Following [~stack]'s advice, we created a 
new subtask (HBASE-16851) focusing on documentation. Please follow the 
configuration instructions in the doc shared thru that jira; they are subject 
to change. 
5. Compacting Memstore performance. We have very good results demonstrated at 
HBaseCon '16, see the presentation attached to HBASE-16851. In a nutshell, the 
more skewed your write workload is, the more value you get from the algorithm. 
We are working on reproducing the results with a much larger dataset, through 
both the YCSB and PE tools; these results will be published on the same public 
blog post. Expected completion Nov '16. 
6. Remaining dev work. There are two issues on the table: (1) automating the 
in-memory compaction policy for minimum configuration (HBASE-16417), and (2) 
off-heap implementation for the flat cell index (CellChunkMap). The latter is a 
standalone project that has no jira yet; in-memory compaction has value 
regardless of off-heaping.  

Long laundry list - but hopefully answers the questions posted before. Please 
keep asking if something is not clear, we'd love to take the questions. 
[~anastas] and I will be in Bay Area during the week of Dec 5, it's possible to 
set up a mini-meetup if this makes sense. 

> Introducing the ability to merge ImmutableSegments without copy-compaction or 
> SQM usage
> ---------------------------------------------------------------------------------------
>                 Key: HBASE-16608
>                 URL: https://issues.apache.org/jira/browse/HBASE-16608
>             Project: HBase
>          Issue Type: Sub-task
>            Reporter: Anastasia Braginsky
>            Assignee: Anastasia Braginsky
>         Attachments: HBASE-16417-V02.patch, HBASE-16417-V04.patch, 
> HBASE-16417-V06.patch, HBASE-16417-V07.patch, HBASE-16417-V08.patch, 
> HBASE-16417-V10.patch, HBASE-16608-V01.patch, HBASE-16608-V03.patch, 
> HBASE-16608-V04.patch, HBASE-16608-V08.patch

This message was sent by Atlassian JIRA

Reply via email to