Eshcar Hillel commented on HBASE-20542:

Pushed to master and branch-2

> Better heap utilization for IMC with MSLABs
> -------------------------------------------
>                 Key: HBASE-20542
>                 URL: https://issues.apache.org/jira/browse/HBASE-20542
>             Project: HBase
>          Issue Type: Sub-task
>          Components: in-memory-compaction
>            Reporter: Eshcar Hillel
>            Assignee: Eshcar Hillel
>            Priority: Major
>             Fix For: 3.0.0, 2.2.0
>         Attachments: HBASE-20542-addendum.master.005.patch, 
> HBASE-20542.branch-2.001.patch, HBASE-20542.branch-2.003.patch, 
> HBASE-20542.branch-2.004.patch, HBASE-20542.branch-2.005.patch, 
> HBASE-20542.master.003.patch, HBASE-20542.master.005-addendum.patch, run.sh, 
> workloada, workloadc, workloadx, workloady
> Following HBASE-20188 we realized in-memory compaction combined with MSLABs 
> may suffer from heap under-utilization due to internal fragmentation. This 
> jira presents a solution to circumvent this problem. The main idea is to have 
> each update operation check if it will cause overflow in the active segment 
> *before* it is writing the new value (instead of checking the size after the 
> write is completed), and if it is then the active segment is atomically 
> swapped with a new empty segment, and is pushed (full-yet-not-overflowed) to 
> the compaction pipeline. Later on the IMC deamon will run its compaction 
> operation (flatten index/merge indices/data compaction) in the background. 
> Some subtle concurrency issues should be handled with care. We next elaborate 
> on them.

This message was sent by Atlassian JIRA

Reply via email to