Vladimir Rodionov commented on HBASE-22749:

Updated design document to v2.2. Added totally new MOB compaction algorithm 
section, which now can limit for sure, overall Read/Write I/O amplification 
(major concern so far) The initial patch is almost done, just need to fix the 
algorithm and run tests. 

> Distributed MOB compactions 
> ----------------------------
>                 Key: HBASE-22749
>                 URL: https://issues.apache.org/jira/browse/HBASE-22749
>             Project: HBase
>          Issue Type: New Feature
>          Components: mob
>            Reporter: Vladimir Rodionov
>            Assignee: Vladimir Rodionov
>            Priority: Major
>         Attachments: HBase-MOB-2.0-v1.pdf, HBase-MOB-2.0-v2.1.pdf, 
> HBase-MOB-2.0-v2.2.pdf, HBase-MOB-2.0-v2.pdf
> There are several  drawbacks in the original MOB 1.0  (Moderate Object 
> Storage) implementation, which can limit the adoption of the MOB feature:  
> # MOB compactions are executed in a Master as a chore, which limits 
> scalability because all I/O goes through a single HBase Master server. 
> # Yarn/Mapreduce framework is required to run MOB compactions in a scalable 
> way, but this won’t work in a stand-alone HBase cluster.
> # Two separate compactors for MOB and for regular store files and their 
> interactions can result in a data loss (see HBASE-22075)
> The design goals for MOB 2.0 were to provide 100% MOB 1.0 - compatible 
> implementation, which is free of the above drawbacks and can be used as a 
> drop in replacement in existing MOB deployments. So, these are design goals 
> of a MOB 2.0:
> # Make MOB compactions scalable without relying on Yarn/Mapreduce framework
> # Provide unified compactor for both MOB and regular store files
> # Make it more robust especially w.r.t. to data losses. 
> # Simplify and reduce the overall MOB code.
> # Provide 100% compatible implementation with MOB 1.0.
> # No migration of data should be required between MOB 1.0 and MOB 2.0 - just 
> software upgrade.

This message was sent by Atlassian Jira

Reply via email to