[
https://issues.apache.org/jira/browse/HBASE-23730?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17023357#comment-17023357
]
Sean Busbey commented on HBASE-23730:
-------------------------------------
I second Stack's request that we implement this in a way that there's a single
code path used both for HDFS and object stores. It'll be much easier to
maintain.
> Optimize Memstore Flush for Hbase on S3(Object Store)
> -----------------------------------------------------
>
> Key: HBASE-23730
> URL: https://issues.apache.org/jira/browse/HBASE-23730
> Project: HBase
> Issue Type: Improvement
> Components: regionserver
> Reporter: Jarred Li
> Priority: Major
>
> The current Memstore Flush Process is divided into 2 stages:
> # Flushcache: In this stage, a “.tmp” region file is written in S3/HDFS for
> the memstore;
> # Commit: In this stage, the “.tmp” file created in the stage 1 is renamed
> to final destination of HBase region file.
> The above design(flush and commit) is OK for HDFS because “rename” is light
> opertion(only metadata operation). However, for storage like S3 or other
> object store, rename is “copy” and “delete” operation.
> We can follow the same pattern from V2 of “FileOutputCommitter” in
> MapReduce. That means, we can write hfile directly to the S3 destination
> directory without “copy” and “paste”. So that we can have less S3 operations
> and the HBase memstore flush is more efficient.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
