[
https://issues.apache.org/jira/browse/HBASE-5476?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrew Purtell resolved HBASE-5476.
-----------------------------------
Resolution: Incomplete
> Merge flush/merging compaction; save on i/o by merging an existing file with
> content memstore
> ---------------------------------------------------------------------------------------------
>
> Key: HBASE-5476
> URL: https://issues.apache.org/jira/browse/HBASE-5476
> Project: HBase
> Issue Type: Improvement
> Reporter: stack
>
> Compactions are slow. To improve, we can work on making compactions faster
> (TODO) or make it so compactions have less work to do. This issue is about
> the later; doing something the bigtable paper talks off where the content of
> memory is merged with existing content in the filesystem (I'm sure we've
> discussed this a bunch in the past but can't find an explicit issue for it).
> We save on the reading of the size of a memstore from the filesystem if we
> merge flush/merge compact.
> BT seems to include memstore content in minor compactions. We should look at
> doing that (would snapshot at start of compaction and then would integrate
> the snapshot into resultant compacted file). Alternatively we could flush
> into a small file that is already in the filesystem (this could be tougher
> given that we have this flush-compaction separation at the moment... how
> would we make it so the file we're merging into is not picked up for a normal
> compaction).
> Doing the merge compaction/flush merge would slow down flushes. It could
> back up memory into the global barrier such that we stop taking on writes
> altogether.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
