[
https://issues.apache.org/jira/browse/HBASE-25972?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Kadir Ozdemir updated HBASE-25972:
----------------------------------
Description:
HBase stores tables row by row in its files, HFiles. An HFile is composed of
blocks. The number of rows stored in a block depends on the row sizes. The
number of rows per block gets lower when rows get larger on disk due to
multiple row versions since HBase stores all row versions sequentially in the
same HFile after compaction. However, applications (e.g., Phoenix) mostly query
the most recent row versions.
The default compactor in HBase compacts HFiles into one file. This Jira
introduces a new store file writer which writes the retained cells by
compaction into two files, which will be called DualFileWriter. One of these
files will include the live cells. This file will be called a live-version
file. The other file will include the rest of the cells, that is, historical
versions. This file will be called a historical-version file. DualFileWriter
will work with the default compactor.
The historical files will not be read for the scans scanning latest row
versions. This eliminates scanning unnecessary cell versions in compacted files
and thus it is expected to improve performance of these scans.
was:
HBase stores tables row by row in its files, HFiles. An HFile is composed of
blocks. The number of rows stored in a block depends on the row sizes. The
number of rows per block gets lower when the rows has more than one version
since HBase stores all row versions sequentially in the same HFile after
compaction. However, applications (e.g., Phoenix) mostly query the most recent
row versions.
Let us assume that the compaction generates two HFiles instead of one. One of
these files stores only the most recent cell versions. Let’s call this
single-version HFile. The other HFile stores all the previous cell versions.
Let’s call this multi-version HFile. The files that are generated by memstore
flushes will be of type multi version. The major and minor compaction processes
will generate single-version files as well as multi-version files. This means
for the queries on the most recent row versions, HBase does not need to look
into multi-version HFiles that are older than the latest single-version HFiles.
The blocks of single-version HFiles will be denser than the current HFiles in
general and this will improve the query times for most recent row version
queries.
> Dual File Compaction
> --------------------
>
> Key: HBASE-25972
> URL: https://issues.apache.org/jira/browse/HBASE-25972
> Project: HBase
> Issue Type: Improvement
> Reporter: Kadir Ozdemir
> Assignee: Kadir Ozdemir
> Priority: Major
>
> HBase stores tables row by row in its files, HFiles. An HFile is composed of
> blocks. The number of rows stored in a block depends on the row sizes. The
> number of rows per block gets lower when rows get larger on disk due to
> multiple row versions since HBase stores all row versions sequentially in the
> same HFile after compaction. However, applications (e.g., Phoenix) mostly
> query the most recent row versions.
> The default compactor in HBase compacts HFiles into one file. This Jira
> introduces a new store file writer which writes the retained cells by
> compaction into two files, which will be called DualFileWriter. One of these
> files will include the live cells. This file will be called a live-version
> file. The other file will include the rest of the cells, that is, historical
> versions. This file will be called a historical-version file. DualFileWriter
> will work with the default compactor.
> The historical files will not be read for the scans scanning latest row
> versions. This eliminates scanning unnecessary cell versions in compacted
> files and thus it is expected to improve performance of these scans.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
