[
https://issues.apache.org/jira/browse/HBASE-25972?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17847372#comment-17847372
]
Andrew Kyle Purtell commented on HBASE-25972:
---------------------------------------------
Summary of compatibility related concerns:
* New HFile metadata key {{HStoreFile.HISTORICAL_KEY}}. Correctly handled by
new code if absent. Queries with old code that don't recognize it will perform
normally. Refer to design document.
* In {{CompactionConfiguration}}, only if the new configuration setting for
historical file structure is enabled then {{minFilesToCompact}} will be
incremented by 1.
All other changes are to Private interfaces.
I don't think there is anything here that would prevent a backport to releasing
lines except philosophical choice on introduction of new features in patch
versions. Per the design document, after upgrade the historical file
structuring is not enabled by default. If it is enabled, and later the cluster
is rolled back, the contents of the HFiles are all still there and readable by
the older code, only the potential read time optimization is obviously not
performed by the older code. YDYT [~bbeaudreault] ?
> Dual File Compaction
> --------------------
>
> Key: HBASE-25972
> URL: https://issues.apache.org/jira/browse/HBASE-25972
> Project: HBase
> Issue Type: Improvement
> Reporter: Kadir Ozdemir
> Assignee: Kadir Ozdemir
> Priority: Major
> Labels: pull-request-available
> Fix For: 2.7.0, 3.0.0-beta-2
>
>
> HBase stores tables row by row in its files, HFiles. An HFile is composed of
> blocks. The number of rows stored in a block depends on the row sizes. The
> number of rows per block gets lower when rows get larger on disk due to
> multiple row versions since HBase stores all row versions sequentially in the
> same HFile after compaction. However, applications (e.g., Phoenix) mostly
> query the most recent row versions.
> The default compactor in HBase compacts HFiles into one file. This Jira
> introduces a new store file writer which writes the retained cells by
> compaction into two files, which will be called DualFileWriter. One of these
> files will include the live cells. This file will be called a live-version
> file. The other file will include the rest of the cells, that is, historical
> versions. This file will be called a historical-version file. DualFileWriter
> will work with the default compactor.
> The historical files will not be read for the scans scanning latest row
> versions. This eliminates scanning unnecessary cell versions in compacted
> files and thus it is expected to improve performance of these scans.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
