[
https://issues.apache.org/jira/browse/HBASE-26826?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrew Kyle Purtell updated HBASE-26826:
----------------------------------------
Description:
In a discussion on dev@ the idea was floated that StoreFileTracker could be
backported into branch-2.5 to be released as part of 2.5.0 as an experimental
feature. This issue considers the backport.
There are sixteen subtasks on HBASE-26067 and several other tangential commits.
Cherry pick the list in sequence, fixing up as necessary. These appear to be
the core commits:
- commit 6aaef8978 HBASE-26064 Introduce a StoreFileTracker to abstract the
store file tracking logic
- commit 43b40e937 HBASE-25988 Store the store file list by a file (#3578)
- commit 6e053765e HBASE-26079 Use StoreFileTracker when splitting and merging
(#3617)
- commit 090b2fecf HBASE-26224 Introduce a MigrationStoreFileTracker to support
migrating from different store file tracker implementations (#3656)
- commit 0ee168933 HBASE-26246 Persist the StoreFileTracker configurations to
TableDescriptor when creating table (#3666)
- commit 2052e80e5 HBASE-26248 Should find a suitable way to let users specify
the store file tracker implementation (#3665)
- commit 5ff0f98a5 HBASE-26264 Add more checks to prevent misconfiguration on
store file tracker (#3681)
- commit fc4f6d10e HBASE-26280 Use store file tracker when snapshoting (#3685)
- commit 06db852aa HBASE-26326 CreateTableProcedure fails when
FileBasedStoreFileTracker… (#3721)
- commit e4e7cf80b HBASE-26386 Refactor StoreFileTracker implementations to
expose the set method (#3774)
- commit 08d117197 HBASE-26328 Clone snapshot doesn't load reference files into
FILE SFT impl (#3749)
- commit 8bec26ea9 HBASE-26263 [Rolling Upgrading] Persist the StoreFileTracker
configurations to TableDescriptor for existing tables (#3700)
- commit a288365f9 HBASE-26271 Cleanup the broken store files under data
directory (#3786)
- commit d00b5faad HBASE-26454 CreateTableProcedure still relies on temp dir
and renames… (#3845)
- commit 771e552cf HBASE-26286: Add support for specifying store file tracker
when restoring or cloning snapshot
- commit f16b7b1bf HBASE-26265 Update ref guide to mention the new store file
tracker im… (#3942)
And from HBASE-26584 and beyond:
- commit 755b3b4cb HBASE-26585 Add SFT configuration to META table descriptor
when creating META (#3998)
- commit 39c42c7dc HBASE-26639 The implementation of
TestMergesSplitsAddToTracker is problematic (#4010)
- commit 6e1f5b7fe HBASE-26586 Should not rely on the global config when
setting SFT implementation for a table while upgrading (#4006)
- commit f1dd865c3 HBASE-26654 ModifyTableDescriptorProcedure shoud load
TableDescriptor while executing (#4034)
- commit 8fbc9a260 HBASE-26674 Should modify filesCompacting under
storeWriteLock (#4040)
- commit 5aa0fd265 HBASE-26675 Data race on Compactor.writer (#4035)
- commit 3021c5851 HBASE-26700 The way we bypass broken track file is not
enough in StoreFileListFile (#4055)
- commit a8b68c9b8 HBASE-26690 Modify FSTableDescriptors to not rely on
renaming when writing TableDescriptor (#4054)
- commit dffeb8e63 HBASE-26587 Introduce a new Admin API to change SFT
implementation (#4030) (#4080)
- commit b265fe55b HBASE-26673 Implement a shell command for change SFT
implementation (#4113)
was:
In a discussion on dev@ the idea was floated that StoreFileTracker could be
backported into branch-2.5 to be released as part of 2.5.0 as an experimental
feature. This issue considers the backport.
There are sixteen subtasks on HBASE-26067 and several other tangential commits.
Cherry pick the list in sequence, fixing up as necessary. These appear to be
the core commits:
- commit 6aaef8978 HBASE-26064 Introduce a StoreFileTracker to abstract the
store file tracking logic
- commit 43b40e937 HBASE-25988 Store the store file list by a file (#3578)
- commit 6e053765e HBASE-26079 Use StoreFileTracker when splitting and merging
(#3617)
- commit 090b2fecf HBASE-26224 Introduce a MigrationStoreFileTracker to support
migrating from different store file tracker implementations (#3656)
- commit 0ee168933 HBASE-26246 Persist the StoreFileTracker configurations to
TableDescriptor when creating table (#3666)
- commit 2052e80e5 HBASE-26248 Should find a suitable way to let users specify
the store file tracker implementation (#3665)
- commit 5ff0f98a5 HBASE-26264 Add more checks to prevent misconfiguration on
store file tracker (#3681)
- commit fc4f6d10e HBASE-26280 Use store file tracker when snapshoting (#3685)
- commit 06db852aa HBASE-26326 CreateTableProcedure fails when
FileBasedStoreFileTracker… (#3721)
- commit e4e7cf80b HBASE-26386 Refactor StoreFileTracker implementations to
expose the set method (#3774)
- commit 08d117197 HBASE-26328 Clone snapshot doesn't load reference files into
FILE SFT impl (#3749)
- commit 8bec26ea9 HBASE-26263 [Rolling Upgrading] Persist the StoreFileTracker
configurations to TableDescriptor for existing tables (#3700)
- commit a288365f9 HBASE-26271 Cleanup the broken store files under data
directory (#3786)
- commit d00b5faad HBASE-26454 CreateTableProcedure still relies on temp dir
and renames… (#3845)
- commit 771e552cf HBASE-26286: Add support for specifying store file tracker
when restoring or cloning snapshot
- commit f16b7b1bf HBASE-26265 Update ref guide to mention the new store file
tracker im… (#3942)
And from HBASE-26584 and beyond:
- commit 755b3b4cb HBASE-26585 Add SFT configuration to META table descriptor
when creating META (#3998)
- commit 39c42c7dc HBASE-26639 The implementation of
TestMergesSplitsAddToTracker is problematic (#4010)
- commit 6e1f5b7fe HBASE-26586 Should not rely on the global config when
setting SFT implementation for a table while upgrading (#4006)
- commit f1dd865c3 HBASE-26654 ModifyTableDescriptorProcedure shoud load
TableDescriptor while executing (#4034)
- commit 8fbc9a260 HBASE-26674 Should modify filesCompacting under
storeWriteLock (#4040)
- commit 5aa0fd265 HBASE-26675 Data race on Compactor.writer (#4035)
- commit 3021c5851 HBASE-26700 The way we bypass broken track file is not
enough in StoreFileListFile (#4055)
- commit a8b68c9b8 HBASE-26690 Modify FSTableDescriptors to not rely on
renaming when writing TableDescriptor (#4054)
- commit dffeb8e63 HBASE-26587 Introduce a new Admin API to change SFT
implementation (#4030) (#4080)
> Backport StoreFileTracker (HBASE-26067, HBASE-26584, and others) to branch-2.5
> ------------------------------------------------------------------------------
>
> Key: HBASE-26826
> URL: https://issues.apache.org/jira/browse/HBASE-26826
> Project: HBase
> Issue Type: Task
> Components: Operability, regionserver
> Reporter: Andrew Kyle Purtell
> Assignee: Andrew Kyle Purtell
> Priority: Major
> Fix For: 2.5.0
>
>
> In a discussion on dev@ the idea was floated that StoreFileTracker could be
> backported into branch-2.5 to be released as part of 2.5.0 as an experimental
> feature. This issue considers the backport.
> There are sixteen subtasks on HBASE-26067 and several other tangential
> commits. Cherry pick the list in sequence, fixing up as necessary. These
> appear to be the core commits:
> - commit 6aaef8978 HBASE-26064 Introduce a StoreFileTracker to abstract the
> store file tracking logic
> - commit 43b40e937 HBASE-25988 Store the store file list by a file (#3578)
> - commit 6e053765e HBASE-26079 Use StoreFileTracker when splitting and
> merging (#3617)
> - commit 090b2fecf HBASE-26224 Introduce a MigrationStoreFileTracker to
> support migrating from different store file tracker implementations (#3656)
> - commit 0ee168933 HBASE-26246 Persist the StoreFileTracker configurations to
> TableDescriptor when creating table (#3666)
> - commit 2052e80e5 HBASE-26248 Should find a suitable way to let users
> specify the store file tracker implementation (#3665)
> - commit 5ff0f98a5 HBASE-26264 Add more checks to prevent misconfiguration on
> store file tracker (#3681)
> - commit fc4f6d10e HBASE-26280 Use store file tracker when snapshoting (#3685)
> - commit 06db852aa HBASE-26326 CreateTableProcedure fails when
> FileBasedStoreFileTracker… (#3721)
> - commit e4e7cf80b HBASE-26386 Refactor StoreFileTracker implementations to
> expose the set method (#3774)
> - commit 08d117197 HBASE-26328 Clone snapshot doesn't load reference files
> into FILE SFT impl (#3749)
> - commit 8bec26ea9 HBASE-26263 [Rolling Upgrading] Persist the
> StoreFileTracker configurations to TableDescriptor for existing tables (#3700)
> - commit a288365f9 HBASE-26271 Cleanup the broken store files under data
> directory (#3786)
> - commit d00b5faad HBASE-26454 CreateTableProcedure still relies on temp dir
> and renames… (#3845)
> - commit 771e552cf HBASE-26286: Add support for specifying store file tracker
> when restoring or cloning snapshot
> - commit f16b7b1bf HBASE-26265 Update ref guide to mention the new store file
> tracker im… (#3942)
> And from HBASE-26584 and beyond:
> - commit 755b3b4cb HBASE-26585 Add SFT configuration to META table descriptor
> when creating META (#3998)
> - commit 39c42c7dc HBASE-26639 The implementation of
> TestMergesSplitsAddToTracker is problematic (#4010)
> - commit 6e1f5b7fe HBASE-26586 Should not rely on the global config when
> setting SFT implementation for a table while upgrading (#4006)
> - commit f1dd865c3 HBASE-26654 ModifyTableDescriptorProcedure shoud load
> TableDescriptor while executing (#4034)
> - commit 8fbc9a260 HBASE-26674 Should modify filesCompacting under
> storeWriteLock (#4040)
> - commit 5aa0fd265 HBASE-26675 Data race on Compactor.writer (#4035)
> - commit 3021c5851 HBASE-26700 The way we bypass broken track file is not
> enough in StoreFileListFile (#4055)
> - commit a8b68c9b8 HBASE-26690 Modify FSTableDescriptors to not rely on
> renaming when writing TableDescriptor (#4054)
> - commit dffeb8e63 HBASE-26587 Introduce a new Admin API to change SFT
> implementation (#4030) (#4080)
> - commit b265fe55b HBASE-26673 Implement a shell command for change SFT
> implementation (#4113)
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
