[
https://issues.apache.org/jira/browse/HBASE-26969?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17536730#comment-17536730
]
Duo Zhang commented on HBASE-26969:
-----------------------------------
OK, so when flushing, we will write a mob file, and then added it to the
metadata of the normal file so when reading, we know where to find the files.
And in this way, I guess if we want to compact the mob files, we always need to
compact the normal files which references the mob files so we can update the
references in the metadata?
Then I do not think we need to use SFT to track the mob files.
The mob files should have a different name prefix or under a different
directory? Or at least, reading the header can let us know that it is a MOB
file? When loading, we just collect all the mob refs from all the normal
storefiles, then we know the list of mob files. So at least for loading, there
will be no problem as we will not load the broken MOB files if we write the
files directly to the final directory.
I think the only problem here is how do we clean up the half written mob files,
I think the logic is mainly the same with what we have now, get all the mob
refs from all the normal storefiles, to construct the base list, and then get
all the mob files which are currently being written, all MOB files besides them
are the ones should be deleted.
> Eliminate MOB renames when SFT is enabled
> -----------------------------------------
>
> Key: HBASE-26969
> URL: https://issues.apache.org/jira/browse/HBASE-26969
> Project: HBase
> Issue Type: Sub-task
> Components: mob
> Affects Versions: 2.5.0, 3.0.0-alpha-3
> Reporter: Szabolcs Bukros
> Assignee: Szabolcs Bukros
> Priority: Major
> Fix For: 2.6.0, 3.0.0-alpha-3
>
>
> MOB file compaction and flush still relies on renames even when SFT is
> enabled.
> My proposed changes are:
> * when requireWritingToTmpDirFirst is false during mob flush/compact instead
> of using the temp writer we should create a different writer using a
> {color:#000000}StoreFileWriterCreationTracker that writes directly to the mob
> store folder{color}
> * {color:#000000}these StoreFileWriterCreationTracker should be stored in
> the MobStore. This would requires us to extend MobStore with a createWriter
> and a finalizeWriter method to handle this{color}
> * {color:#000000}refactor {color}MobFileCleanerChore to run on the RS
> instead on Master to allow access to the
> {color:#000000}StoreFileWriterCreationTracker{color}s to make sure the
> currently written files are not cleaned up
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
