Ashu Pachauri created HBASE-18399:
-------------------------------------
Summary: Files in a snapshot can go missing even after the
snapshot is taken successfully
Key: HBASE-18399
URL: https://issues.apache.org/jira/browse/HBASE-18399
Project: HBase
Issue Type: Sub-task
Reporter: Ashu Pachauri
Files missing after the snapshot is taken (only applicable when the TTL for the
TimeToLiveHFileCleaner is small, like the default 5 mins)
* SnapshotManifest#addRegion visits store_file_A, but is yet to write it to
the manifest.
* store_file_A is marked as compacted away and HFileArchiver moves the file
to archive.
* HFileCleaner comes in and sees the store_file_A in archive. It adds the
file to the list of files that might need to be cleaned up.
* HFileCleaner's SnapshotHFileCleaner plugin is kicked in.
* SnapshotFileCache#getUnreferencedFiles also says that store_file_A is
unreferenced and should be cleaned up (It has not yet been written to the
manifest).
* SnapshotHFileCleaner is still going through rest of the files in archive.
* store_file_A reference is created and written to snapshot manifest.
* Snapshot verification runs and sees the store_file_A is present in
archive, and thus the verification passes.
* Now, the SnapshotHFileCleaner finishes and TimeToLiveHFileCleaner is
triggered. If TTL has passed since the store_file_A was moved to archive
(SnapshotHFileCleaner could take easily several minutes to go through rest of
the files), the TimeToLiveHFileCleaner also marks the file as deletable.
* Since all cleaner plugins marked file as deletable, the store_file_A is
deleted.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
