[
https://issues.apache.org/jira/browse/HBASE-30499?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Hernan Romer reassigned HBASE-30499:
------------------------------------
Assignee: Hernan Romer
> A bypassed SnapshotProcedure leaves its tmp working directory behind, which
> pins archived HFiles until the HMaster restarts
> ---------------------------------------------------------------------------------------------------------------------------
>
> Key: HBASE-30499
> URL: https://issues.apache.org/jira/browse/HBASE-30499
> Project: HBase
> Issue Type: Improvement
> Components: snapshots
> Affects Versions: 2.0.0, 2.6.0, 3.0.0
> Reporter: Hernan Romer
> Assignee: Hernan Romer
> Priority: Major
>
> Bypassing a SnapshotProcedure before SNAPSHOT_COMPLETE_SNAPSHOT skips both
> the commit and the rollback. The directory <snapshot dir>/.tmp/<snapshot
> name> stays on disk, and nothing in the running master tracks it any more.
> list_snapshots and list_procedures don't show it, and deleteSnapshot doesn't
> remove it. The only cleanup is SnapshotManager#resetTempDir(), which only
> runs when the master starts.
> Until then, SnapshotFileCache#getSnapshotsInProgress() treats the directory
> as a live snapshot and keeps every HFile its manifest references. As the
> table compacts, those files pile up in the archive.
> Consequence: the archive can grow to roughly the size the table had when the
> procedure was bypassed, and stays there until the master restarts. The only
> hint is CleanerChore TRACE logging not deletable according to:
> SnapshotHFileCleaner, which doesn't name the snapshot. We hit this in
> production: the master stayed up for about a month after a bypass, and the
> archive grew to about the size of the table's live data.
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)
