Hernan Romer created HBASE-30291:
------------------------------------
Summary: BackupHFileCleaner scans backups system table redundantly
for every region, causing excessive I/O and archive buildup
Key: HBASE-30291
URL: https://issues.apache.org/jira/browse/HBASE-30291
Project: HBase
Issue Type: Bug
Components: backup&restore
Reporter: Hernan Romer
The BackupHFileCleaner scans the backups system table once per call to
getDeletableFiles, which is invoked separately for every region in the cluster.
On clusters with many regions (~40k), this results in tens of thousands of
redundant full scans of the same table in a single cleaner cycle. This causes
the cleaner to take so long that it misses its scheduled start time and runs
continuously, generating sustained high read request volume against the host
holding the system tables.
Because the cleaner can't keep up, archived HFiles accumulate on disk, leading
to unexpected spikes in disk utilization (40% increase observed in one case).
The problem is exacerbated on clusters that do frequent bulk loads, since
compactions triggered by those bulk loads produce a large volume of archived
HFiles that the cleaner needs to process.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
