Liu Shaohui created HBASE-8690:
----------------------------------
Summary: Reduce unnecessary getFileStatus hdfs calls in TTL hfile
and hlog cleanners
Key: HBASE-8690
URL: https://issues.apache.org/jira/browse/HBASE-8690
Project: HBase
Issue Type: Improvement
Components: master
Reporter: Liu Shaohui
Priority: Minor
For each in file in archive dir, the TimeToLiveHFileCleaner need call
getFileStatus to get the modify time of file. Actually the CleanerChore have
had the file status when listing its parent dir.
When we set the TTL to 7 days in our cluster for data security, the number of
files left in archive dir is up to 65 thousands. In each clean period,
TimeToLiveHFileCleaner will generate ten thousand getFileStatus call in a short
time, which is very heavy for hdfs namenode.
Fix: Change the path param to FileStatus in isFileDeletable method and reduce
unnecessary getFileStatus hdfs calls in TTL cleaners.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
