[
https://issues.apache.org/jira/browse/HBASE-8690?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13678746#comment-13678746
]
Hudson commented on HBASE-8690:
-------------------------------
Integrated in HBase-TRUNK-on-Hadoop-2.0.0 #559 (See
[https://builds.apache.org/job/HBase-TRUNK-on-Hadoop-2.0.0/559/])
HBASE-8690 Reduce unnecessary getFileStatus hdfs calls in TTL hfile and
hlog cleanners (Shaohui) (Revision 1490886)
Result = FAILURE
tedyu :
Files :
*
/hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/backup/example/LongTermArchivingHFileCleaner.java
*
/hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/master/cleaner/BaseLogCleanerDelegate.java
*
/hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/master/cleaner/CleanerChore.java
*
/hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/master/cleaner/FileCleanerDelegate.java
*
/hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/master/cleaner/HFileLinkCleaner.java
*
/hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/master/cleaner/TimeToLiveHFileCleaner.java
*
/hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/master/cleaner/TimeToLiveLogCleaner.java
*
/hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/master/snapshot/SnapshotHFileCleaner.java
*
/hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/master/snapshot/SnapshotLogCleaner.java
*
/hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/replication/master/ReplicationLogCleaner.java
*
/hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/backup/example/TestZooKeeperTableArchiveClient.java
*
/hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/master/cleaner/TestCleanerChore.java
*
/hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/master/cleaner/TestHFileCleaner.java
*
/hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/master/snapshot/TestSnapshotHFileCleaner.java
*
/hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/master/snapshot/TestSnapshotLogCleaner.java
> Reduce unnecessary getFileStatus hdfs calls in TTL hfile and hlog cleanners
> ---------------------------------------------------------------------------
>
> Key: HBASE-8690
> URL: https://issues.apache.org/jira/browse/HBASE-8690
> Project: HBase
> Issue Type: Improvement
> Components: master
> Reporter: Liu Shaohui
> Assignee: Liu Shaohui
> Priority: Minor
> Attachments: 8690-trunk-v3.patch, 8690-trunk-v4.patch,
> HBASE-8690-0.94-v1.patch, HBASE-8690-trunk-v1.patch, HBASE-8690-trunk-v2.patch
>
>
> For each in file in archive dir, the TimeToLiveHFileCleaner need call
> getFileStatus to get the modify time of file. Actually the CleanerChore have
> had the file status when listing its parent dir.
> When we set the TTL to 7 days in our cluster for data security, the number of
> files left in archive dir is up to 65 thousands. In each clean period,
> TimeToLiveHFileCleaner will generate ten thousand getFileStatus call in a
> short time, which is very heavy for hdfs namenode.
> Fix: Change the path param to FileStatus in isFileDeletable method and reduce
> unnecessary getFileStatus hdfs calls in TTL cleaners.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
