He Xiaoqiao created HDFS-13529:
----------------------------------
Summary: Fix default trash policy emptier trigger time correctly
Key: HDFS-13529
URL: https://issues.apache.org/jira/browse/HDFS-13529
Project: Hadoop HDFS
Issue Type: Bug
Components: namenode
Affects Versions: 2.7.6, 3.1.0, 3.2.0, 2.9.2, 2.8.5
Reporter: He Xiaoqiao
Trash emptier is designed to auto trigger at UTC 00:00, however I am confused
all the time that it usually triggers at a few minutes even half a hour after
UTC 00:00 actually in our production cluster.
The main reason is default policy emptier thread sleep more time than as
expect, since it does not consider the delete operation time cost itself.
especially for a large cluster, auto trash cleaner may cost dozens of minutes.
The right way is that gets current time {{now}} before calculate {{end}} time.
{code:java}
long now = Time.now();
while (true) {
end = ceiling(now, emptierInterval);
try { // sleep for interval
Thread.sleep(end - now);
} catch (InterruptedException e) {
break; // exit on interrupt
}
try {
now = Time.now();
...... // delete trash checkpoint
} catch (Exception e) {
LOG.warn("RuntimeException during Trash.Emptier.run(): ", e);
}
}
{code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
