[
https://issues.apache.org/jira/browse/HDFS-3194?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13399085#comment-13399085
]
Andy Isaacson commented on HDFS-3194:
-------------------------------------
I don't understand why we call startNewPeriod in scanBlockPoolSlice.
startNewPeriod should be called exactly once per 3-week period.
scanBlockPoolSlice gets called every 5 seconds by DataBlockScanner.run(). So
we should not be starting a new period when we scan a pool slice.
So I think this trivial patch fixes the main issue:
{code}
---
a/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/BlockPoolSliceScanner.java
+++
b/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/BlockPoolSliceScanner.java
@@ -560,7 +560,6 @@ class BlockPoolSliceScanner {
}
void scanBlockPoolSlice() {
- startNewPeriod();
// Create a new processedBlocks structure
processedBlocks = new HashMap<Long, Integer>();
if (!assignInitialVerificationTimes()) {
{code}
There may be another issue that's being addressed in HDFS-3194_7.patch but that
patch does not appear to address the core issue that we're restarting the
period far too frequently. I don't understand what problem is being addressed
in _7 though, so maybe I'm missing something.
Amith, does my patch look correct to you?
> DataNode block scanner is running too frequently
> ------------------------------------------------
>
> Key: HDFS-3194
> URL: https://issues.apache.org/jira/browse/HDFS-3194
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: data-node
> Affects Versions: 2.0.0-alpha
> Reporter: suja s
> Assignee: amith
> Attachments: HDFS-3194.patch, HDFS-3194_1.patch, HDFS-3194_2.patch,
> HDFS-3194_4.patch, HDFS-3194_6.patch, HDFS-3194_7.patch
>
>
> Block scanning interval by default should be taken as 21 days(3 weeks) and
> each block scanning should happen once in 21 days.
> Here the block is being scanned continuosly.
> 2012-04-03 10:44:47,056 INFO
> org.apache.hadoop.hdfs.server.datanode.BlockPoolSliceScanner: Verification
> succeeded for
> BP-241703115-xx.xx.xx.55-1333086229434:blk_-2666054955039014473_1003
> 2012-04-03 10:45:02,064 INFO
> org.apache.hadoop.hdfs.server.datanode.BlockPoolSliceScanner: Verification
> succeeded for
> BP-241703115-xx.xx.xx.55-1333086229434:blk_-2666054955039014473_1003
> 2012-04-03 10:45:17,071 INFO
> org.apache.hadoop.hdfs.server.datanode.BlockPoolSliceScanner: Verification
> succeeded for
> BP-241703115-xx.xx.xx.55-1333086229434:blk_-2666054955039014473_1003
> 2012-04-03 10:45:32,079 INFO
> org.apache.hadoop.hdfs.server.datanode.BlockPoolSliceScanner: Verification
> succeeded for BP-241703115-xx.xx.xx.55-1333086229434:blk_-2666054955039014473
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
