[
https://issues.apache.org/jira/browse/HDFS-3194?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13286337#comment-13286337
]
amith commented on HDFS-3194:
-----------------------------
Hi Eli
Thanks for looking into this issue
This issue may be induced due to some federation related changes (I don't know
which)
{quote}I think the current behavior (rolling the log on each period) is
intended. I think the bug is that we're starting a new period way too
frequently (every 5 seconds instead of every three weeks by default).
DataBlockScanner#run calls scanBlockPoolSlice every 5 seconds{quote}
I feel that code will be modified some thing like
{code}
//Sleep everytime except in the first interation.
if (!firstRun) {
try {
Thread.sleep(21*24*3600);
} catch (InterruptedException ex) {
// Interrupt itself again to set the interrupt status
blockScannerThread.interrupt();
continue;
}
{code}
If I change the code like this then there is a possibly we will not be able to
do incremental block scan
eg: I will write 10 blocks @time 14:53:46
now block scanner has scanned the blocks
I wrote another set of blocks in 15:53:46 then this blocks will not be scanned
by block scanner for next 3 weeks.
Now in my change this is same as Hadoop 0.20.X behavior (Even new period is
started logs are rolled only after reaching certain limit).
where logs will be rolled only after certain number of blocks are reached in
the logs.
Please correct me if I am wrong :)
Thanks
Amith
> DataNode block scanner is running too frequently
> ------------------------------------------------
>
> Key: HDFS-3194
> URL: https://issues.apache.org/jira/browse/HDFS-3194
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: data-node
> Affects Versions: 2.0.0-alpha
> Reporter: suja s
> Assignee: amith
> Attachments: HDFS-3194.patch, HDFS-3194_1.patch, HDFS-3194_2.patch,
> HDFS-3194_4.patch, HDFS-3194_6.patch
>
>
> Block scanning interval by default should be taken as 21 days(3 weeks) and
> each block scanning should happen once in 21 days.
> Here the block is being scanned continuosly.
> 2012-04-03 10:44:47,056 INFO
> org.apache.hadoop.hdfs.server.datanode.BlockPoolSliceScanner: Verification
> succeeded for
> BP-241703115-xx.xx.xx.55-1333086229434:blk_-2666054955039014473_1003
> 2012-04-03 10:45:02,064 INFO
> org.apache.hadoop.hdfs.server.datanode.BlockPoolSliceScanner: Verification
> succeeded for
> BP-241703115-xx.xx.xx.55-1333086229434:blk_-2666054955039014473_1003
> 2012-04-03 10:45:17,071 INFO
> org.apache.hadoop.hdfs.server.datanode.BlockPoolSliceScanner: Verification
> succeeded for
> BP-241703115-xx.xx.xx.55-1333086229434:blk_-2666054955039014473_1003
> 2012-04-03 10:45:32,079 INFO
> org.apache.hadoop.hdfs.server.datanode.BlockPoolSliceScanner: Verification
> succeeded for BP-241703115-xx.xx.xx.55-1333086229434:blk_-2666054955039014473
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
