[
https://issues.apache.org/jira/browse/HDFS-1655?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13001026#comment-13001026
]
Jitendra Nath Pandey commented on HDFS-1655:
--------------------------------------------
Changes in the patch.
- The block pools are scanned one at a time. Most of the original
DataBlockScanner functionality is moved to BlockPoolScanner. The
DataBlockScanner runs a scanner thread as before and selects next block pool to
scan and invokes BlockPoolScanner to do the scan work.
- One BlockPoolScanner is instantiated for each block pool which maintains the
state of the scan, blockMap and blockInfoSet similar to what DataBlockScanner
had before.
- The current verification log file is created for each block pool and once the
block pool scan is complete it is rolled into the previous log file, before
processing the next block pool. Therefore, at any instant there should be only
one current verification log file.
- To find the next block pool to scan, the datablock scanner iterates over the
set of blockPools. However, if a datanode is restarted it needs to pick the
block pool which was being scanned before. Therefore, first it looks for the
block pool that contains the current verification log file and that is used as
the starting block pool id. If no current files are found it starts with first
block-pool.
> Federation: DatablockScanner should scan blocks for all the block pools.
> ------------------------------------------------------------------------
>
> Key: HDFS-1655
> URL: https://issues.apache.org/jira/browse/HDFS-1655
> Project: Hadoop HDFS
> Issue Type: New Feature
> Affects Versions: Federation Branch
> Reporter: Jitendra Nath Pandey
> Assignee: Jitendra Nath Pandey
> Fix For: Federation Branch
>
> Attachments: HDFS-1655.1.patch
>
>
> DataBlockScanner changes are needed to work with federation.
> Goal is to have DataBlockScanner visit one volume at a time, scanning block
> pools under it one at a time.
--
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
