[
https://issues.apache.org/jira/browse/HDFS-1245?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13551757#comment-13551757
]
Konstantin Shvachko commented on HDFS-1245:
-------------------------------------------
I would like to resurrect this jira.
Now that we have {{GenerationStamp}} and {{INodeId}} generators, it would be
logical to wrap block id generation into a class.
I plan to introduce BlockIdGenerator, which generates random ids.
Making it pluggable will depend on whether anybody wants to build another
generator.
It may make sense to implement Dhruba's suggestion in HDFS-898 to let people
use random generator on old clusters, and format new ones with the sequential
block id generator. Old clusters can also go through the procedure I described
in HDFS-898, which can be achieved by one-time image processing to find the
biggest hole between existing random block ids, and then switch to sequential
generation within that hole.
> Plugable block id generation
> -----------------------------
>
> Key: HDFS-1245
> URL: https://issues.apache.org/jira/browse/HDFS-1245
> Project: Hadoop HDFS
> Issue Type: New Feature
> Components: namenode
> Reporter: Dmytro Molkov
> Assignee: Dmytro Molkov
>
> The idea is to have a way to easily create block id generation engines that
> may fit a certain purpose. One of them could be HDFS-898 started by
> Konstantin, but potentially others.
> We chatted with Dhruba about this for a while and came up with the following
> approach:
> There should be a BlockIDGenerator interface that has following methods:
> void blockAdded(Block)
> void blockRemoved(Block)
> Block nextBlock()
> First two methods are needed for block generation engines that hold a certain
> state. During the restart, when namenode reads the fsimage it will notify
> generator about all the blocks it reads from the image and during runtime
> namenode will notify the generator about block removals on file deletion.
> The instance of the generator will also have a reference to the block
> registry, the interface that BlockManager implements. The only method there
> is __blockExists(Block)__, so that the current random block id generation can
> be implemented, since it needs to check with the block manager if the id is
> already present.
> What does the community think about this proposal?
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
