[
https://issues.apache.org/jira/browse/HADOOP-3799?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12708035#action_12708035
]
dhruba borthakur commented on HADOOP-3799:
------------------------------------------
I would like to figure out how to expose the filename in the API. It can be
easily done when this API is invoked at the time when a new block is being
allocated to a file. However, when the replicator takes a under-replicated
block and tries to get a new replica location for that block, it is costly to
the find the filename.
The filename can be deduced by traversing the INode-tree that is maintained by
the namenode. But it is a costly operation beccause one has to traverse the
entire branch starting from the specified INOde to the root. One option is to
pass in the INode into the block-placement algorithm. f the algorithm needs the
complete path name of the file in question, then it has to do the costly
operation of generating the full path name of the file. However, this makes the
interface kind-of less elegant. I am still debating how to do it right.
> Design a pluggable interface to place replicas of blocks in HDFS
> ----------------------------------------------------------------
>
> Key: HADOOP-3799
> URL: https://issues.apache.org/jira/browse/HADOOP-3799
> Project: Hadoop Core
> Issue Type: Improvement
> Components: dfs
> Reporter: dhruba borthakur
> Assignee: dhruba borthakur
> Attachments: BlockPlacementPluggable.txt
>
>
> The current HDFS code typically places one replica on local rack, the second
> replica on remote random rack and the third replica on a random node of that
> remote rack. This algorithm is baked in the NameNode's code. It would be nice
> to make the block placement algorithm a pluggable interface. This will allow
> experimentation of different placement algorithms based on workloads,
> availability guarantees and failure models.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
