[
https://issues.apache.org/jira/browse/HDFS-5183?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14286183#comment-14286183
]
Arpit Agarwal commented on HDFS-5183:
-------------------------------------
Also thanks for filing this and the suggested solutions [~djp] and [~sirianni].
This is another of the issues that got pushed out as part of Heterogeneous
Storages phase-2 work and was later covered under the Archival Storage feature.
> Combine ReplicaPlacementPolicy with VolumeChoosingPolicy together to have a
> global view in choosing DN storage for replica.
> ---------------------------------------------------------------------------------------------------------------------------
>
> Key: HDFS-5183
> URL: https://issues.apache.org/jira/browse/HDFS-5183
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: datanode, namenode, performance
> Affects Versions: Heterogeneous Storage (HDFS-2832)
> Reporter: Junping Du
>
> Per discussion in HDFS-5157, There are two different ways to handle
> BlockPlacementPolicy and ReplicaChoosingPolicy in case of multiple storage
> types:
> 1. Client specifies the required storage type when calling addBlock(..) to
> NN. BlockPlacementPolicy in NN chooses a set of datanodes accounting for the
> storage type. Then, client passes the required storage type to the datanode
> set and each datanode chooses a particular storage using a
> VolumeChoosingPolicy.
> 2. Same as before, client specifies the required storage type when calling
> addBlock(..) to NN. Now, BlockPlacementPolicy in NN chooses a set of storages
> (instead of datanodes). Then, client writes to the corresponding storages.
> VolumeChoosingPolicy is no longer needed and it should be removed.
> We think #2 is more powerful as it will bring global view to volume choosing
> or bring storage status into consideration in replica choosing, so we propose
> to combine two polices together.
> One concern here is it may increase the load of NameNode as previously volume
> choosing is decided by DN. We may verify it later (that's why I put
> performance in component).
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
