[
https://issues.apache.org/jira/browse/HDFS-9918?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15215796#comment-15215796
]
Walter Su commented on HDFS-9918:
---------------------------------
We also need to sort locations by distance. It's unlikely but from 2 duplicated
in-service blocks we want to choose the nearer one. The logic is like
{{sortLocatedBlocks(..)}}, how about reuse it?
We move {{BlockIndex}} & {{BlockToken}} according to {{Location}}. Like
{code}
//public void sortLocatedBlocks(final String targethost,
for (LocatedBlock b : locatedblocks) {
DatanodeInfo[] di = b.getLocations();
+ HashMap<DatanodeInfo, Byte> locToIndex = null;
+ HashMap<DatanodeInfo, Token<BlockTokenIdentifier>> locToToken = null;
+ if(b instanceof LocatedStripedBlock){
+ locToIndex = new HashMap<>();
+ locToToken = new HashMap<>();
+ LocatedStripedBlock lb = (LocatedStripedBlock) b;
+ for(int i=0; i<di.length; i++){
+ locToIndex.put(di[i], lb.getBlockIndices()[i]);
+ locToToken.put(di[i], lb.getBlockTokens()[i]);
+ }
+ }
// Move decommissioned/stale datanodes to the bottom
...
networktopology.sortByDistance(client, b.getLocations(), activeLen);
// must update cache since we modified locations array
b.updateCachedStorageInfo();
+
+ if(b instanceof LocatedStripedBlock){
+ LocatedStripedBlock lb = (LocatedStripedBlock) b;
+ for(int i =0 ;i<di.length;i++){
+ lb.getBlockIndices()[i] = locToIndex.get(di[i]);
+ lb.getBlockTokens()[i] = locToToken.get(di[i]);
+ }
+ }
{code}
It passes your tests.
> Erasure Coding: Sort located striped blocks based on decommissioned states
> --------------------------------------------------------------------------
>
> Key: HDFS-9918
> URL: https://issues.apache.org/jira/browse/HDFS-9918
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Reporter: Rakesh R
> Assignee: Rakesh R
> Attachments: HDFS-9918-001.patch, HDFS-9918-002.patch,
> HDFS-9918-003.patch, HDFS-9918-004.patch, HDFS-9918-005.patch,
> HDFS-9918-006.patch
>
>
> This jira is a follow-on work of HDFS-8786, where we do decommissioning of
> datanodes having striped blocks.
> Now, after decommissioning it requires to change the ordering of the storage
> list so that the decommissioned datanodes should only be last node in list.
> For example, assume we have a block group with storage list:-
> d0, d1, d2, d3, d4, d5, d6, d7, d8, d9
> mapping to indices
> 0, 1, 2, 3, 4, 5, 6, 7, 8, 2
> Here the internal block b2 is duplicated, locating in d2 and d9. If d2 is a
> decommissioning node then should switch d2 and d9 in the storage list.
> Thanks [~jingzhao] for the
> [discussions|https://issues.apache.org/jira/browse/HDFS-8786?focusedCommentId=15180415&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-15180415]
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
