[
https://issues.apache.org/jira/browse/HDFS-9918?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15218965#comment-15218965
]
Zhe Zhang commented on HDFS-9918:
---------------------------------
In my original comment when I said preserving index ordering, I meant something
like below (using Rakesh's example):
bq. Assume in the below list 0, 1, 7, 8 is decommissioned and extra redundant
index is 1.
bq. 0, 1', 2, 3, 4, 5, 6, 7', 8', 0', 1, 7, 8, 1
In the above, I marked decommissioning internal blocks with a {{'}}. I think we
should order the list as:
{code}
0, 1, 2, 3, 4, 5, 6, 7, 8, 1, 0', 1', 7', 8'
{code}
The main motivation is to enable future optimization HDFS-8032. So actually I
think we should do a decomm-based sorting within a block index, and only "swap"
2 internal blocks with the same block index.
In the v8 patch, I think we can use a {{DECOM_COMPARATOR}}-based sort on each
list in {{blkIndex2LocationsMap}}. Then for the first {{numDataUnits}} slots in
the result, we take the first location from the index. From sorting priority's
perspective, I think a decommissioning internal block is still more useful then
a redundant internal block.
LMK if you guys agree.
> Erasure Coding: Sort located striped blocks based on decommissioned states
> --------------------------------------------------------------------------
>
> Key: HDFS-9918
> URL: https://issues.apache.org/jira/browse/HDFS-9918
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Reporter: Rakesh R
> Assignee: Rakesh R
> Attachments: HDFS-9918-001.patch, HDFS-9918-002.patch,
> HDFS-9918-003.patch, HDFS-9918-004.patch, HDFS-9918-005.patch,
> HDFS-9918-006.patch, HDFS-9918-007.patch, HDFS-9918-008.patch
>
>
> This jira is a follow-on work of HDFS-8786, where we do decommissioning of
> datanodes having striped blocks.
> Now, after decommissioning it requires to change the ordering of the storage
> list so that the decommissioned datanodes should only be last node in list.
> For example, assume we have a block group with storage list:-
> d0, d1, d2, d3, d4, d5, d6, d7, d8, d9
> mapping to indices
> 0, 1, 2, 3, 4, 5, 6, 7, 8, 2
> Here the internal block b2 is duplicated, locating in d2 and d9. If d2 is a
> decommissioning node then should switch d2 and d9 in the storage list.
> Thanks [~jingzhao] for the
> [discussions|https://issues.apache.org/jira/browse/HDFS-8786?focusedCommentId=15180415&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-15180415]
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
