tomscut opened a new pull request, #6980:
URL: https://github.com/apache/hadoop/pull/6980
<!--
Thanks for sending a pull request!
1. If this is your first time, please read our contributor guidelines:
https://cwiki.apache.org/confluence/display/HADOOP/How+To+Contribute
2. Make sure your PR title starts with JIRA issue id, e.g.,
'HADOOP-17799. Your PR title ...'.
-->
### Description of PR
JIRA: [HDFS-16557](https://issues.apache.org/jira/browse/HDFS-16557).
We set the EC policy to (6+3) and also have nodes that were in state
`ENTERING_MAINTENANCE`.
When we move the data of some directories from SSD to HDD, some blocks move
fail due to disk full, as shown in the figure below (blk_-9223372033441574269).
We tried to move again and found the following error "`Replica does not
exist`".
Observing the information of fsck, it can be found that the wrong
blockid(blk_-9223372033441574270) was found when moving block.
**Mover Logs**:
**FSCK Info**:
**Root Cause**:
Similar to this
[HDFS-16333](https://issues.apache.org/jira/browse/HDFS-16333), when mover is
initialized, only the `LIVE` node is processed. As a result, the datanode in
the `ENTERING_MAINTENANCE` state in the locations is filtered when initializing
`DBlockStriped`, but the indices are not adapted, resulting in a mismatch
between the location and indices lengths. Finally, ec block calculates the
wrong blockid when getting internal block (see
`DBlockStriped#getInternalBlock`).
**Solution**:
When initializing `DBlockStriped`, if any location is filtered out, we need
to remove the corresponding element in the indices to do the adaptation.
### How was this patch tested?
Pass the unit test.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
