[
https://issues.apache.org/jira/browse/HDFS-12556?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16187347#comment-16187347
]
Surendra Singh Lilhore commented on HDFS-12556:
-----------------------------------------------
Some time {{TestPersistentStoragePolicySatisfier#testWithRestarts}} failing
because of {{ArrayIndexOutOfBoundsException}}. This is happening because when
block movement analysis is going on, same time NN receive block report from one
DN and it will update the block storage list.
{{BlockManager#getStorages(BlockInfo)}} API create internal array {{storages}}
based on current number of node for a block but later number of node is
increased while filling the array and it cause
{{ArrayIndexOutOfBoundsException}}.
{code}
public DatanodeStorageInfo[] getStorages(BlockInfo block) {
final DatanodeStorageInfo[] storages = new
DatanodeStorageInfo[block.numNodes()];
int i = 0;
for(DatanodeStorageInfo s : blocksMap.getStorages(block)) {
storages[i++] = s;
}
return storages;
}
{code}
Block analysis should be done in read lock, so we can avoid in-between block
location storage update.
Attached initial patch. Please review...
> [SPS] : Block movement analysis should be done in read lock.
> ------------------------------------------------------------
>
> Key: HDFS-12556
> URL: https://issues.apache.org/jira/browse/HDFS-12556
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: datanode, namenode
> Reporter: Surendra Singh Lilhore
> Assignee: Surendra Singh Lilhore
> Attachments: HDFS-12556-HDFS-10285-01.patch
>
>
> {noformat}
> 2017-09-27 15:58:32,852 [StoragePolicySatisfier] ERROR
> namenode.StoragePolicySatisfier
> (StoragePolicySatisfier.java:handleException(308)) - StoragePolicySatisfier
> thread received runtime exception. Stopping Storage policy satisfier work
> java.lang.ArrayIndexOutOfBoundsException: 1
> at
> org.apache.hadoop.hdfs.server.blockmanagement.BlockManager.getStorages(BlockManager.java:4130)
> at
> org.apache.hadoop.hdfs.server.namenode.StoragePolicySatisfier.analyseBlocksStorageMovementsAndAssignToDN(StoragePolicySatisfier.java:362)
> at
> org.apache.hadoop.hdfs.server.namenode.StoragePolicySatisfier.run(StoragePolicySatisfier.java:236)
> at java.lang.Thread.run(Thread.java:745)
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
