[
https://issues.apache.org/jira/browse/HDFS-1332?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13032123#comment-13032123
]
Todd Lipcon commented on HDFS-1332:
-----------------------------------
I think there's at least one more case worth adding "explanation" to. In
testing this, I tried creating a file when there were no datanodes running:
java.io.IOException: File /user/todd/x2 could only be replicated to 0 nodes,
instead of 1
This is another case that new users run into a lot. It would be great to change
the code in FSNamesystem which throws this exception to check the total number
of available nodes in the system, and if it's < minReplication, include that in
the message. Perhaps something like:
"File /foo/bar could only be replicated to 0 nodes, instead of 1, because there
are 0 datanodes running."
Also, I tried testing this code by putting a file with a very large blocksize
and my DN configured on a small partition. I got the following NPE:
java.io.IOException: java.lang.NullPointerException
at
org.apache.hadoop.hdfs.server.namenode.BlockPlacementPolicyDefault.throwNotEnoughReplicasExceptionWithReason(BlockPlacementPolicyDefault.java:324)
at
org.apache.hadoop.hdfs.server.namenode.BlockPlacementPolicyDefault.chooseRandom(BlockPlacementPolicyDefault.java:367)
at
org.apache.hadoop.hdfs.server.namenode.BlockPlacementPolicyDefault.chooseLocalRack(BlockPlacementPolicyDefault.java:262)
at
org.apache.hadoop.hdfs.server.namenode.BlockPlacementPolicyDefault.chooseLocalNode(BlockPlacementPolicyDefault.java:236)
at
org.apache.hadoop.hdfs.server.namenode.BlockPlacementPolicyDefault.chooseTarget(BlockPlacementPolicyDefault.java:171)
at
org.apache.hadoop.hdfs.server.namenode.BlockPlacementPolicyDefault.chooseTarget(BlockPlacementPolicyDefault.java:139)
at
org.apache.hadoop.hdfs.server.namenode.BlockPlacementPolicyDefault.chooseTarget(BlockPlacementPolicyDefault.java:89)
at
org.apache.hadoop.hdfs.server.namenode.BlockPlacementPolicy.chooseTarget(BlockPlacementPolicy.java:78)
at
org.apache.hadoop.hdfs.server.namenode.BlockPlacementPolicy.chooseTarget(BlockPlacementPolicy.java:234)
at
org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getAdditionalBlock(FSNamesystem.java:1637)
at
org.apache.hadoop.hdfs.server.namenode.NameNode.addBlock(NameNode.java:841)
> When unable to place replicas, BlockPlacementPolicy should log reasons nodes
> were excluded
> ------------------------------------------------------------------------------------------
>
> Key: HDFS-1332
> URL: https://issues.apache.org/jira/browse/HDFS-1332
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: name-node
> Reporter: Todd Lipcon
> Assignee: Ted Yu
> Priority: Minor
> Labels: newbie
> Attachments: HDFS-1332.patch
>
>
> Whenever the block placement policy determines that a node is not a "good
> target" it could add the reason for exclusion to a list, and then when we log
> "Not able to place enough replicas" we could say why each node was refused.
> This would help new users who are having issues on pseudo-distributed (eg
> because their data dir is on /tmp and /tmp is full). Right now it's very
> difficult to figure out the issue.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
