[
https://issues.apache.org/jira/browse/HDFS-12567?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16193968#comment-16193968
]
Hudson commented on HDFS-12567:
-------------------------------
SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #13038 (See
[https://builds.apache.org/job/Hadoop-trunk-Commit/13038/])
HDFS-12567. BlockPlacementPolicyRackFaultTolerant fails with racks with (wang:
rev 644c2f6924f341f51d809c91dccfff88fc82f6f0)
* (edit)
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/BlockPlacementPolicy.java
* (edit)
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/BlockPlacementPolicyRackFaultTolerant.java
* (add)
hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestErasureCodingMultipleRacks.java
> BlockPlacementPolicyRackFaultTolerant fails with racks with very few nodes
> --------------------------------------------------------------------------
>
> Key: HDFS-12567
> URL: https://issues.apache.org/jira/browse/HDFS-12567
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: erasure-coding
> Affects Versions: 3.0.0-alpha1
> Reporter: Andrew Wang
> Assignee: Andrew Wang
> Labels: hdfs-ec-3.0-must-do
> Fix For: 3.0.0
>
> Attachments: HDFS-12567.001.patch, HDFS-12567.002.patch,
> HDFS-12567.003.patch, HDFS-12567.repro.patch
>
>
> Found this while doing some testing on an internal cluster with an unusual
> setup. We have a rack with ~20 nodes, then a few more with just a few nodes.
> It would fail to get (# data blocks) datanodes even though there were plenty
> of DNs on the rack with 20 DNs.
> I managed to reproduce this same issue in a unit test, stack trace like this:
> {noformat}
> java.io.IOException: File /testfile0 could only be written to 5 of the 6
> required nodes for RS-6-3-1024k. There are 9 datanode(s) running and no
> node(s) are excluded in this operation.
> at
> org.apache.hadoop.hdfs.server.blockmanagement.BlockManager.chooseTarget4NewBlock(BlockManager.java:2083)
> at
> org.apache.hadoop.hdfs.server.namenode.FSDirWriteFileOp.chooseTargetForNewBlock(FSDirWriteFileOp.java:286)
> at
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getAdditionalBlock(FSNamesystem.java:2609)
> at
> org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.addBlock(NameNodeRpcServer.java:863)
> at
> org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.addBlock(ClientNamenodeProtocolServerSideTranslatorPB.java:548)
> at
> org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java)
> at
> org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:523)
> at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:991)
> at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:869)
> at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:815)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1962)
> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2675)
> {noformat}
> This isn't a very critical bug since it's an unusual rack configuration, but
> it can easily happen during testing.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
