[
https://issues.apache.org/jira/browse/HDFS-11946?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16042105#comment-16042105
]
Tsz Wo Nicholas Sze edited comment on HDFS-11946 at 6/8/17 2:39 AM:
--------------------------------------------------------------------
For reproducing the problem, run
TestOzoneContainerRatis.testBothGetandPutSmallFileRatisNetty. It is easier to
understand the log if we add a message for "Created a new container" and
comment out the single node test as shown below
{code}
diff --git
a/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/ozone/container/common/helpers/ContainerUtils.java
b/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/ozone/container/common/helpers/ContainerUtils.java
index 7d0e75667c..f7b191c887 100644
---
a/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/ozone/container/common/helpers/ContainerUtils.java
+++
b/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/ozone/container/common/helpers/ContainerUtils.java
@@ -218,6 +218,9 @@ public static void verifyIsNewContainer(File containerFile,
File metadataFile)
log.error("creation of a new container file failed. File: {}",
containerFile.toPath());
throw new IOException("creation of a new container file failed.");
+ } else {
+ log.info("Created a new container. File: {}",
+ containerFile.toPath());
}
if (!metadataFile.createNewFile()) {
diff --git
a/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/ozone/container/ozoneimpl/TestOzoneContainerRatis.java
b/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/ozone/container/ozoneimpl/TestOzoneContainerRatis.java
index f77e731d45..8cbaf569fd 100644
---
a/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/ozone/container/ozoneimpl/TestOzoneContainerRatis.java
+++
b/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/ozone/container/ozoneimpl/TestOzoneContainerRatis.java
@@ -130,7 +130,7 @@ private static void runTestBothGetandPutSmallFileRatis(
@Test
public void testBothGetandPutSmallFileRatisNetty() throws Exception {
- runTestBothGetandPutSmallFileRatis(SupportedRpcType.NETTY, 1);
+// runTestBothGetandPutSmallFileRatis(SupportedRpcType.NETTY, 1);
runTestBothGetandPutSmallFileRatis(SupportedRpcType.NETTY, 3);
}
{code}
was (Author: szetszwo):
For reproducing the problem, run
TestOzoneContainerRatis.testBothGetandPutSmallFileRatisNetty. It is easier to
understand the log if we add a message for "Created of a new container" and
comment out the single node test as shown below
{code}
diff --git
a/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/ozone/container/common/helpers/ContainerUtils.java
b/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/ozone/container/common/helpers/ContainerUtils.java
index 7d0e75667c..f7b191c887 100644
---
a/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/ozone/container/common/helpers/ContainerUtils.java
+++
b/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/ozone/container/common/helpers/ContainerUtils.java
@@ -218,6 +218,9 @@ public static void verifyIsNewContainer(File containerFile,
File metadataFile)
log.error("creation of a new container file failed. File: {}",
containerFile.toPath());
throw new IOException("creation of a new container file failed.");
+ } else {
+ log.info("Created of a new container. File: {}",
+ containerFile.toPath());
}
if (!metadataFile.createNewFile()) {
diff --git
a/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/ozone/container/ozoneimpl/TestOzoneContainerRatis.java
b/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/ozone/container/ozoneimpl/TestOzoneContainerRatis.java
index f77e731d45..8cbaf569fd 100644
---
a/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/ozone/container/ozoneimpl/TestOzoneContainerRatis.java
+++
b/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/ozone/container/ozoneimpl/TestOzoneContainerRatis.java
@@ -130,7 +130,7 @@ private static void runTestBothGetandPutSmallFileRatis(
@Test
public void testBothGetandPutSmallFileRatisNetty() throws Exception {
- runTestBothGetandPutSmallFileRatis(SupportedRpcType.NETTY, 1);
+// runTestBothGetandPutSmallFileRatis(SupportedRpcType.NETTY, 1);
runTestBothGetandPutSmallFileRatis(SupportedRpcType.NETTY, 3);
}
{code}
> Ozone: Containers in different datanodes are mapped to the same location
> ------------------------------------------------------------------------
>
> Key: HDFS-11946
> URL: https://issues.apache.org/jira/browse/HDFS-11946
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: ozone
> Reporter: Tsz Wo Nicholas Sze
> Assignee: Anu Engineer
>
> This is a problem in unit tests. Containers with the same container name in
> different datanodes are mapped to the same local path location. As a result,
> the first datanode will be able to succeed creating the container file but
> the remaining datanodes will fail to create the container file with
> FileAlreadyExistsException.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
