[
https://issues.apache.org/jira/browse/HDFS-16548?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17525115#comment-17525115
]
Hemanth Boyina commented on HDFS-16548:
---
thanks for reporting [~tomscut] , cc [~sodonnell]
i have done some analysis regarding the issue, the test failures were related
to replication on top of rename with snapshots, We create Inode References,
FileDiffs and File with snapshot features when renames were involved on the
directories which has snapshots, in our current test case these were involved
on calling setReplication , in FSDirAttrOp#unprotectedSetReplication we get
{quote}{{// Make sure the directory has sufficient quotas}}
{{short oldBR = file.getPreferredBlockReplication();}}
{quote}
In getPreferredBlockReplication, we compare the file replication along with max
replication which was their in file diffs
{quote}short max = getFileReplication(CURRENT_STATE_ID);
FileWithSnapshotFeature sf = this.getFileWithSnapshotFeature();
if (sf != null) {
short maxInSnapshot = sf.getMaxBlockRepInDiffs(null);
}
max = maxInSnapshot > max ? maxInSnapshot : max;{quote}
in our test case we modified the replication on each rename
{quote}/dir1/foo/bar1 -> set replication 3
/dir1/foo/bar1 renameTo /dir2/foo/bar1
/dir2/foo/bar1 -> set replication 2
/dir2/foo/bar1 renameTo /dir3/foo/bar1
/dir3/foo/bar1 -> set replication 1
/dir3/foo/bar1 renameTo /dir2/foo/bar1
/dir2/foo/bar1 -> set replication 3
{quote}
in the last step while setting replication as 3 to /dir2/foo/bar1 (which
currently has replication 1) , bar1 already has diffs with replication 3 and
replication 2 , so getPreferredBlockReplicaton returns 3 , as we are trying to
set replication as 3 we exit with SetRepStatus.UNCHANGED, but the actual file
replication of bar1 is still 1
Please let me know if i am missing something here
> Failed unit test testRenameMoreThanOnceAcrossSnapDirs_2
> ---
>
> Key: HDFS-16548
> URL: https://issues.apache.org/jira/browse/HDFS-16548
> Project: Hadoop HDFS
> Issue Type: Bug
>Reporter: tomscut
>Priority: Major
>
> It seems to be related to HDFS-16531.
> {code:java}
> [ERROR] Tests run: 44, Failures: 6, Errors: 0, Skipped: 0, Time elapsed:
> 143.701 s <<< FAILURE! - in
> org.apache.hadoop.hdfs.server.namenode.snapshot.TestRenameWithSnapshots
> [ERROR]
> testRenameMoreThanOnceAcrossSnapDirs_2(org.apache.hadoop.hdfs.server.namenode.snapshot.TestRenameWithSnapshots)
> Time elapsed: 6.606 s <<< FAILURE!
> java.lang.AssertionError: expected:<3> but was:<1>
> at org.junit.Assert.fail(Assert.java:89)
> at org.junit.Assert.failNotEquals(Assert.java:835)
> at org.junit.Assert.assertEquals(Assert.java:647)
> at org.junit.Assert.assertEquals(Assert.java:633)
> at
> org.apache.hadoop.hdfs.server.namenode.snapshot.TestRenameWithSnapshots.testRenameMoreThanOnceAcrossSnapDirs_2(TestRenameWithSnapshots.java:985)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59)
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56)
> at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> at
> org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
> at
> org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
> at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306)
> at
> org.junit.runners.BlockJUnit4ClassRunner$1.evaluate(BlockJUnit4ClassRunner.java:100)
> at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:366)
> at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103)
> at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63)
> at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331)
> at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79)
> at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329)
> at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66)
> at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293)
> at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306)
> at org.junit.runners.ParentRunner.run(ParentRunner.java:413)
>
