[jira] [Work logged] (HDFS-16213) Flaky test TestFsDatasetImpl#testDnRestartWithHardLink
[
https://issues.apache.org/jira/browse/HDFS-16213?focusedWorklogId=653330=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-653330
]
ASF GitHub Bot logged work on HDFS-16213:
-
Author: ASF GitHub Bot
Created on: 21/Sep/21 01:12
Start Date: 21/Sep/21 01:12
Worklog Time Spent: 10m
Work Description: aajisaka commented on pull request #3386:
URL: https://github.com/apache/hadoop/pull/3386#issuecomment-923496517
Merged. Thank you @virajjasani @jojochuang @ferhui @LeonGao91
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 653330)
Time Spent: 9.5h (was: 9h 20m)
> Flaky test TestFsDatasetImpl#testDnRestartWithHardLink
> --
>
> Key: HDFS-16213
> URL: https://issues.apache.org/jira/browse/HDFS-16213
> Project: Hadoop HDFS
> Issue Type: Sub-task
>Reporter: Viraj Jasani
>Assignee: Viraj Jasani
>Priority: Major
> Labels: pull-request-available
> Time Spent: 9.5h
> Remaining Estimate: 0h
>
> Failure case:
> [here|https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3359/4/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt]
> {code:java}
> [ERROR]
> testDnRestartWithHardLink(org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.TestFsDatasetImpl)
> Time elapsed: 7.768 s <<< FAILURE![ERROR]
> testDnRestartWithHardLink(org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.TestFsDatasetImpl)
> Time elapsed: 7.768 s <<< FAILURE!java.lang.AssertionError at
> org.junit.Assert.fail(Assert.java:87) at
> org.junit.Assert.assertTrue(Assert.java:42) at
> org.junit.Assert.assertTrue(Assert.java:53) at
> org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.TestFsDatasetImpl.testDnRestartWithHardLink(TestFsDatasetImpl.java:1344)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498) at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59)
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56)
> at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> at
> org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:299)
> at
> org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:293)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266) at
> java.lang.Thread.run(Thread.java:748)
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDFS-16213) Flaky test TestFsDatasetImpl#testDnRestartWithHardLink
[
https://issues.apache.org/jira/browse/HDFS-16213?focusedWorklogId=653329=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-653329
]
ASF GitHub Bot logged work on HDFS-16213:
-
Author: ASF GitHub Bot
Created on: 21/Sep/21 01:11
Start Date: 21/Sep/21 01:11
Worklog Time Spent: 10m
Work Description: aajisaka merged pull request #3386:
URL: https://github.com/apache/hadoop/pull/3386
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 653329)
Time Spent: 9h 20m (was: 9h 10m)
> Flaky test TestFsDatasetImpl#testDnRestartWithHardLink
> --
>
> Key: HDFS-16213
> URL: https://issues.apache.org/jira/browse/HDFS-16213
> Project: Hadoop HDFS
> Issue Type: Sub-task
>Reporter: Viraj Jasani
>Assignee: Viraj Jasani
>Priority: Major
> Labels: pull-request-available
> Time Spent: 9h 20m
> Remaining Estimate: 0h
>
> Failure case:
> [here|https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3359/4/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt]
> {code:java}
> [ERROR]
> testDnRestartWithHardLink(org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.TestFsDatasetImpl)
> Time elapsed: 7.768 s <<< FAILURE![ERROR]
> testDnRestartWithHardLink(org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.TestFsDatasetImpl)
> Time elapsed: 7.768 s <<< FAILURE!java.lang.AssertionError at
> org.junit.Assert.fail(Assert.java:87) at
> org.junit.Assert.assertTrue(Assert.java:42) at
> org.junit.Assert.assertTrue(Assert.java:53) at
> org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.TestFsDatasetImpl.testDnRestartWithHardLink(TestFsDatasetImpl.java:1344)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498) at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59)
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56)
> at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> at
> org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:299)
> at
> org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:293)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266) at
> java.lang.Thread.run(Thread.java:748)
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDFS-16213) Flaky test TestFsDatasetImpl#testDnRestartWithHardLink
[
https://issues.apache.org/jira/browse/HDFS-16213?focusedWorklogId=653241=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-653241
]
ASF GitHub Bot logged work on HDFS-16213:
-
Author: ASF GitHub Bot
Created on: 20/Sep/21 22:22
Start Date: 20/Sep/21 22:22
Worklog Time Spent: 10m
Work Description: hadoop-yetus commented on pull request #3386:
URL: https://github.com/apache/hadoop/pull/3386#issuecomment-923381202
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 12m 13s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 2 new or modified test files. |
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 32m 13s | | trunk passed |
| +1 :green_heart: | compile | 1m 24s | | trunk passed with JDK
Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 |
| +1 :green_heart: | compile | 1m 14s | | trunk passed with JDK
Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| +1 :green_heart: | checkstyle | 1m 3s | | trunk passed |
| +1 :green_heart: | mvnsite | 1m 27s | | trunk passed |
| +1 :green_heart: | javadoc | 0m 58s | | trunk passed with JDK
Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 |
| +1 :green_heart: | javadoc | 1m 28s | | trunk passed with JDK
Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| +1 :green_heart: | spotbugs | 3m 6s | | trunk passed |
| +1 :green_heart: | shadedclient | 21m 1s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 1m 9s | | the patch passed |
| +1 :green_heart: | compile | 1m 14s | | the patch passed with JDK
Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 |
| +1 :green_heart: | javac | 1m 14s | | the patch passed |
| +1 :green_heart: | compile | 1m 8s | | the patch passed with JDK
Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| +1 :green_heart: | javac | 1m 8s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| +1 :green_heart: | checkstyle | 0m 54s | |
hadoop-hdfs-project/hadoop-hdfs: The patch generated 0 new + 240 unchanged - 3
fixed = 240 total (was 243) |
| +1 :green_heart: | mvnsite | 1m 13s | | the patch passed |
| +1 :green_heart: | javadoc | 0m 47s | | the patch passed with JDK
Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 |
| +1 :green_heart: | javadoc | 1m 23s | | the patch passed with JDK
Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| +1 :green_heart: | spotbugs | 3m 9s | | the patch passed |
| +1 :green_heart: | shadedclient | 21m 7s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 229m 50s | | hadoop-hdfs in the patch
passed. |
| +1 :green_heart: | asflicense | 0m 46s | | The patch does not
generate ASF License warnings. |
| | | 336m 57s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3386/19/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/3386 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient spotbugs checkstyle codespell |
| uname | Linux d0aa77e16202 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9
23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 6bfbb3071f187242989ba9631f65929b0d9d9553 |
| Default Java | Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| Multi-JDK versions |
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
/usr/lib/jvm/java-8-openjdk-amd64:Private
Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| Test Results |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3386/19/testReport/ |
| Max. process+thread count | 3178 (vs. ulimit of 5500) |
| modules | C: hadoop-hdfs-project/hadoop-hdfs U:
hadoop-hdfs-project/hadoop-hdfs |
| Console output |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3386/19/console |
| versions | git=2.25.1 maven=3.6.3 spotbugs=4.2.2 |
[jira] [Work logged] (HDFS-16213) Flaky test TestFsDatasetImpl#testDnRestartWithHardLink
[
https://issues.apache.org/jira/browse/HDFS-16213?focusedWorklogId=653080=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-653080
]
ASF GitHub Bot logged work on HDFS-16213:
-
Author: ASF GitHub Bot
Created on: 20/Sep/21 16:44
Start Date: 20/Sep/21 16:44
Worklog Time Spent: 10m
Work Description: virajjasani commented on a change in pull request #3386:
URL: https://github.com/apache/hadoop/pull/3386#discussion_r712340855
##
File path:
hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/TestFsDatasetImpl.java
##
@@ -1344,26 +1359,34 @@ public void testDnRestartWithHardLink() {
assertTrue(Files.exists(Paths.get(newReplicaInfo.getBlockURI(;
assertTrue(Files.exists(Paths.get(oldReplicaInfo.getBlockURI(;
- DirectoryScanner scanner = new DirectoryScanner(
- cluster.getDataNodes().get(0).getFSDataset(), conf);
+ // Before starting Dir Scanner, we should enable deleteDuplicateReplicas.
+ FsDatasetSpi fsDataset = cluster.getDataNodes().get(0).getFSDataset();
+ DirectoryScanner scanner = new DirectoryScanner(fsDataset, conf);
+ FsVolumeImpl fsVolume =
+ (FsVolumeImpl) fsDataset.getFsVolumeReferences().get(0);
+ fsVolume.getBlockPoolSlice(fsVolume.getBlockPoolList()[0])
+ .setDeleteDuplicateReplicasForTests(true);
scanner.start();
scanner.run();
- GenericTestUtils.waitFor(new Supplier() {
-@Override public Boolean get() {
- return !Files.exists(Paths.get(oldReplicaInfo.getBlockURI()));
-}
- }, 100, 1);
+ GenericTestUtils.waitFor(
+ () -> !Files.exists(Paths.get(oldReplicaInfo.getBlockURI())),
+ 100, 1, "Old replica is not deleted by DirScanner even after "
+ + "10s of waiting has elapsed");
assertTrue(Files.exists(Paths.get(newReplicaInfo.getBlockURI(;
validateFileLen(fs, fileLen, filePath);
-} catch (Exception ex) {
- LOG.info("Exception in testDnRestartWithHardLink ", ex);
- fail("Exception while testing testDnRestartWithHardLink ");
+ // Additional tests to ensure latest replica gets deleted after file
+ // deletion.
+ fs.delete(filePath, false);
+ GenericTestUtils.waitFor(
+ () -> !Files.exists(Paths.get(newReplicaInfo.getBlockURI())),
+ 100, 1);
} finally {
- if (cluster.isClusterUp()) {
-cluster.shutdown();
+ conf.unset(DFSConfigKeys.DFS_DATANODE_DUPLICATE_REPLICA_DELETION);
Review comment:
Done
##
File path:
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/FsDatasetImpl.java
##
@@ -1049,12 +1050,17 @@ static File moveBlockFiles(Block b, ReplicaInfo
replicaInfo, File destdir)
static File[] hardLinkBlockFiles(ReplicaInfo srcReplica, File dstMeta,
File dstFile)
throws IOException {
+FsVolumeSpi srcReplicaVolume = srcReplica.getVolume();
Review comment:
That's right, these changes are to simplify using same variable for
logging as well as method arguments.
And one more improvement is that before patch, we have isDebugEnabled()
guarding info log, this is also fixed with this PR. It's the last change of
this file.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 653080)
Time Spent: 9h (was: 8h 50m)
> Flaky test TestFsDatasetImpl#testDnRestartWithHardLink
> --
>
> Key: HDFS-16213
> URL: https://issues.apache.org/jira/browse/HDFS-16213
> Project: Hadoop HDFS
> Issue Type: Sub-task
>Reporter: Viraj Jasani
>Assignee: Viraj Jasani
>Priority: Major
> Labels: pull-request-available
> Time Spent: 9h
> Remaining Estimate: 0h
>
> Failure case:
> [here|https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3359/4/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt]
> {code:java}
> [ERROR]
> testDnRestartWithHardLink(org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.TestFsDatasetImpl)
> Time elapsed: 7.768 s <<< FAILURE![ERROR]
> testDnRestartWithHardLink(org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.TestFsDatasetImpl)
> Time elapsed: 7.768 s <<< FAILURE!java.lang.AssertionError at
> org.junit.Assert.fail(Assert.java:87) at
> org.junit.Assert.assertTrue(Assert.java:42) at
>
[jira] [Work logged] (HDFS-16213) Flaky test TestFsDatasetImpl#testDnRestartWithHardLink
[
https://issues.apache.org/jira/browse/HDFS-16213?focusedWorklogId=653069=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-653069
]
ASF GitHub Bot logged work on HDFS-16213:
-
Author: ASF GitHub Bot
Created on: 20/Sep/21 16:04
Start Date: 20/Sep/21 16:04
Worklog Time Spent: 10m
Work Description: ferhui commented on pull request #3386:
URL: https://github.com/apache/hadoop/pull/3386#issuecomment-923064017
Sorry for late response, I'm too busy recently.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 653069)
Time Spent: 8h 50m (was: 8h 40m)
> Flaky test TestFsDatasetImpl#testDnRestartWithHardLink
> --
>
> Key: HDFS-16213
> URL: https://issues.apache.org/jira/browse/HDFS-16213
> Project: Hadoop HDFS
> Issue Type: Sub-task
>Reporter: Viraj Jasani
>Assignee: Viraj Jasani
>Priority: Major
> Labels: pull-request-available
> Time Spent: 8h 50m
> Remaining Estimate: 0h
>
> Failure case:
> [here|https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3359/4/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt]
> {code:java}
> [ERROR]
> testDnRestartWithHardLink(org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.TestFsDatasetImpl)
> Time elapsed: 7.768 s <<< FAILURE![ERROR]
> testDnRestartWithHardLink(org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.TestFsDatasetImpl)
> Time elapsed: 7.768 s <<< FAILURE!java.lang.AssertionError at
> org.junit.Assert.fail(Assert.java:87) at
> org.junit.Assert.assertTrue(Assert.java:42) at
> org.junit.Assert.assertTrue(Assert.java:53) at
> org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.TestFsDatasetImpl.testDnRestartWithHardLink(TestFsDatasetImpl.java:1344)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498) at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59)
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56)
> at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> at
> org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:299)
> at
> org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:293)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266) at
> java.lang.Thread.run(Thread.java:748)
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDFS-16213) Flaky test TestFsDatasetImpl#testDnRestartWithHardLink
[
https://issues.apache.org/jira/browse/HDFS-16213?focusedWorklogId=653068=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-653068
]
ASF GitHub Bot logged work on HDFS-16213:
-
Author: ASF GitHub Bot
Created on: 20/Sep/21 16:03
Start Date: 20/Sep/21 16:03
Worklog Time Spent: 10m
Work Description: ferhui commented on pull request #3386:
URL: https://github.com/apache/hadoop/pull/3386#issuecomment-923062915
BTW, @virajjasani could you please change this PR title as the same as the
title of HDFS-16213 ? Thanks
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 653068)
Time Spent: 8h 40m (was: 8.5h)
> Flaky test TestFsDatasetImpl#testDnRestartWithHardLink
> --
>
> Key: HDFS-16213
> URL: https://issues.apache.org/jira/browse/HDFS-16213
> Project: Hadoop HDFS
> Issue Type: Sub-task
>Reporter: Viraj Jasani
>Assignee: Viraj Jasani
>Priority: Major
> Labels: pull-request-available
> Time Spent: 8h 40m
> Remaining Estimate: 0h
>
> Failure case:
> [here|https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3359/4/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt]
> {code:java}
> [ERROR]
> testDnRestartWithHardLink(org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.TestFsDatasetImpl)
> Time elapsed: 7.768 s <<< FAILURE![ERROR]
> testDnRestartWithHardLink(org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.TestFsDatasetImpl)
> Time elapsed: 7.768 s <<< FAILURE!java.lang.AssertionError at
> org.junit.Assert.fail(Assert.java:87) at
> org.junit.Assert.assertTrue(Assert.java:42) at
> org.junit.Assert.assertTrue(Assert.java:53) at
> org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.TestFsDatasetImpl.testDnRestartWithHardLink(TestFsDatasetImpl.java:1344)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498) at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59)
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56)
> at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> at
> org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:299)
> at
> org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:293)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266) at
> java.lang.Thread.run(Thread.java:748)
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDFS-16213) Flaky test TestFsDatasetImpl#testDnRestartWithHardLink
[
https://issues.apache.org/jira/browse/HDFS-16213?focusedWorklogId=653066=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-653066
]
ASF GitHub Bot logged work on HDFS-16213:
-
Author: ASF GitHub Bot
Created on: 20/Sep/21 15:57
Start Date: 20/Sep/21 15:57
Worklog Time Spent: 10m
Work Description: ferhui commented on a change in pull request #3386:
URL: https://github.com/apache/hadoop/pull/3386#discussion_r712303703
##
File path:
hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/TestFsDatasetImpl.java
##
@@ -1344,26 +1359,34 @@ public void testDnRestartWithHardLink() {
assertTrue(Files.exists(Paths.get(newReplicaInfo.getBlockURI(;
assertTrue(Files.exists(Paths.get(oldReplicaInfo.getBlockURI(;
- DirectoryScanner scanner = new DirectoryScanner(
- cluster.getDataNodes().get(0).getFSDataset(), conf);
+ // Before starting Dir Scanner, we should enable deleteDuplicateReplicas.
+ FsDatasetSpi fsDataset = cluster.getDataNodes().get(0).getFSDataset();
+ DirectoryScanner scanner = new DirectoryScanner(fsDataset, conf);
+ FsVolumeImpl fsVolume =
+ (FsVolumeImpl) fsDataset.getFsVolumeReferences().get(0);
+ fsVolume.getBlockPoolSlice(fsVolume.getBlockPoolList()[0])
+ .setDeleteDuplicateReplicasForTests(true);
scanner.start();
scanner.run();
- GenericTestUtils.waitFor(new Supplier() {
-@Override public Boolean get() {
- return !Files.exists(Paths.get(oldReplicaInfo.getBlockURI()));
-}
- }, 100, 1);
+ GenericTestUtils.waitFor(
+ () -> !Files.exists(Paths.get(oldReplicaInfo.getBlockURI())),
+ 100, 1, "Old replica is not deleted by DirScanner even after "
+ + "10s of waiting has elapsed");
assertTrue(Files.exists(Paths.get(newReplicaInfo.getBlockURI(;
validateFileLen(fs, fileLen, filePath);
-} catch (Exception ex) {
- LOG.info("Exception in testDnRestartWithHardLink ", ex);
- fail("Exception while testing testDnRestartWithHardLink ");
+ // Additional tests to ensure latest replica gets deleted after file
+ // deletion.
+ fs.delete(filePath, false);
+ GenericTestUtils.waitFor(
+ () -> !Files.exists(Paths.get(newReplicaInfo.getBlockURI())),
+ 100, 1);
} finally {
- if (cluster.isClusterUp()) {
-cluster.shutdown();
+ conf.unset(DFSConfigKeys.DFS_DATANODE_DUPLICATE_REPLICA_DELETION);
Review comment:
Maybe we need to reserve original value and set it finally
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 653066)
Time Spent: 8.5h (was: 8h 20m)
> Flaky test TestFsDatasetImpl#testDnRestartWithHardLink
> --
>
> Key: HDFS-16213
> URL: https://issues.apache.org/jira/browse/HDFS-16213
> Project: Hadoop HDFS
> Issue Type: Sub-task
>Reporter: Viraj Jasani
>Assignee: Viraj Jasani
>Priority: Major
> Labels: pull-request-available
> Time Spent: 8.5h
> Remaining Estimate: 0h
>
> Failure case:
> [here|https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3359/4/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt]
> {code:java}
> [ERROR]
> testDnRestartWithHardLink(org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.TestFsDatasetImpl)
> Time elapsed: 7.768 s <<< FAILURE![ERROR]
> testDnRestartWithHardLink(org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.TestFsDatasetImpl)
> Time elapsed: 7.768 s <<< FAILURE!java.lang.AssertionError at
> org.junit.Assert.fail(Assert.java:87) at
> org.junit.Assert.assertTrue(Assert.java:42) at
> org.junit.Assert.assertTrue(Assert.java:53) at
> org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.TestFsDatasetImpl.testDnRestartWithHardLink(TestFsDatasetImpl.java:1344)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498) at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59)
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> at
>
[jira] [Work logged] (HDFS-16213) Flaky test TestFsDatasetImpl#testDnRestartWithHardLink
[
https://issues.apache.org/jira/browse/HDFS-16213?focusedWorklogId=653063=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-653063
]
ASF GitHub Bot logged work on HDFS-16213:
-
Author: ASF GitHub Bot
Created on: 20/Sep/21 15:46
Start Date: 20/Sep/21 15:46
Worklog Time Spent: 10m
Work Description: ferhui commented on a change in pull request #3386:
URL: https://github.com/apache/hadoop/pull/3386#discussion_r712267897
##
File path:
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/FsDatasetImpl.java
##
@@ -1049,12 +1050,17 @@ static File moveBlockFiles(Block b, ReplicaInfo
replicaInfo, File destdir)
static File[] hardLinkBlockFiles(ReplicaInfo srcReplica, File dstMeta,
File dstFile)
throws IOException {
+FsVolumeSpi srcReplicaVolume = srcReplica.getVolume();
Review comment:
Hi @virajjasani here I have one question to ask you.
It seems that there is no change in code logic. Are these code changes of
the whole file for adding debug or trace info?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 653063)
Time Spent: 8h 20m (was: 8h 10m)
> Flaky test TestFsDatasetImpl#testDnRestartWithHardLink
> --
>
> Key: HDFS-16213
> URL: https://issues.apache.org/jira/browse/HDFS-16213
> Project: Hadoop HDFS
> Issue Type: Sub-task
>Reporter: Viraj Jasani
>Assignee: Viraj Jasani
>Priority: Major
> Labels: pull-request-available
> Time Spent: 8h 20m
> Remaining Estimate: 0h
>
> Failure case:
> [here|https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3359/4/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt]
> {code:java}
> [ERROR]
> testDnRestartWithHardLink(org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.TestFsDatasetImpl)
> Time elapsed: 7.768 s <<< FAILURE![ERROR]
> testDnRestartWithHardLink(org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.TestFsDatasetImpl)
> Time elapsed: 7.768 s <<< FAILURE!java.lang.AssertionError at
> org.junit.Assert.fail(Assert.java:87) at
> org.junit.Assert.assertTrue(Assert.java:42) at
> org.junit.Assert.assertTrue(Assert.java:53) at
> org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.TestFsDatasetImpl.testDnRestartWithHardLink(TestFsDatasetImpl.java:1344)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498) at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59)
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56)
> at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> at
> org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:299)
> at
> org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:293)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266) at
> java.lang.Thread.run(Thread.java:748)
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDFS-16213) Flaky test TestFsDatasetImpl#testDnRestartWithHardLink
[
https://issues.apache.org/jira/browse/HDFS-16213?focusedWorklogId=653034=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-653034
]
ASF GitHub Bot logged work on HDFS-16213:
-
Author: ASF GitHub Bot
Created on: 20/Sep/21 15:16
Start Date: 20/Sep/21 15:16
Worklog Time Spent: 10m
Work Description: ferhui commented on a change in pull request #3386:
URL: https://github.com/apache/hadoop/pull/3386#discussion_r712267897
##
File path:
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/FsDatasetImpl.java
##
@@ -1049,12 +1050,17 @@ static File moveBlockFiles(Block b, ReplicaInfo
replicaInfo, File destdir)
static File[] hardLinkBlockFiles(ReplicaInfo srcReplica, File dstMeta,
File dstFile)
throws IOException {
+FsVolumeSpi srcReplicaVolume = srcReplica.getVolume();
Review comment:
Hi @virajjasani here I have one question to ask you.
It seems that there is no change in code logic. Are these codes for adding
debug or trace info?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 653034)
Time Spent: 8h 10m (was: 8h)
> Flaky test TestFsDatasetImpl#testDnRestartWithHardLink
> --
>
> Key: HDFS-16213
> URL: https://issues.apache.org/jira/browse/HDFS-16213
> Project: Hadoop HDFS
> Issue Type: Sub-task
>Reporter: Viraj Jasani
>Assignee: Viraj Jasani
>Priority: Major
> Labels: pull-request-available
> Time Spent: 8h 10m
> Remaining Estimate: 0h
>
> Failure case:
> [here|https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3359/4/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt]
> {code:java}
> [ERROR]
> testDnRestartWithHardLink(org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.TestFsDatasetImpl)
> Time elapsed: 7.768 s <<< FAILURE![ERROR]
> testDnRestartWithHardLink(org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.TestFsDatasetImpl)
> Time elapsed: 7.768 s <<< FAILURE!java.lang.AssertionError at
> org.junit.Assert.fail(Assert.java:87) at
> org.junit.Assert.assertTrue(Assert.java:42) at
> org.junit.Assert.assertTrue(Assert.java:53) at
> org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.TestFsDatasetImpl.testDnRestartWithHardLink(TestFsDatasetImpl.java:1344)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498) at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59)
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56)
> at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> at
> org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:299)
> at
> org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:293)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266) at
> java.lang.Thread.run(Thread.java:748)
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDFS-16213) Flaky test TestFsDatasetImpl#testDnRestartWithHardLink
[
https://issues.apache.org/jira/browse/HDFS-16213?focusedWorklogId=653025=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-653025
]
ASF GitHub Bot logged work on HDFS-16213:
-
Author: ASF GitHub Bot
Created on: 20/Sep/21 15:03
Start Date: 20/Sep/21 15:03
Worklog Time Spent: 10m
Work Description: jojochuang commented on pull request #3386:
URL: https://github.com/apache/hadoop/pull/3386#issuecomment-923010909
+1 thanks Viraj
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 653025)
Time Spent: 8h (was: 7h 50m)
> Flaky test TestFsDatasetImpl#testDnRestartWithHardLink
> --
>
> Key: HDFS-16213
> URL: https://issues.apache.org/jira/browse/HDFS-16213
> Project: Hadoop HDFS
> Issue Type: Sub-task
>Reporter: Viraj Jasani
>Assignee: Viraj Jasani
>Priority: Major
> Labels: pull-request-available
> Time Spent: 8h
> Remaining Estimate: 0h
>
> Failure case:
> [here|https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3359/4/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt]
> {code:java}
> [ERROR]
> testDnRestartWithHardLink(org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.TestFsDatasetImpl)
> Time elapsed: 7.768 s <<< FAILURE![ERROR]
> testDnRestartWithHardLink(org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.TestFsDatasetImpl)
> Time elapsed: 7.768 s <<< FAILURE!java.lang.AssertionError at
> org.junit.Assert.fail(Assert.java:87) at
> org.junit.Assert.assertTrue(Assert.java:42) at
> org.junit.Assert.assertTrue(Assert.java:53) at
> org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.TestFsDatasetImpl.testDnRestartWithHardLink(TestFsDatasetImpl.java:1344)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498) at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59)
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56)
> at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> at
> org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:299)
> at
> org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:293)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266) at
> java.lang.Thread.run(Thread.java:748)
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDFS-16213) Flaky test TestFsDatasetImpl#testDnRestartWithHardLink
[
https://issues.apache.org/jira/browse/HDFS-16213?focusedWorklogId=652939=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-652939
]
ASF GitHub Bot logged work on HDFS-16213:
-
Author: ASF GitHub Bot
Created on: 20/Sep/21 12:50
Start Date: 20/Sep/21 12:50
Worklog Time Spent: 10m
Work Description: virajjasani commented on pull request #3386:
URL: https://github.com/apache/hadoop/pull/3386#issuecomment-922897494
Just to confirm, is this good to go?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 652939)
Time Spent: 7h 50m (was: 7h 40m)
> Flaky test TestFsDatasetImpl#testDnRestartWithHardLink
> --
>
> Key: HDFS-16213
> URL: https://issues.apache.org/jira/browse/HDFS-16213
> Project: Hadoop HDFS
> Issue Type: Sub-task
>Reporter: Viraj Jasani
>Assignee: Viraj Jasani
>Priority: Major
> Labels: pull-request-available
> Time Spent: 7h 50m
> Remaining Estimate: 0h
>
> Failure case:
> [here|https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3359/4/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt]
> {code:java}
> [ERROR]
> testDnRestartWithHardLink(org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.TestFsDatasetImpl)
> Time elapsed: 7.768 s <<< FAILURE![ERROR]
> testDnRestartWithHardLink(org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.TestFsDatasetImpl)
> Time elapsed: 7.768 s <<< FAILURE!java.lang.AssertionError at
> org.junit.Assert.fail(Assert.java:87) at
> org.junit.Assert.assertTrue(Assert.java:42) at
> org.junit.Assert.assertTrue(Assert.java:53) at
> org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.TestFsDatasetImpl.testDnRestartWithHardLink(TestFsDatasetImpl.java:1344)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498) at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59)
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56)
> at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> at
> org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:299)
> at
> org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:293)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266) at
> java.lang.Thread.run(Thread.java:748)
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDFS-16213) Flaky test TestFsDatasetImpl#testDnRestartWithHardLink
[
https://issues.apache.org/jira/browse/HDFS-16213?focusedWorklogId=652141=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-652141
]
ASF GitHub Bot logged work on HDFS-16213:
-
Author: ASF GitHub Bot
Created on: 17/Sep/21 07:29
Start Date: 17/Sep/21 07:29
Worklog Time Spent: 10m
Work Description: virajjasani commented on pull request #3386:
URL: https://github.com/apache/hadoop/pull/3386#issuecomment-921565797
Thanks for the reviews @aajisaka @LeonGao91
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 652141)
Time Spent: 7h 40m (was: 7.5h)
> Flaky test TestFsDatasetImpl#testDnRestartWithHardLink
> --
>
> Key: HDFS-16213
> URL: https://issues.apache.org/jira/browse/HDFS-16213
> Project: Hadoop HDFS
> Issue Type: Sub-task
>Reporter: Viraj Jasani
>Assignee: Viraj Jasani
>Priority: Major
> Labels: pull-request-available
> Time Spent: 7h 40m
> Remaining Estimate: 0h
>
> Failure case:
> [here|https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3359/4/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt]
> {code:java}
> [ERROR]
> testDnRestartWithHardLink(org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.TestFsDatasetImpl)
> Time elapsed: 7.768 s <<< FAILURE![ERROR]
> testDnRestartWithHardLink(org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.TestFsDatasetImpl)
> Time elapsed: 7.768 s <<< FAILURE!java.lang.AssertionError at
> org.junit.Assert.fail(Assert.java:87) at
> org.junit.Assert.assertTrue(Assert.java:42) at
> org.junit.Assert.assertTrue(Assert.java:53) at
> org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.TestFsDatasetImpl.testDnRestartWithHardLink(TestFsDatasetImpl.java:1344)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498) at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59)
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56)
> at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> at
> org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:299)
> at
> org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:293)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266) at
> java.lang.Thread.run(Thread.java:748)
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDFS-16213) Flaky test TestFsDatasetImpl#testDnRestartWithHardLink
[
https://issues.apache.org/jira/browse/HDFS-16213?focusedWorklogId=651479=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-651479
]
ASF GitHub Bot logged work on HDFS-16213:
-
Author: ASF GitHub Bot
Created on: 16/Sep/21 06:19
Start Date: 16/Sep/21 06:19
Worklog Time Spent: 10m
Work Description: virajjasani commented on pull request #3386:
URL: https://github.com/apache/hadoop/pull/3386#issuecomment-920611614
Recent QA failures are not relevant and majority are facing timeout issues,
will take a look as part of separate Jira.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 651479)
Time Spent: 7.5h (was: 7h 20m)
> Flaky test TestFsDatasetImpl#testDnRestartWithHardLink
> --
>
> Key: HDFS-16213
> URL: https://issues.apache.org/jira/browse/HDFS-16213
> Project: Hadoop HDFS
> Issue Type: Sub-task
>Reporter: Viraj Jasani
>Assignee: Viraj Jasani
>Priority: Major
> Labels: pull-request-available
> Time Spent: 7.5h
> Remaining Estimate: 0h
>
> Failure case:
> [here|https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3359/4/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt]
> {code:java}
> [ERROR]
> testDnRestartWithHardLink(org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.TestFsDatasetImpl)
> Time elapsed: 7.768 s <<< FAILURE![ERROR]
> testDnRestartWithHardLink(org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.TestFsDatasetImpl)
> Time elapsed: 7.768 s <<< FAILURE!java.lang.AssertionError at
> org.junit.Assert.fail(Assert.java:87) at
> org.junit.Assert.assertTrue(Assert.java:42) at
> org.junit.Assert.assertTrue(Assert.java:53) at
> org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.TestFsDatasetImpl.testDnRestartWithHardLink(TestFsDatasetImpl.java:1344)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498) at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59)
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56)
> at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> at
> org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:299)
> at
> org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:293)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266) at
> java.lang.Thread.run(Thread.java:748)
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDFS-16213) Flaky test TestFsDatasetImpl#testDnRestartWithHardLink
[
https://issues.apache.org/jira/browse/HDFS-16213?focusedWorklogId=651455=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-651455
]
ASF GitHub Bot logged work on HDFS-16213:
-
Author: ASF GitHub Bot
Created on: 16/Sep/21 03:32
Start Date: 16/Sep/21 03:32
Worklog Time Spent: 10m
Work Description: hadoop-yetus commented on pull request #3386:
URL: https://github.com/apache/hadoop/pull/3386#issuecomment-920548456
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 1m 13s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 1s | | codespell was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 2 new or modified test files. |
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 32m 2s | | trunk passed |
| +1 :green_heart: | compile | 1m 27s | | trunk passed with JDK
Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 |
| +1 :green_heart: | compile | 1m 17s | | trunk passed with JDK
Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| +1 :green_heart: | checkstyle | 1m 3s | | trunk passed |
| +1 :green_heart: | mvnsite | 1m 24s | | trunk passed |
| +1 :green_heart: | javadoc | 0m 59s | | trunk passed with JDK
Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 |
| +1 :green_heart: | javadoc | 1m 29s | | trunk passed with JDK
Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| +1 :green_heart: | spotbugs | 3m 27s | | trunk passed |
| +1 :green_heart: | shadedclient | 34m 3s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 1m 18s | | the patch passed |
| +1 :green_heart: | compile | 1m 19s | | the patch passed with JDK
Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 |
| +1 :green_heart: | javac | 1m 19s | | the patch passed |
| +1 :green_heart: | compile | 1m 13s | | the patch passed with JDK
Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| +1 :green_heart: | javac | 1m 13s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| +1 :green_heart: | checkstyle | 0m 53s | |
hadoop-hdfs-project/hadoop-hdfs: The patch generated 0 new + 240 unchanged - 3
fixed = 240 total (was 243) |
| +1 :green_heart: | mvnsite | 1m 22s | | the patch passed |
| +1 :green_heart: | javadoc | 0m 53s | | the patch passed with JDK
Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 |
| +1 :green_heart: | javadoc | 1m 29s | | the patch passed with JDK
Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| +1 :green_heart: | spotbugs | 3m 29s | | the patch passed |
| +1 :green_heart: | shadedclient | 34m 18s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| -1 :x: | unit | 517m 7s |
[/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3386/15/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt)
| hadoop-hdfs in the patch passed. |
| +1 :green_heart: | asflicense | 0m 49s | | The patch does not
generate ASF License warnings. |
| | | 640m 4s | | |
| Reason | Tests |
|---:|:--|
| Failed junit tests | hadoop.hdfs.TestViewDistributedFileSystemContract |
| | hadoop.hdfs.TestSnapshotCommands |
| | hadoop.fs.viewfs.TestViewFileSystemOverloadSchemeWithHdfsScheme |
| |
hadoop.fs.viewfs.TestViewFileSystemOverloadSchemeHdfsFileSystemContract |
| | hadoop.hdfs.TestViewDistributedFileSystemWithMountLinks |
| | hadoop.hdfs.server.namenode.ha.TestSeveralNameNodes |
| | hadoop.fs.viewfs.TestViewFSOverloadSchemeWithMountTableConfigInHDFS |
| | hadoop.hdfs.server.namenode.ha.TestEditLogTailer |
| | hadoop.hdfs.TestHDFSFileSystemContract |
| | hadoop.hdfs.web.TestWebHdfsFileSystemContract |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3386/15/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/3386 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient spotbugs checkstyle codespell |
| uname | Linux be13ef196036 4.15.0-65-generic #74-Ubuntu SMP Tue Sep
[jira] [Work logged] (HDFS-16213) Flaky test TestFsDatasetImpl#testDnRestartWithHardLink
[
https://issues.apache.org/jira/browse/HDFS-16213?focusedWorklogId=651448=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-651448
]
ASF GitHub Bot logged work on HDFS-16213:
-
Author: ASF GitHub Bot
Created on: 16/Sep/21 02:48
Start Date: 16/Sep/21 02:48
Worklog Time Spent: 10m
Work Description: hadoop-yetus commented on pull request #3386:
URL: https://github.com/apache/hadoop/pull/3386#issuecomment-920532949
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 58s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 1s | | codespell was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 2 new or modified test files. |
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 36m 15s | | trunk passed |
| +1 :green_heart: | compile | 1m 35s | | trunk passed with JDK
Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 |
| +1 :green_heart: | compile | 1m 24s | | trunk passed with JDK
Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| +1 :green_heart: | checkstyle | 1m 4s | | trunk passed |
| +1 :green_heart: | mvnsite | 1m 30s | | trunk passed |
| +1 :green_heart: | javadoc | 1m 0s | | trunk passed with JDK
Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 |
| +1 :green_heart: | javadoc | 1m 31s | | trunk passed with JDK
Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| +1 :green_heart: | spotbugs | 3m 33s | | trunk passed |
| +1 :green_heart: | shadedclient | 37m 4s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 1m 21s | | the patch passed |
| +1 :green_heart: | compile | 1m 27s | | the patch passed with JDK
Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 |
| +1 :green_heart: | javac | 1m 27s | | the patch passed |
| +1 :green_heart: | compile | 1m 18s | | the patch passed with JDK
Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| +1 :green_heart: | javac | 1m 18s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| +1 :green_heart: | checkstyle | 0m 57s | |
hadoop-hdfs-project/hadoop-hdfs: The patch generated 0 new + 240 unchanged - 3
fixed = 240 total (was 243) |
| +1 :green_heart: | mvnsite | 1m 22s | | the patch passed |
| +1 :green_heart: | javadoc | 0m 53s | | the patch passed with JDK
Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 |
| +1 :green_heart: | javadoc | 1m 27s | | the patch passed with JDK
Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| +1 :green_heart: | spotbugs | 3m 43s | | the patch passed |
| +1 :green_heart: | shadedclient | 38m 49s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| -1 :x: | unit | 361m 57s |
[/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3386/17/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt)
| hadoop-hdfs in the patch passed. |
| +1 :green_heart: | asflicense | 0m 40s | | The patch does not
generate ASF License warnings. |
| | | 497m 5s | | |
| Reason | Tests |
|---:|:--|
| Failed junit tests | hadoop.hdfs.TestSnapshotCommands |
| | hadoop.hdfs.TestHDFSFileSystemContract |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3386/17/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/3386 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient spotbugs checkstyle codespell |
| uname | Linux 543350b07c13 4.15.0-147-generic #151-Ubuntu SMP Fri Jun 18
19:21:19 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 8fcce65ab54f61a8d97adace65ea847df4d69bb0 |
| Default Java | Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| Multi-JDK versions |
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
/usr/lib/jvm/java-8-openjdk-amd64:Private
Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| Test Results |
[jira] [Work logged] (HDFS-16213) Flaky test TestFsDatasetImpl#testDnRestartWithHardLink
[
https://issues.apache.org/jira/browse/HDFS-16213?focusedWorklogId=651257=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-651257
]
ASF GitHub Bot logged work on HDFS-16213:
-
Author: ASF GitHub Bot
Created on: 15/Sep/21 18:15
Start Date: 15/Sep/21 18:15
Worklog Time Spent: 10m
Work Description: hadoop-yetus commented on pull request #3386:
URL: https://github.com/apache/hadoop/pull/3386#issuecomment-920264187
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 50s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 2 new or modified test files. |
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 32m 38s | | trunk passed |
| +1 :green_heart: | compile | 1m 23s | | trunk passed with JDK
Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 |
| +1 :green_heart: | compile | 1m 14s | | trunk passed with JDK
Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| +1 :green_heart: | checkstyle | 1m 1s | | trunk passed |
| +1 :green_heart: | mvnsite | 1m 24s | | trunk passed |
| +1 :green_heart: | javadoc | 0m 56s | | trunk passed with JDK
Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 |
| +1 :green_heart: | javadoc | 1m 28s | | trunk passed with JDK
Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| +1 :green_heart: | spotbugs | 3m 7s | | trunk passed |
| +1 :green_heart: | shadedclient | 30m 53s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 1m 18s | | the patch passed |
| +1 :green_heart: | compile | 1m 19s | | the patch passed with JDK
Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 |
| +1 :green_heart: | javac | 1m 19s | | the patch passed |
| +1 :green_heart: | compile | 1m 9s | | the patch passed with JDK
Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| +1 :green_heart: | javac | 1m 9s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| +1 :green_heart: | checkstyle | 0m 54s | |
hadoop-hdfs-project/hadoop-hdfs: The patch generated 0 new + 240 unchanged - 3
fixed = 240 total (was 243) |
| +1 :green_heart: | mvnsite | 1m 17s | | the patch passed |
| -1 :x: | javadoc | 0m 20s |
[/patch-javadoc-hadoop-hdfs-project_hadoop-hdfs-jdkUbuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3386/16/artifact/out/patch-javadoc-hadoop-hdfs-project_hadoop-hdfs-jdkUbuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04.txt)
| hadoop-hdfs in the patch failed with JDK
Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04. |
| -1 :x: | javadoc | 0m 15s |
[/patch-javadoc-hadoop-hdfs-project_hadoop-hdfs-jdkPrivateBuild-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3386/16/artifact/out/patch-javadoc-hadoop-hdfs-project_hadoop-hdfs-jdkPrivateBuild-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10.txt)
| hadoop-hdfs in the patch failed with JDK Private
Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10. |
| -1 :x: | spotbugs | 0m 15s |
[/patch-spotbugs-hadoop-hdfs-project_hadoop-hdfs.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3386/16/artifact/out/patch-spotbugs-hadoop-hdfs-project_hadoop-hdfs.txt)
| hadoop-hdfs in the patch failed. |
| -1 :x: | shadedclient | 5m 33s | | patch has errors when building
and testing our client artifacts. |
_ Other Tests _ |
| -1 :x: | unit | 0m 15s |
[/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3386/16/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt)
| hadoop-hdfs in the patch failed. |
| +0 :ok: | asflicense | 0m 16s | | ASF License check generated no
output? |
| | | 83m 6s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3386/16/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/3386 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient spotbugs checkstyle codespell |
| uname | Linux 883a454c538e
[jira] [Work logged] (HDFS-16213) Flaky test TestFsDatasetImpl#testDnRestartWithHardLink
[
https://issues.apache.org/jira/browse/HDFS-16213?focusedWorklogId=651205=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-651205
]
ASF GitHub Bot logged work on HDFS-16213:
-
Author: ASF GitHub Bot
Created on: 15/Sep/21 16:38
Start Date: 15/Sep/21 16:38
Worklog Time Spent: 10m
Work Description: hadoop-yetus commented on pull request #3386:
URL: https://github.com/apache/hadoop/pull/3386#issuecomment-920182740
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 1m 3s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 2 new or modified test files. |
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 31m 33s | | trunk passed |
| +1 :green_heart: | compile | 1m 22s | | trunk passed with JDK
Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 |
| +1 :green_heart: | compile | 1m 18s | | trunk passed with JDK
Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| +1 :green_heart: | checkstyle | 1m 1s | | trunk passed |
| +1 :green_heart: | mvnsite | 1m 28s | | trunk passed |
| +1 :green_heart: | javadoc | 0m 55s | | trunk passed with JDK
Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 |
| +1 :green_heart: | javadoc | 1m 27s | | trunk passed with JDK
Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| +1 :green_heart: | spotbugs | 3m 4s | | trunk passed |
| +1 :green_heart: | shadedclient | 30m 53s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 1m 18s | | the patch passed |
| +1 :green_heart: | compile | 1m 15s | | the patch passed with JDK
Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 |
| +1 :green_heart: | javac | 1m 15s | | the patch passed |
| +1 :green_heart: | compile | 1m 8s | | the patch passed with JDK
Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| +1 :green_heart: | javac | 1m 8s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| +1 :green_heart: | checkstyle | 0m 51s | |
hadoop-hdfs-project/hadoop-hdfs: The patch generated 0 new + 240 unchanged - 3
fixed = 240 total (was 243) |
| +1 :green_heart: | mvnsite | 1m 15s | | the patch passed |
| +1 :green_heart: | javadoc | 0m 46s | | the patch passed with JDK
Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 |
| +1 :green_heart: | javadoc | 1m 18s | | the patch passed with JDK
Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| +1 :green_heart: | spotbugs | 3m 11s | | the patch passed |
| +1 :green_heart: | shadedclient | 30m 30s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| -1 :x: | unit | 435m 34s |
[/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3386/14/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt)
| hadoop-hdfs in the patch passed. |
| +1 :green_heart: | asflicense | 0m 41s | | The patch does not
generate ASF License warnings. |
| | | 550m 14s | | |
| Reason | Tests |
|---:|:--|
| Failed junit tests | hadoop.hdfs.TestViewDistributedFileSystemContract |
| | hadoop.hdfs.TestSnapshotCommands |
| | hadoop.fs.viewfs.TestViewFileSystemOverloadSchemeWithHdfsScheme |
| | hadoop.hdfs.server.namenode.ha.TestSeveralNameNodes |
| | hadoop.fs.viewfs.TestViewFSOverloadSchemeWithMountTableConfigInHDFS |
| | hadoop.hdfs.TestHDFSFileSystemContract |
| | hadoop.hdfs.web.TestWebHdfsFileSystemContract |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3386/14/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/3386 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient spotbugs checkstyle codespell |
| uname | Linux 206b96093af2 4.15.0-65-generic #74-Ubuntu SMP Tue Sep 17
17:06:04 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 8fcce65ab54f61a8d97adace65ea847df4d69bb0 |
|
[jira] [Work logged] (HDFS-16213) Flaky test TestFsDatasetImpl#testDnRestartWithHardLink
[
https://issues.apache.org/jira/browse/HDFS-16213?focusedWorklogId=650949=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-650949
]
ASF GitHub Bot logged work on HDFS-16213:
-
Author: ASF GitHub Bot
Created on: 15/Sep/21 07:08
Start Date: 15/Sep/21 07:08
Worklog Time Spent: 10m
Work Description: aajisaka commented on a change in pull request #3386:
URL: https://github.com/apache/hadoop/pull/3386#discussion_r708896452
##
File path:
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/BlockPoolSlice.java
##
@@ -1081,4 +1086,11 @@ public static void reInitializeAddReplicaThreadPool() {
addReplicaThreadPool.shutdown();
addReplicaThreadPool = null;
}
+
+ @VisibleForTesting
+ public void setDeleteDuplicateReplicasForTests(
Review comment:
The method can be package-private.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 650949)
Time Spent: 6h 40m (was: 6.5h)
> Flaky test TestFsDatasetImpl#testDnRestartWithHardLink
> --
>
> Key: HDFS-16213
> URL: https://issues.apache.org/jira/browse/HDFS-16213
> Project: Hadoop HDFS
> Issue Type: Sub-task
>Reporter: Viraj Jasani
>Assignee: Viraj Jasani
>Priority: Major
> Labels: pull-request-available
> Time Spent: 6h 40m
> Remaining Estimate: 0h
>
> Failure case:
> [here|https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3359/4/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt]
> {code:java}
> [ERROR]
> testDnRestartWithHardLink(org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.TestFsDatasetImpl)
> Time elapsed: 7.768 s <<< FAILURE![ERROR]
> testDnRestartWithHardLink(org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.TestFsDatasetImpl)
> Time elapsed: 7.768 s <<< FAILURE!java.lang.AssertionError at
> org.junit.Assert.fail(Assert.java:87) at
> org.junit.Assert.assertTrue(Assert.java:42) at
> org.junit.Assert.assertTrue(Assert.java:53) at
> org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.TestFsDatasetImpl.testDnRestartWithHardLink(TestFsDatasetImpl.java:1344)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498) at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59)
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56)
> at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> at
> org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:299)
> at
> org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:293)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266) at
> java.lang.Thread.run(Thread.java:748)
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDFS-16213) Flaky test TestFsDatasetImpl#testDnRestartWithHardLink
[
https://issues.apache.org/jira/browse/HDFS-16213?focusedWorklogId=649960=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-649960
]
ASF GitHub Bot logged work on HDFS-16213:
-
Author: ASF GitHub Bot
Created on: 13/Sep/21 12:01
Start Date: 13/Sep/21 12:01
Worklog Time Spent: 10m
Work Description: hadoop-yetus commented on pull request #3386:
URL: https://github.com/apache/hadoop/pull/3386#issuecomment-918121084
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 43s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 1s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 2 new or modified test files. |
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 31m 7s | | trunk passed |
| +1 :green_heart: | compile | 1m 24s | | trunk passed with JDK
Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 |
| +1 :green_heart: | compile | 1m 19s | | trunk passed with JDK
Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| +1 :green_heart: | checkstyle | 1m 4s | | trunk passed |
| +1 :green_heart: | mvnsite | 1m 27s | | trunk passed |
| +1 :green_heart: | javadoc | 1m 1s | | trunk passed with JDK
Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 |
| +1 :green_heart: | javadoc | 1m 30s | | trunk passed with JDK
Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| +1 :green_heart: | spotbugs | 3m 19s | | trunk passed |
| +1 :green_heart: | shadedclient | 17m 32s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 1m 17s | | the patch passed |
| +1 :green_heart: | compile | 1m 19s | | the patch passed with JDK
Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 |
| +1 :green_heart: | javac | 1m 19s | | the patch passed |
| +1 :green_heart: | compile | 1m 11s | | the patch passed with JDK
Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| +1 :green_heart: | javac | 1m 11s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| +1 :green_heart: | checkstyle | 0m 50s | |
hadoop-hdfs-project/hadoop-hdfs: The patch generated 0 new + 240 unchanged - 3
fixed = 240 total (was 243) |
| +1 :green_heart: | mvnsite | 1m 13s | | the patch passed |
| +1 :green_heart: | javadoc | 0m 47s | | the patch passed with JDK
Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 |
| +1 :green_heart: | javadoc | 1m 18s | | the patch passed with JDK
Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| +1 :green_heart: | spotbugs | 3m 12s | | the patch passed |
| +1 :green_heart: | shadedclient | 16m 51s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 243m 49s | | hadoop-hdfs in the patch
passed. |
| +1 :green_heart: | asflicense | 0m 46s | | The patch does not
generate ASF License warnings. |
| | | 330m 47s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3386/13/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/3386 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient spotbugs checkstyle codespell |
| uname | Linux e29d417400b5 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9
23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / f1f832b5116f6d356f83436f788f1080cdd0790a |
| Default Java | Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| Multi-JDK versions |
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
/usr/lib/jvm/java-8-openjdk-amd64:Private
Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| Test Results |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3386/13/testReport/ |
| Max. process+thread count | 3616 (vs. ulimit of 5500) |
| modules | C: hadoop-hdfs-project/hadoop-hdfs U:
hadoop-hdfs-project/hadoop-hdfs |
| Console output |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3386/13/console |
| versions | git=2.25.1 maven=3.6.3 spotbugs=4.2.2 |
[jira] [Work logged] (HDFS-16213) Flaky test TestFsDatasetImpl#testDnRestartWithHardLink
[
https://issues.apache.org/jira/browse/HDFS-16213?focusedWorklogId=649839=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-649839
]
ASF GitHub Bot logged work on HDFS-16213:
-
Author: ASF GitHub Bot
Created on: 13/Sep/21 06:32
Start Date: 13/Sep/21 06:32
Worklog Time Spent: 10m
Work Description: virajjasani commented on a change in pull request #3386:
URL: https://github.com/apache/hadoop/pull/3386#discussion_r707040398
##
File path:
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/FsDatasetImpl.java
##
@@ -1050,8 +1050,10 @@ static File moveBlockFiles(Block b, ReplicaInfo
replicaInfo, File destdir)
File dstFile)
throws IOException {
// Create parent folder if not exists.
-srcReplica.getFileIoProvider()
+boolean isDirCreated = srcReplica.getFileIoProvider()
.mkdirs(srcReplica.getVolume(), dstFile.getParentFile());
+LOG.trace("Dir creation of {} on volume {} {}", dstFile.getParentFile(),
Review comment:
Updated to use already extracted objects so that only if TRACE level is
enabled, those objects' toString() will be extracted. Without TRACE level being
enabled, placeholder object's toString() won't be called so perf penalty (if
applicable) will be only restricted to TRACE level logging. Does this sound
good?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 649839)
Time Spent: 6h 20m (was: 6h 10m)
> Flaky test TestFsDatasetImpl#testDnRestartWithHardLink
> --
>
> Key: HDFS-16213
> URL: https://issues.apache.org/jira/browse/HDFS-16213
> Project: Hadoop HDFS
> Issue Type: Bug
>Reporter: Viraj Jasani
>Assignee: Viraj Jasani
>Priority: Major
> Labels: pull-request-available
> Time Spent: 6h 20m
> Remaining Estimate: 0h
>
> Failure case:
> [here|https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3359/4/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt]
> {code:java}
> [ERROR]
> testDnRestartWithHardLink(org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.TestFsDatasetImpl)
> Time elapsed: 7.768 s <<< FAILURE![ERROR]
> testDnRestartWithHardLink(org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.TestFsDatasetImpl)
> Time elapsed: 7.768 s <<< FAILURE!java.lang.AssertionError at
> org.junit.Assert.fail(Assert.java:87) at
> org.junit.Assert.assertTrue(Assert.java:42) at
> org.junit.Assert.assertTrue(Assert.java:53) at
> org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.TestFsDatasetImpl.testDnRestartWithHardLink(TestFsDatasetImpl.java:1344)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498) at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59)
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56)
> at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> at
> org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:299)
> at
> org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:293)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266) at
> java.lang.Thread.run(Thread.java:748)
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDFS-16213) Flaky test TestFsDatasetImpl#testDnRestartWithHardLink
[
https://issues.apache.org/jira/browse/HDFS-16213?focusedWorklogId=649828=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-649828
]
ASF GitHub Bot logged work on HDFS-16213:
-
Author: ASF GitHub Bot
Created on: 13/Sep/21 05:36
Start Date: 13/Sep/21 05:36
Worklog Time Spent: 10m
Work Description: jojochuang commented on a change in pull request #3386:
URL: https://github.com/apache/hadoop/pull/3386#discussion_r707017464
##
File path:
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/FsDatasetImpl.java
##
@@ -1050,8 +1050,10 @@ static File moveBlockFiles(Block b, ReplicaInfo
replicaInfo, File destdir)
File dstFile)
throws IOException {
// Create parent folder if not exists.
-srcReplica.getFileIoProvider()
+boolean isDirCreated = srcReplica.getFileIoProvider()
.mkdirs(srcReplica.getVolume(), dstFile.getParentFile());
+LOG.trace("Dir creation of {} on volume {} {}", dstFile.getParentFile(),
Review comment:
File.getParentFile() allocates and returns a new string. It's not the
kind of parameter you want to log only at TRACE level.
I doubt it'd have a meaningful performance penalty though because the actual
file system hard link probably spend much more time.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 649828)
Time Spent: 6h 10m (was: 6h)
> Flaky test TestFsDatasetImpl#testDnRestartWithHardLink
> --
>
> Key: HDFS-16213
> URL: https://issues.apache.org/jira/browse/HDFS-16213
> Project: Hadoop HDFS
> Issue Type: Bug
>Reporter: Viraj Jasani
>Assignee: Viraj Jasani
>Priority: Major
> Labels: pull-request-available
> Time Spent: 6h 10m
> Remaining Estimate: 0h
>
> Failure case:
> [here|https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3359/4/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt]
> {code:java}
> [ERROR]
> testDnRestartWithHardLink(org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.TestFsDatasetImpl)
> Time elapsed: 7.768 s <<< FAILURE![ERROR]
> testDnRestartWithHardLink(org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.TestFsDatasetImpl)
> Time elapsed: 7.768 s <<< FAILURE!java.lang.AssertionError at
> org.junit.Assert.fail(Assert.java:87) at
> org.junit.Assert.assertTrue(Assert.java:42) at
> org.junit.Assert.assertTrue(Assert.java:53) at
> org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.TestFsDatasetImpl.testDnRestartWithHardLink(TestFsDatasetImpl.java:1344)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498) at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59)
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56)
> at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> at
> org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:299)
> at
> org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:293)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266) at
> java.lang.Thread.run(Thread.java:748)
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDFS-16213) Flaky test TestFsDatasetImpl#testDnRestartWithHardLink
[
https://issues.apache.org/jira/browse/HDFS-16213?focusedWorklogId=649797=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-649797
]
ASF GitHub Bot logged work on HDFS-16213:
-
Author: ASF GitHub Bot
Created on: 12/Sep/21 23:42
Start Date: 12/Sep/21 23:42
Worklog Time Spent: 10m
Work Description: hadoop-yetus commented on pull request #3386:
URL: https://github.com/apache/hadoop/pull/3386#issuecomment-917734240
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 46s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 1s | | codespell was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 2 new or modified test files. |
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 32m 19s | | trunk passed |
| +1 :green_heart: | compile | 1m 22s | | trunk passed with JDK
Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 |
| +1 :green_heart: | compile | 1m 16s | | trunk passed with JDK
Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| +1 :green_heart: | checkstyle | 1m 2s | | trunk passed |
| +1 :green_heart: | mvnsite | 1m 24s | | trunk passed |
| +1 :green_heart: | javadoc | 0m 57s | | trunk passed with JDK
Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 |
| +1 :green_heart: | javadoc | 1m 24s | | trunk passed with JDK
Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| +1 :green_heart: | spotbugs | 3m 12s | | trunk passed |
| +1 :green_heart: | shadedclient | 16m 33s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 1m 12s | | the patch passed |
| +1 :green_heart: | compile | 1m 13s | | the patch passed with JDK
Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 |
| +1 :green_heart: | javac | 1m 13s | | the patch passed |
| +1 :green_heart: | compile | 1m 11s | | the patch passed with JDK
Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| +1 :green_heart: | javac | 1m 11s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| +1 :green_heart: | checkstyle | 0m 54s | |
hadoop-hdfs-project/hadoop-hdfs: The patch generated 0 new + 240 unchanged - 3
fixed = 240 total (was 243) |
| +1 :green_heart: | mvnsite | 1m 15s | | the patch passed |
| +1 :green_heart: | javadoc | 0m 47s | | the patch passed with JDK
Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 |
| +1 :green_heart: | javadoc | 1m 20s | | the patch passed with JDK
Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| +1 :green_heart: | spotbugs | 3m 9s | | the patch passed |
| +1 :green_heart: | shadedclient | 16m 22s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 229m 10s | | hadoop-hdfs in the patch
passed. |
| +1 :green_heart: | asflicense | 0m 46s | | The patch does not
generate ASF License warnings. |
| | | 315m 32s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3386/12/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/3386 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient spotbugs checkstyle codespell |
| uname | Linux f91c3af3b149 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9
23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 8e1eab0b1cd4dddbddc9cea02a657c87bab2b9bb |
| Default Java | Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| Multi-JDK versions |
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
/usr/lib/jvm/java-8-openjdk-amd64:Private
Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| Test Results |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3386/12/testReport/ |
| Max. process+thread count | 3406 (vs. ulimit of 5500) |
| modules | C: hadoop-hdfs-project/hadoop-hdfs U:
hadoop-hdfs-project/hadoop-hdfs |
| Console output |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3386/12/console |
| versions | git=2.25.1 maven=3.6.3 spotbugs=4.2.2 |
[jira] [Work logged] (HDFS-16213) Flaky test TestFsDatasetImpl#testDnRestartWithHardLink
[
https://issues.apache.org/jira/browse/HDFS-16213?focusedWorklogId=649723=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-649723
]
ASF GitHub Bot logged work on HDFS-16213:
-
Author: ASF GitHub Bot
Created on: 12/Sep/21 18:16
Start Date: 12/Sep/21 18:16
Worklog Time Spent: 10m
Work Description: virajjasani commented on a change in pull request #3386:
URL: https://github.com/apache/hadoop/pull/3386#discussion_r706874063
##
File path:
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/FileIoProvider.java
##
@@ -513,6 +513,7 @@ public boolean fullyDelete(@Nullable FsVolumeSpi volume,
File dir) {
try {
faultInjectorEventHook.beforeMetadataOp(volume, DELETE);
boolean deleted = FileUtil.fullyDelete(dir);
+ LOG.trace("Deletion of dir {} {}", dir, deleted ? "succeeded" :
"failed");
Review comment:
While debugging this issue, I felt that a log line here would make it
easier for operator to confirm the output of Delete operation, and trace log
level is sufficient on the other hand, hence I thought of keeping this log line.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 649723)
Time Spent: 5h 50m (was: 5h 40m)
> Flaky test TestFsDatasetImpl#testDnRestartWithHardLink
> --
>
> Key: HDFS-16213
> URL: https://issues.apache.org/jira/browse/HDFS-16213
> Project: Hadoop HDFS
> Issue Type: Bug
>Reporter: Viraj Jasani
>Assignee: Viraj Jasani
>Priority: Major
> Labels: pull-request-available
> Time Spent: 5h 50m
> Remaining Estimate: 0h
>
> Failure case:
> [here|https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3359/4/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt]
> {code:java}
> [ERROR]
> testDnRestartWithHardLink(org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.TestFsDatasetImpl)
> Time elapsed: 7.768 s <<< FAILURE![ERROR]
> testDnRestartWithHardLink(org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.TestFsDatasetImpl)
> Time elapsed: 7.768 s <<< FAILURE!java.lang.AssertionError at
> org.junit.Assert.fail(Assert.java:87) at
> org.junit.Assert.assertTrue(Assert.java:42) at
> org.junit.Assert.assertTrue(Assert.java:53) at
> org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.TestFsDatasetImpl.testDnRestartWithHardLink(TestFsDatasetImpl.java:1344)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498) at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59)
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56)
> at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> at
> org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:299)
> at
> org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:293)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266) at
> java.lang.Thread.run(Thread.java:748)
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDFS-16213) Flaky test TestFsDatasetImpl#testDnRestartWithHardLink
[
https://issues.apache.org/jira/browse/HDFS-16213?focusedWorklogId=649722=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-649722
]
ASF GitHub Bot logged work on HDFS-16213:
-
Author: ASF GitHub Bot
Created on: 12/Sep/21 18:13
Start Date: 12/Sep/21 18:13
Worklog Time Spent: 10m
Work Description: LeonGao91 commented on pull request #3386:
URL: https://github.com/apache/hadoop/pull/3386#issuecomment-917684061
thanks @virajjasani , looks good to me. Just leaving few nit comments.
@ferhui @ayushtkn Could you please take a second look.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 649722)
Time Spent: 5h 40m (was: 5.5h)
> Flaky test TestFsDatasetImpl#testDnRestartWithHardLink
> --
>
> Key: HDFS-16213
> URL: https://issues.apache.org/jira/browse/HDFS-16213
> Project: Hadoop HDFS
> Issue Type: Bug
>Reporter: Viraj Jasani
>Assignee: Viraj Jasani
>Priority: Major
> Labels: pull-request-available
> Time Spent: 5h 40m
> Remaining Estimate: 0h
>
> Failure case:
> [here|https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3359/4/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt]
> {code:java}
> [ERROR]
> testDnRestartWithHardLink(org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.TestFsDatasetImpl)
> Time elapsed: 7.768 s <<< FAILURE![ERROR]
> testDnRestartWithHardLink(org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.TestFsDatasetImpl)
> Time elapsed: 7.768 s <<< FAILURE!java.lang.AssertionError at
> org.junit.Assert.fail(Assert.java:87) at
> org.junit.Assert.assertTrue(Assert.java:42) at
> org.junit.Assert.assertTrue(Assert.java:53) at
> org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.TestFsDatasetImpl.testDnRestartWithHardLink(TestFsDatasetImpl.java:1344)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498) at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59)
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56)
> at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> at
> org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:299)
> at
> org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:293)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266) at
> java.lang.Thread.run(Thread.java:748)
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDFS-16213) Flaky test TestFsDatasetImpl#testDnRestartWithHardLink
[
https://issues.apache.org/jira/browse/HDFS-16213?focusedWorklogId=649721=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-649721
]
ASF GitHub Bot logged work on HDFS-16213:
-
Author: ASF GitHub Bot
Created on: 12/Sep/21 18:09
Start Date: 12/Sep/21 18:09
Worklog Time Spent: 10m
Work Description: LeonGao91 commented on a change in pull request #3386:
URL: https://github.com/apache/hadoop/pull/3386#discussion_r706873351
##
File path:
hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/TestFsDatasetImpl.java
##
@@ -1307,35 +1310,42 @@ public void testDnRestartWithHardLinkInTmp() {
* DiskScanner should clean up the hardlink correctly.
*/
@Test(timeout = 3)
- public void testDnRestartWithHardLink() {
+ public void testDnRestartWithHardLink() throws Exception {
MiniDFSCluster cluster = null;
try {
conf.setBoolean(DFSConfigKeys
.DFS_DATANODE_ALLOW_SAME_DISK_TIERING, true);
conf.setDouble(DFSConfigKeys
.DFS_DATANODE_RESERVE_FOR_ARCHIVE_DEFAULT_PERCENTAGE, 0.5);
+ conf.setBoolean(DFSConfigKeys.DFS_DATANODE_DUPLICATE_REPLICA_DELETION,
Review comment:
Maybe add 1 line on why we added this? Something like:
"This is to guarantee the corner case that datanode restart in the middle of
the block movement may leave uncleaned hardlink."
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 649721)
Time Spent: 5.5h (was: 5h 20m)
> Flaky test TestFsDatasetImpl#testDnRestartWithHardLink
> --
>
> Key: HDFS-16213
> URL: https://issues.apache.org/jira/browse/HDFS-16213
> Project: Hadoop HDFS
> Issue Type: Bug
>Reporter: Viraj Jasani
>Assignee: Viraj Jasani
>Priority: Major
> Labels: pull-request-available
> Time Spent: 5.5h
> Remaining Estimate: 0h
>
> Failure case:
> [here|https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3359/4/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt]
> {code:java}
> [ERROR]
> testDnRestartWithHardLink(org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.TestFsDatasetImpl)
> Time elapsed: 7.768 s <<< FAILURE![ERROR]
> testDnRestartWithHardLink(org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.TestFsDatasetImpl)
> Time elapsed: 7.768 s <<< FAILURE!java.lang.AssertionError at
> org.junit.Assert.fail(Assert.java:87) at
> org.junit.Assert.assertTrue(Assert.java:42) at
> org.junit.Assert.assertTrue(Assert.java:53) at
> org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.TestFsDatasetImpl.testDnRestartWithHardLink(TestFsDatasetImpl.java:1344)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498) at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59)
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56)
> at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> at
> org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:299)
> at
> org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:293)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266) at
> java.lang.Thread.run(Thread.java:748)
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDFS-16213) Flaky test TestFsDatasetImpl#testDnRestartWithHardLink
[
https://issues.apache.org/jira/browse/HDFS-16213?focusedWorklogId=649720=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-649720
]
ASF GitHub Bot logged work on HDFS-16213:
-
Author: ASF GitHub Bot
Created on: 12/Sep/21 18:04
Start Date: 12/Sep/21 18:04
Worklog Time Spent: 10m
Work Description: LeonGao91 commented on a change in pull request #3386:
URL: https://github.com/apache/hadoop/pull/3386#discussion_r706872610
##
File path:
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/FileIoProvider.java
##
@@ -513,6 +513,7 @@ public boolean fullyDelete(@Nullable FsVolumeSpi volume,
File dir) {
try {
faultInjectorEventHook.beforeMetadataOp(volume, DELETE);
boolean deleted = FileUtil.fullyDelete(dir);
+ LOG.trace("Deletion of dir {} {}", dir, deleted ? "succeeded" :
"failed");
Review comment:
Do we need to add the trace logs as it is unrelated to the fix?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 649720)
Time Spent: 5h 20m (was: 5h 10m)
> Flaky test TestFsDatasetImpl#testDnRestartWithHardLink
> --
>
> Key: HDFS-16213
> URL: https://issues.apache.org/jira/browse/HDFS-16213
> Project: Hadoop HDFS
> Issue Type: Bug
>Reporter: Viraj Jasani
>Assignee: Viraj Jasani
>Priority: Major
> Labels: pull-request-available
> Time Spent: 5h 20m
> Remaining Estimate: 0h
>
> Failure case:
> [here|https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3359/4/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt]
> {code:java}
> [ERROR]
> testDnRestartWithHardLink(org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.TestFsDatasetImpl)
> Time elapsed: 7.768 s <<< FAILURE![ERROR]
> testDnRestartWithHardLink(org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.TestFsDatasetImpl)
> Time elapsed: 7.768 s <<< FAILURE!java.lang.AssertionError at
> org.junit.Assert.fail(Assert.java:87) at
> org.junit.Assert.assertTrue(Assert.java:42) at
> org.junit.Assert.assertTrue(Assert.java:53) at
> org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.TestFsDatasetImpl.testDnRestartWithHardLink(TestFsDatasetImpl.java:1344)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498) at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59)
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56)
> at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> at
> org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:299)
> at
> org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:293)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266) at
> java.lang.Thread.run(Thread.java:748)
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDFS-16213) Flaky test TestFsDatasetImpl#testDnRestartWithHardLink
[
https://issues.apache.org/jira/browse/HDFS-16213?focusedWorklogId=649719=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-649719
]
ASF GitHub Bot logged work on HDFS-16213:
-
Author: ASF GitHub Bot
Created on: 12/Sep/21 17:17
Start Date: 12/Sep/21 17:17
Worklog Time Spent: 10m
Work Description: hadoop-yetus commented on pull request #3386:
URL: https://github.com/apache/hadoop/pull/3386#issuecomment-917675592
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 59s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 2 new or modified test files. |
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 33m 17s | | trunk passed |
| +1 :green_heart: | compile | 1m 23s | | trunk passed with JDK
Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 |
| +1 :green_heart: | compile | 1m 15s | | trunk passed with JDK
Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| +1 :green_heart: | checkstyle | 1m 0s | | trunk passed |
| +1 :green_heart: | mvnsite | 1m 22s | | trunk passed |
| +1 :green_heart: | javadoc | 0m 56s | | trunk passed with JDK
Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 |
| +1 :green_heart: | javadoc | 1m 26s | | trunk passed with JDK
Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| +1 :green_heart: | spotbugs | 3m 17s | | trunk passed |
| +1 :green_heart: | shadedclient | 18m 52s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 1m 15s | | the patch passed |
| +1 :green_heart: | compile | 1m 18s | | the patch passed with JDK
Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 |
| +1 :green_heart: | javac | 1m 18s | | the patch passed |
| +1 :green_heart: | compile | 1m 11s | | the patch passed with JDK
Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| +1 :green_heart: | javac | 1m 11s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| +1 :green_heart: | checkstyle | 0m 54s | |
hadoop-hdfs-project/hadoop-hdfs: The patch generated 0 new + 240 unchanged - 3
fixed = 240 total (was 243) |
| +1 :green_heart: | mvnsite | 1m 15s | | the patch passed |
| +1 :green_heart: | javadoc | 0m 48s | | the patch passed with JDK
Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 |
| +1 :green_heart: | javadoc | 1m 23s | | the patch passed with JDK
Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| +1 :green_heart: | spotbugs | 3m 20s | | the patch passed |
| +1 :green_heart: | shadedclient | 19m 2s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 350m 48s | | hadoop-hdfs in the patch
passed. |
| +1 :green_heart: | asflicense | 0m 39s | | The patch does not
generate ASF License warnings. |
| | | 443m 7s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3386/11/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/3386 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient spotbugs checkstyle codespell |
| uname | Linux 9757ee312e51 4.15.0-147-generic #151-Ubuntu SMP Fri Jun 18
19:21:19 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / cf7dc937fb00691f74ca7a54292735d87dd8b7af |
| Default Java | Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| Multi-JDK versions |
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
/usr/lib/jvm/java-8-openjdk-amd64:Private
Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| Test Results |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3386/11/testReport/ |
| Max. process+thread count | 2140 (vs. ulimit of 5500) |
| modules | C: hadoop-hdfs-project/hadoop-hdfs U:
hadoop-hdfs-project/hadoop-hdfs |
| Console output |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3386/11/console |
| versions | git=2.25.1 maven=3.6.3 spotbugs=4.2.2 |
[jira] [Work logged] (HDFS-16213) Flaky test TestFsDatasetImpl#testDnRestartWithHardLink
[
https://issues.apache.org/jira/browse/HDFS-16213?focusedWorklogId=649716=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-649716
]
ASF GitHub Bot logged work on HDFS-16213:
-
Author: ASF GitHub Bot
Created on: 12/Sep/21 15:06
Start Date: 12/Sep/21 15:06
Worklog Time Spent: 10m
Work Description: hadoop-yetus commented on pull request #3386:
URL: https://github.com/apache/hadoop/pull/3386#issuecomment-917653913
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 52s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 1s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 2 new or modified test files. |
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 30m 41s | | trunk passed |
| +1 :green_heart: | compile | 1m 22s | | trunk passed with JDK
Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 |
| +1 :green_heart: | compile | 1m 17s | | trunk passed with JDK
Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| +1 :green_heart: | checkstyle | 1m 1s | | trunk passed |
| +1 :green_heart: | mvnsite | 1m 28s | | trunk passed |
| +1 :green_heart: | javadoc | 0m 56s | | trunk passed with JDK
Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 |
| +1 :green_heart: | javadoc | 1m 28s | | trunk passed with JDK
Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| +1 :green_heart: | spotbugs | 3m 5s | | trunk passed |
| +1 :green_heart: | shadedclient | 16m 23s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 1m 10s | | the patch passed |
| +1 :green_heart: | compile | 1m 14s | | the patch passed with JDK
Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 |
| +1 :green_heart: | javac | 1m 14s | | the patch passed |
| +1 :green_heart: | compile | 1m 10s | | the patch passed with JDK
Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| +1 :green_heart: | javac | 1m 10s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| +1 :green_heart: | checkstyle | 0m 54s | |
hadoop-hdfs-project/hadoop-hdfs: The patch generated 0 new + 240 unchanged - 3
fixed = 240 total (was 243) |
| +1 :green_heart: | mvnsite | 1m 14s | | the patch passed |
| +1 :green_heart: | javadoc | 0m 46s | | the patch passed with JDK
Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 |
| +1 :green_heart: | javadoc | 1m 20s | | the patch passed with JDK
Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| +1 :green_heart: | spotbugs | 3m 7s | | the patch passed |
| +1 :green_heart: | shadedclient | 15m 59s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 231m 33s | | hadoop-hdfs in the patch
passed. |
| +1 :green_heart: | asflicense | 0m 48s | | The patch does not
generate ASF License warnings. |
| | | 316m 1s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3386/9/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/3386 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient spotbugs checkstyle codespell |
| uname | Linux 935d6934fa36 4.15.0-151-generic #157-Ubuntu SMP Fri Jul 9
23:07:57 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / cf7dc937fb00691f74ca7a54292735d87dd8b7af |
| Default Java | Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| Multi-JDK versions |
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
/usr/lib/jvm/java-8-openjdk-amd64:Private
Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| Test Results |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3386/9/testReport/ |
| Max. process+thread count | 2938 (vs. ulimit of 5500) |
| modules | C: hadoop-hdfs-project/hadoop-hdfs U:
hadoop-hdfs-project/hadoop-hdfs |
| Console output |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3386/9/console |
| versions | git=2.25.1 maven=3.6.3 spotbugs=4.2.2 |
[jira] [Work logged] (HDFS-16213) Flaky test TestFsDatasetImpl#testDnRestartWithHardLink
[
https://issues.apache.org/jira/browse/HDFS-16213?focusedWorklogId=649715=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-649715
]
ASF GitHub Bot logged work on HDFS-16213:
-
Author: ASF GitHub Bot
Created on: 12/Sep/21 15:02
Start Date: 12/Sep/21 15:02
Worklog Time Spent: 10m
Work Description: hadoop-yetus commented on pull request #3386:
URL: https://github.com/apache/hadoop/pull/3386#issuecomment-917653119
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 45s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 1s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 2 new or modified test files. |
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 30m 56s | | trunk passed |
| +1 :green_heart: | compile | 1m 24s | | trunk passed with JDK
Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 |
| +1 :green_heart: | compile | 1m 16s | | trunk passed with JDK
Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| +1 :green_heart: | checkstyle | 1m 3s | | trunk passed |
| +1 :green_heart: | mvnsite | 1m 22s | | trunk passed |
| +1 :green_heart: | javadoc | 0m 57s | | trunk passed with JDK
Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 |
| +1 :green_heart: | javadoc | 1m 29s | | trunk passed with JDK
Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| +1 :green_heart: | spotbugs | 3m 10s | | trunk passed |
| +1 :green_heart: | shadedclient | 16m 27s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 1m 14s | | the patch passed |
| +1 :green_heart: | compile | 1m 14s | | the patch passed with JDK
Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 |
| +1 :green_heart: | javac | 1m 14s | | the patch passed |
| +1 :green_heart: | compile | 1m 9s | | the patch passed with JDK
Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| +1 :green_heart: | javac | 1m 9s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| +1 :green_heart: | checkstyle | 0m 52s | |
hadoop-hdfs-project/hadoop-hdfs: The patch generated 0 new + 240 unchanged - 3
fixed = 240 total (was 243) |
| +1 :green_heart: | mvnsite | 1m 14s | | the patch passed |
| +1 :green_heart: | javadoc | 0m 46s | | the patch passed with JDK
Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 |
| +1 :green_heart: | javadoc | 1m 21s | | the patch passed with JDK
Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| +1 :green_heart: | spotbugs | 3m 6s | | the patch passed |
| +1 :green_heart: | shadedclient | 16m 14s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 231m 9s | | hadoop-hdfs in the patch
passed. |
| +1 :green_heart: | asflicense | 0m 46s | | The patch does not
generate ASF License warnings. |
| | | 316m 3s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3386/8/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/3386 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient spotbugs checkstyle codespell |
| uname | Linux eb4ed62f2b49 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9
23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 31cd896568bcd3f95e3cd6741dc2d4563ea8ab85 |
| Default Java | Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| Multi-JDK versions |
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
/usr/lib/jvm/java-8-openjdk-amd64:Private
Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| Test Results |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3386/8/testReport/ |
| Max. process+thread count | 3081 (vs. ulimit of 5500) |
| modules | C: hadoop-hdfs-project/hadoop-hdfs U:
hadoop-hdfs-project/hadoop-hdfs |
| Console output |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3386/8/console |
| versions | git=2.25.1 maven=3.6.3 spotbugs=4.2.2 |
[jira] [Work logged] (HDFS-16213) Flaky test TestFsDatasetImpl#testDnRestartWithHardLink
[
https://issues.apache.org/jira/browse/HDFS-16213?focusedWorklogId=649674=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-649674
]
ASF GitHub Bot logged work on HDFS-16213:
-
Author: ASF GitHub Bot
Created on: 12/Sep/21 08:00
Start Date: 12/Sep/21 08:00
Worklog Time Spent: 10m
Work Description: virajjasani commented on pull request #3386:
URL: https://github.com/apache/hadoop/pull/3386#issuecomment-917580384
> Depends on the sequence of loading cache and constructing volume map,
sometimes hardlink will just be removed by datanode restart.
Exactly
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 649674)
Time Spent: 4h 40m (was: 4.5h)
> Flaky test TestFsDatasetImpl#testDnRestartWithHardLink
> --
>
> Key: HDFS-16213
> URL: https://issues.apache.org/jira/browse/HDFS-16213
> Project: Hadoop HDFS
> Issue Type: Bug
>Reporter: Viraj Jasani
>Assignee: Viraj Jasani
>Priority: Major
> Labels: pull-request-available
> Time Spent: 4h 40m
> Remaining Estimate: 0h
>
> Failure case:
> [here|https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3359/4/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt]
> {code:java}
> [ERROR]
> testDnRestartWithHardLink(org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.TestFsDatasetImpl)
> Time elapsed: 7.768 s <<< FAILURE![ERROR]
> testDnRestartWithHardLink(org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.TestFsDatasetImpl)
> Time elapsed: 7.768 s <<< FAILURE!java.lang.AssertionError at
> org.junit.Assert.fail(Assert.java:87) at
> org.junit.Assert.assertTrue(Assert.java:42) at
> org.junit.Assert.assertTrue(Assert.java:53) at
> org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.TestFsDatasetImpl.testDnRestartWithHardLink(TestFsDatasetImpl.java:1344)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498) at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59)
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56)
> at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> at
> org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:299)
> at
> org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:293)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266) at
> java.lang.Thread.run(Thread.java:748)
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDFS-16213) Flaky test TestFsDatasetImpl#testDnRestartWithHardLink
[
https://issues.apache.org/jira/browse/HDFS-16213?focusedWorklogId=649673=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-649673
]
ASF GitHub Bot logged work on HDFS-16213:
-
Author: ASF GitHub Bot
Created on: 12/Sep/21 08:00
Start Date: 12/Sep/21 08:00
Worklog Time Spent: 10m
Work Description: virajjasani commented on pull request #3386:
URL: https://github.com/apache/hadoop/pull/3386#issuecomment-917580278
I see, yes this seems finer-grained control over the target we want to
achieve, let me spend some more time with this and will make changes
accordingly after verifying. Thanks @LeonGao91 for providing your viewpoint on
this!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 649673)
Time Spent: 4.5h (was: 4h 20m)
> Flaky test TestFsDatasetImpl#testDnRestartWithHardLink
> --
>
> Key: HDFS-16213
> URL: https://issues.apache.org/jira/browse/HDFS-16213
> Project: Hadoop HDFS
> Issue Type: Bug
>Reporter: Viraj Jasani
>Assignee: Viraj Jasani
>Priority: Major
> Labels: pull-request-available
> Time Spent: 4.5h
> Remaining Estimate: 0h
>
> Failure case:
> [here|https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3359/4/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt]
> {code:java}
> [ERROR]
> testDnRestartWithHardLink(org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.TestFsDatasetImpl)
> Time elapsed: 7.768 s <<< FAILURE![ERROR]
> testDnRestartWithHardLink(org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.TestFsDatasetImpl)
> Time elapsed: 7.768 s <<< FAILURE!java.lang.AssertionError at
> org.junit.Assert.fail(Assert.java:87) at
> org.junit.Assert.assertTrue(Assert.java:42) at
> org.junit.Assert.assertTrue(Assert.java:53) at
> org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.TestFsDatasetImpl.testDnRestartWithHardLink(TestFsDatasetImpl.java:1344)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498) at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59)
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56)
> at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> at
> org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:299)
> at
> org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:293)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266) at
> java.lang.Thread.run(Thread.java:748)
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDFS-16213) Flaky test TestFsDatasetImpl#testDnRestartWithHardLink
[
https://issues.apache.org/jira/browse/HDFS-16213?focusedWorklogId=649668=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-649668
]
ASF GitHub Bot logged work on HDFS-16213:
-
Author: ASF GitHub Bot
Created on: 12/Sep/21 07:33
Start Date: 12/Sep/21 07:33
Worklog Time Spent: 10m
Work Description: LeonGao91 commented on pull request #3386:
URL: https://github.com/apache/hadoop/pull/3386#issuecomment-917576109
@virajjasani Yes, I just did some more research. Depends on the sequence of
loading cache and constructing volume map, sometimes hardlink will just be
removed by datanode restart. So the corner case that needs to be tested here is
not guaranteed.
I think a less disrupting way to fix this is by controlling
"deleteDuplicateReplicas" value in the BlockPoolSlice. We can disable this
until the DirectoryScanner kicks in. Therefore we only need to add a
VisibleForTesting method in BlockPoolSlice
```
@VisibleForTesting
void setDeleteDuplicateReplicasForTesting(boolean value) {
deleteDuplicateReplicas = value;
}
```
And in the test, we can do this
```
// In the beginning
conf.setBoolean(
DFSConfigKeys.DFS_DATANODE_DUPLICATE_REPLICA_DELETION,
false);
...
// When testing DirectoryScanner should clean up the hardlink
DirectoryScanner scanner = new DirectoryScanner(
cluster.getDataNodes().get(0).getFSDataset(), conf);
FsVolumeImpl volume = (FsVolumeImpl)
cluster.getDataNodes().get(0).getFSDataset().getFsVolumeReferences().get(0);
volume.getBlockPoolSlice(volume.getBlockPoolList()[0]).setDeleteDuplicateReplicasForTesting(true);
scanner.start();
scanner.run();
```
What do you think?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 649668)
Time Spent: 4h 20m (was: 4h 10m)
> Flaky test TestFsDatasetImpl#testDnRestartWithHardLink
> --
>
> Key: HDFS-16213
> URL: https://issues.apache.org/jira/browse/HDFS-16213
> Project: Hadoop HDFS
> Issue Type: Bug
>Reporter: Viraj Jasani
>Assignee: Viraj Jasani
>Priority: Major
> Labels: pull-request-available
> Time Spent: 4h 20m
> Remaining Estimate: 0h
>
> Failure case:
> [here|https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3359/4/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt]
> {code:java}
> [ERROR]
> testDnRestartWithHardLink(org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.TestFsDatasetImpl)
> Time elapsed: 7.768 s <<< FAILURE![ERROR]
> testDnRestartWithHardLink(org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.TestFsDatasetImpl)
> Time elapsed: 7.768 s <<< FAILURE!java.lang.AssertionError at
> org.junit.Assert.fail(Assert.java:87) at
> org.junit.Assert.assertTrue(Assert.java:42) at
> org.junit.Assert.assertTrue(Assert.java:53) at
> org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.TestFsDatasetImpl.testDnRestartWithHardLink(TestFsDatasetImpl.java:1344)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498) at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59)
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56)
> at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> at
> org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:299)
> at
> org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:293)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266) at
> java.lang.Thread.run(Thread.java:748)
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDFS-16213) Flaky test TestFsDatasetImpl#testDnRestartWithHardLink
[
https://issues.apache.org/jira/browse/HDFS-16213?focusedWorklogId=649602=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-649602
]
ASF GitHub Bot logged work on HDFS-16213:
-
Author: ASF GitHub Bot
Created on: 11/Sep/21 17:46
Start Date: 11/Sep/21 17:46
Worklog Time Spent: 10m
Work Description: virajjasani commented on pull request #3386:
URL: https://github.com/apache/hadoop/pull/3386#issuecomment-917445820
Two additional recent flaky instances of this failure:
https://user-images.githubusercontent.com/34790606/132956632-7fab5f45-b186-4e54-b204-6c15a2a770f7.png;>
https://user-images.githubusercontent.com/34790606/132956634-0b58fe42-1cac-4f16-904f-f496543d99cf.png;>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 649602)
Time Spent: 4h 10m (was: 4h)
> Flaky test TestFsDatasetImpl#testDnRestartWithHardLink
> --
>
> Key: HDFS-16213
> URL: https://issues.apache.org/jira/browse/HDFS-16213
> Project: Hadoop HDFS
> Issue Type: Bug
>Reporter: Viraj Jasani
>Assignee: Viraj Jasani
>Priority: Major
> Labels: pull-request-available
> Time Spent: 4h 10m
> Remaining Estimate: 0h
>
> Failure case:
> [here|https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3359/4/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt]
> {code:java}
> [ERROR]
> testDnRestartWithHardLink(org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.TestFsDatasetImpl)
> Time elapsed: 7.768 s <<< FAILURE![ERROR]
> testDnRestartWithHardLink(org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.TestFsDatasetImpl)
> Time elapsed: 7.768 s <<< FAILURE!java.lang.AssertionError at
> org.junit.Assert.fail(Assert.java:87) at
> org.junit.Assert.assertTrue(Assert.java:42) at
> org.junit.Assert.assertTrue(Assert.java:53) at
> org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.TestFsDatasetImpl.testDnRestartWithHardLink(TestFsDatasetImpl.java:1344)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498) at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59)
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56)
> at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> at
> org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:299)
> at
> org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:293)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266) at
> java.lang.Thread.run(Thread.java:748)
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDFS-16213) Flaky test TestFsDatasetImpl#testDnRestartWithHardLink
[
https://issues.apache.org/jira/browse/HDFS-16213?focusedWorklogId=648375=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-648375
]
ASF GitHub Bot logged work on HDFS-16213:
-
Author: ASF GitHub Bot
Created on: 09/Sep/21 04:23
Start Date: 09/Sep/21 04:23
Worklog Time Spent: 10m
Work Description: virajjasani edited a comment on pull request #3386:
URL: https://github.com/apache/hadoop/pull/3386#issuecomment-915750759
Thanks for spending some time @LeonGao91. Although I understand we can repro
it only when we run twice locally but Jenkins does report this as failure
sometimes in single run and hence, this is flaky already.
Please check:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3359/4/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt
https://user-images.githubusercontent.com/34790606/132622011-32d7c4bf-757d-4309-ac6d-ec4ef21b310e.png;>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 648375)
Time Spent: 4h (was: 3h 50m)
> Flaky test TestFsDatasetImpl#testDnRestartWithHardLink
> --
>
> Key: HDFS-16213
> URL: https://issues.apache.org/jira/browse/HDFS-16213
> Project: Hadoop HDFS
> Issue Type: Bug
>Reporter: Viraj Jasani
>Assignee: Viraj Jasani
>Priority: Major
> Labels: pull-request-available
> Time Spent: 4h
> Remaining Estimate: 0h
>
> Failure case:
> [here|https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3359/4/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt]
> {code:java}
> [ERROR]
> testDnRestartWithHardLink(org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.TestFsDatasetImpl)
> Time elapsed: 7.768 s <<< FAILURE![ERROR]
> testDnRestartWithHardLink(org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.TestFsDatasetImpl)
> Time elapsed: 7.768 s <<< FAILURE!java.lang.AssertionError at
> org.junit.Assert.fail(Assert.java:87) at
> org.junit.Assert.assertTrue(Assert.java:42) at
> org.junit.Assert.assertTrue(Assert.java:53) at
> org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.TestFsDatasetImpl.testDnRestartWithHardLink(TestFsDatasetImpl.java:1344)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498) at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59)
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56)
> at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> at
> org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:299)
> at
> org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:293)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266) at
> java.lang.Thread.run(Thread.java:748)
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDFS-16213) Flaky test TestFsDatasetImpl#testDnRestartWithHardLink
[
https://issues.apache.org/jira/browse/HDFS-16213?focusedWorklogId=648374=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-648374
]
ASF GitHub Bot logged work on HDFS-16213:
-
Author: ASF GitHub Bot
Created on: 09/Sep/21 04:22
Start Date: 09/Sep/21 04:22
Worklog Time Spent: 10m
Work Description: virajjasani commented on pull request #3386:
URL: https://github.com/apache/hadoop/pull/3386#issuecomment-915750759
Thanks for spending some time @LeonGao91. Although I understand we can repro
it only when we run twice locally but Jenkins does report this as failure
sometimes.
Please check:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3359/4/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt
https://user-images.githubusercontent.com/34790606/132622011-32d7c4bf-757d-4309-ac6d-ec4ef21b310e.png;>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 648374)
Time Spent: 3h 50m (was: 3h 40m)
> Flaky test TestFsDatasetImpl#testDnRestartWithHardLink
> --
>
> Key: HDFS-16213
> URL: https://issues.apache.org/jira/browse/HDFS-16213
> Project: Hadoop HDFS
> Issue Type: Bug
>Reporter: Viraj Jasani
>Assignee: Viraj Jasani
>Priority: Major
> Labels: pull-request-available
> Time Spent: 3h 50m
> Remaining Estimate: 0h
>
> Failure case:
> [here|https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3359/4/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt]
> {code:java}
> [ERROR]
> testDnRestartWithHardLink(org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.TestFsDatasetImpl)
> Time elapsed: 7.768 s <<< FAILURE![ERROR]
> testDnRestartWithHardLink(org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.TestFsDatasetImpl)
> Time elapsed: 7.768 s <<< FAILURE!java.lang.AssertionError at
> org.junit.Assert.fail(Assert.java:87) at
> org.junit.Assert.assertTrue(Assert.java:42) at
> org.junit.Assert.assertTrue(Assert.java:53) at
> org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.TestFsDatasetImpl.testDnRestartWithHardLink(TestFsDatasetImpl.java:1344)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498) at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59)
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56)
> at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> at
> org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:299)
> at
> org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:293)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266) at
> java.lang.Thread.run(Thread.java:748)
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDFS-16213) Flaky test TestFsDatasetImpl#testDnRestartWithHardLink
[
https://issues.apache.org/jira/browse/HDFS-16213?focusedWorklogId=648372=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-648372
]
ASF GitHub Bot logged work on HDFS-16213:
-
Author: ASF GitHub Bot
Created on: 09/Sep/21 04:19
Start Date: 09/Sep/21 04:19
Worklog Time Spent: 10m
Work Description: virajjasani commented on a change in pull request #3386:
URL: https://github.com/apache/hadoop/pull/3386#discussion_r704939307
##
File path:
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/FsDatasetImpl.java
##
@@ -1068,9 +1070,8 @@ static File moveBlockFiles(Block b, ReplicaInfo
replicaInfo, File destdir)
+ srcReplica + " metadata to "
+ dstMeta, e);
}
-if (LOG.isDebugEnabled()) {
- LOG.info("Linked " + srcReplica.getBlockURI() + " to " + dstFile);
-}
+LOG.info("Linked {} to {} . Dest meta file: {}",
Review comment:
This seems a bug, we are guarding with isDebugEnabled and if DEBUG is
enabled, we are logging at INFO level. Either we can use INFO or DEBUG directly.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 648372)
Time Spent: 3h 40m (was: 3.5h)
> Flaky test TestFsDatasetImpl#testDnRestartWithHardLink
> --
>
> Key: HDFS-16213
> URL: https://issues.apache.org/jira/browse/HDFS-16213
> Project: Hadoop HDFS
> Issue Type: Bug
>Reporter: Viraj Jasani
>Assignee: Viraj Jasani
>Priority: Major
> Labels: pull-request-available
> Time Spent: 3h 40m
> Remaining Estimate: 0h
>
> Failure case:
> [here|https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3359/4/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt]
> {code:java}
> [ERROR]
> testDnRestartWithHardLink(org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.TestFsDatasetImpl)
> Time elapsed: 7.768 s <<< FAILURE![ERROR]
> testDnRestartWithHardLink(org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.TestFsDatasetImpl)
> Time elapsed: 7.768 s <<< FAILURE!java.lang.AssertionError at
> org.junit.Assert.fail(Assert.java:87) at
> org.junit.Assert.assertTrue(Assert.java:42) at
> org.junit.Assert.assertTrue(Assert.java:53) at
> org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.TestFsDatasetImpl.testDnRestartWithHardLink(TestFsDatasetImpl.java:1344)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498) at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59)
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56)
> at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> at
> org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:299)
> at
> org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:293)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266) at
> java.lang.Thread.run(Thread.java:748)
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDFS-16213) Flaky test TestFsDatasetImpl#testDnRestartWithHardLink
[
https://issues.apache.org/jira/browse/HDFS-16213?focusedWorklogId=648242=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-648242
]
ASF GitHub Bot logged work on HDFS-16213:
-
Author: ASF GitHub Bot
Created on: 08/Sep/21 21:52
Start Date: 08/Sep/21 21:52
Worklog Time Spent: 10m
Work Description: LeonGao91 commented on pull request #3386:
URL: https://github.com/apache/hadoop/pull/3386#issuecomment-915598330
Thanks @virajjasani for reporting this issue!
Seems like this happens consistently when running the same test multiple
times, but doesn't fail when running it the first time (happy case), like you
mentioned, this can reproduce it consistently:
@Test
public void t1() throws Exception {
testDnRestartWithHardLink();
testDnRestartWithHardLink();
}
Based on this I am not sure if the root cause is the rare race condition you
mentioned. I suspect it is due to the cache behavior when rerun the same code
the replica loading changed a little bit.
I can spend some more time investigating in the next few days. Let's try to
fix this in the unit test itself
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 648242)
Time Spent: 3.5h (was: 3h 20m)
> Flaky test TestFsDatasetImpl#testDnRestartWithHardLink
> --
>
> Key: HDFS-16213
> URL: https://issues.apache.org/jira/browse/HDFS-16213
> Project: Hadoop HDFS
> Issue Type: Bug
>Reporter: Viraj Jasani
>Assignee: Viraj Jasani
>Priority: Major
> Labels: pull-request-available
> Time Spent: 3.5h
> Remaining Estimate: 0h
>
> Failure case:
> [here|https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3359/4/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt]
> {code:java}
> [ERROR]
> testDnRestartWithHardLink(org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.TestFsDatasetImpl)
> Time elapsed: 7.768 s <<< FAILURE![ERROR]
> testDnRestartWithHardLink(org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.TestFsDatasetImpl)
> Time elapsed: 7.768 s <<< FAILURE!java.lang.AssertionError at
> org.junit.Assert.fail(Assert.java:87) at
> org.junit.Assert.assertTrue(Assert.java:42) at
> org.junit.Assert.assertTrue(Assert.java:53) at
> org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.TestFsDatasetImpl.testDnRestartWithHardLink(TestFsDatasetImpl.java:1344)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498) at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59)
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56)
> at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> at
> org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:299)
> at
> org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:293)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266) at
> java.lang.Thread.run(Thread.java:748)
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDFS-16213) Flaky test TestFsDatasetImpl#testDnRestartWithHardLink
[
https://issues.apache.org/jira/browse/HDFS-16213?focusedWorklogId=648189=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-648189
]
ASF GitHub Bot logged work on HDFS-16213:
-
Author: ASF GitHub Bot
Created on: 08/Sep/21 19:39
Start Date: 08/Sep/21 19:39
Worklog Time Spent: 10m
Work Description: hadoop-yetus commented on pull request #3386:
URL: https://github.com/apache/hadoop/pull/3386#issuecomment-915516780
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 1m 42s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 2 new or modified test files. |
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 41m 25s | | trunk passed |
| +1 :green_heart: | compile | 2m 1s | | trunk passed with JDK
Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 |
| +1 :green_heart: | compile | 1m 49s | | trunk passed with JDK
Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| +1 :green_heart: | checkstyle | 1m 19s | | trunk passed |
| +1 :green_heart: | mvnsite | 2m 4s | | trunk passed |
| +1 :green_heart: | javadoc | 1m 30s | | trunk passed with JDK
Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 |
| +1 :green_heart: | javadoc | 1m 50s | | trunk passed with JDK
Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| +1 :green_heart: | spotbugs | 4m 40s | | trunk passed |
| +1 :green_heart: | shadedclient | 24m 58s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 1m 35s | | the patch passed |
| +1 :green_heart: | compile | 1m 33s | | the patch passed with JDK
Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 |
| +1 :green_heart: | javac | 1m 33s | | the patch passed |
| +1 :green_heart: | compile | 1m 16s | | the patch passed with JDK
Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| +1 :green_heart: | javac | 1m 16s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| +1 :green_heart: | checkstyle | 0m 58s | |
hadoop-hdfs-project/hadoop-hdfs: The patch generated 0 new + 240 unchanged - 3
fixed = 240 total (was 243) |
| +1 :green_heart: | mvnsite | 1m 27s | | the patch passed |
| +1 :green_heart: | javadoc | 0m 58s | | the patch passed with JDK
Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 |
| +1 :green_heart: | javadoc | 1m 46s | | the patch passed with JDK
Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| +1 :green_heart: | spotbugs | 4m 45s | | the patch passed |
| +1 :green_heart: | shadedclient | 25m 17s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| -1 :x: | unit | 391m 45s |
[/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3386/7/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt)
| hadoop-hdfs in the patch passed. |
| +1 :green_heart: | asflicense | 0m 42s | | The patch does not
generate ASF License warnings. |
| | | 511m 12s | | |
| Reason | Tests |
|---:|:--|
| Failed junit tests | hadoop.hdfs.TestBlockTokenWrappingQOP |
| | hadoop.hdfs.TestReconstructStripedFileWithValidator |
| | hadoop.hdfs.TestDFSStripedOutputStreamWithRandomECPolicy |
| | hadoop.hdfs.tools.TestECAdmin |
| | hadoop.hdfs.TestDecommissionWithStripedBackoffMonitor |
| | hadoop.hdfs.TestHDFSFileSystemContract |
| | hadoop.hdfs.TestWriteConfigurationToDFS |
| | hadoop.hdfs.tools.TestDFSZKFailoverController |
| | hadoop.hdfs.web.TestWebHdfsFileSystemContract |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3386/7/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/3386 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient spotbugs checkstyle codespell |
| uname | Linux 889f73ecc5d5 4.15.0-143-generic #147-Ubuntu SMP Wed Apr 14
16:10:11 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git
[jira] [Work logged] (HDFS-16213) Flaky test TestFsDatasetImpl#testDnRestartWithHardLink
[
https://issues.apache.org/jira/browse/HDFS-16213?focusedWorklogId=648159=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-648159
]
ASF GitHub Bot logged work on HDFS-16213:
-
Author: ASF GitHub Bot
Created on: 08/Sep/21 19:00
Start Date: 08/Sep/21 19:00
Worklog Time Spent: 10m
Work Description: LeonGao91 commented on a change in pull request #3386:
URL: https://github.com/apache/hadoop/pull/3386#discussion_r704693282
##
File path:
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/FsDatasetImpl.java
##
@@ -1068,9 +1070,8 @@ static File moveBlockFiles(Block b, ReplicaInfo
replicaInfo, File destdir)
+ srcReplica + " metadata to "
+ dstMeta, e);
}
-if (LOG.isDebugEnabled()) {
- LOG.info("Linked " + srcReplica.getBlockURI() + " to " + dstFile);
-}
+LOG.info("Linked {} to {} . Dest meta file: {}",
Review comment:
Should be debug or trace level?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 648159)
Time Spent: 3h 10m (was: 3h)
> Flaky test TestFsDatasetImpl#testDnRestartWithHardLink
> --
>
> Key: HDFS-16213
> URL: https://issues.apache.org/jira/browse/HDFS-16213
> Project: Hadoop HDFS
> Issue Type: Bug
>Reporter: Viraj Jasani
>Assignee: Viraj Jasani
>Priority: Major
> Labels: pull-request-available
> Time Spent: 3h 10m
> Remaining Estimate: 0h
>
> Failure case:
> [here|https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3359/4/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt]
> {code:java}
> [ERROR]
> testDnRestartWithHardLink(org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.TestFsDatasetImpl)
> Time elapsed: 7.768 s <<< FAILURE![ERROR]
> testDnRestartWithHardLink(org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.TestFsDatasetImpl)
> Time elapsed: 7.768 s <<< FAILURE!java.lang.AssertionError at
> org.junit.Assert.fail(Assert.java:87) at
> org.junit.Assert.assertTrue(Assert.java:42) at
> org.junit.Assert.assertTrue(Assert.java:53) at
> org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.TestFsDatasetImpl.testDnRestartWithHardLink(TestFsDatasetImpl.java:1344)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498) at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59)
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56)
> at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> at
> org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:299)
> at
> org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:293)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266) at
> java.lang.Thread.run(Thread.java:748)
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDFS-16213) Flaky test TestFsDatasetImpl#testDnRestartWithHardLink
[
https://issues.apache.org/jira/browse/HDFS-16213?focusedWorklogId=647927=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-647927
]
ASF GitHub Bot logged work on HDFS-16213:
-
Author: ASF GitHub Bot
Created on: 08/Sep/21 12:38
Start Date: 08/Sep/21 12:38
Worklog Time Spent: 10m
Work Description: virajjasani commented on pull request #3386:
URL: https://github.com/apache/hadoop/pull/3386#issuecomment-915201911
The only way we could avoid making changes in source code is by avoiding
restart of Datanode as part of this test, however, confirmation of test results
after Datanode restart is the primary purpose of this test, hence we can't even
avoid restarting Datanode in the test IMHO.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 647927)
Time Spent: 3h (was: 2h 50m)
> Flaky test TestFsDatasetImpl#testDnRestartWithHardLink
> --
>
> Key: HDFS-16213
> URL: https://issues.apache.org/jira/browse/HDFS-16213
> Project: Hadoop HDFS
> Issue Type: Bug
>Reporter: Viraj Jasani
>Assignee: Viraj Jasani
>Priority: Major
> Labels: pull-request-available
> Time Spent: 3h
> Remaining Estimate: 0h
>
> Failure case:
> [here|https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3359/4/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt]
> {code:java}
> [ERROR]
> testDnRestartWithHardLink(org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.TestFsDatasetImpl)
> Time elapsed: 7.768 s <<< FAILURE![ERROR]
> testDnRestartWithHardLink(org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.TestFsDatasetImpl)
> Time elapsed: 7.768 s <<< FAILURE!java.lang.AssertionError at
> org.junit.Assert.fail(Assert.java:87) at
> org.junit.Assert.assertTrue(Assert.java:42) at
> org.junit.Assert.assertTrue(Assert.java:53) at
> org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.TestFsDatasetImpl.testDnRestartWithHardLink(TestFsDatasetImpl.java:1344)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498) at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59)
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56)
> at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> at
> org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:299)
> at
> org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:293)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266) at
> java.lang.Thread.run(Thread.java:748)
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDFS-16213) Flaky test TestFsDatasetImpl#testDnRestartWithHardLink
[
https://issues.apache.org/jira/browse/HDFS-16213?focusedWorklogId=647925=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-647925
]
ASF GitHub Bot logged work on HDFS-16213:
-
Author: ASF GitHub Bot
Created on: 08/Sep/21 12:34
Start Date: 08/Sep/21 12:34
Worklog Time Spent: 10m
Work Description: virajjasani edited a comment on pull request #3386:
URL: https://github.com/apache/hadoop/pull/3386#issuecomment-915135845
Thanks @ayushtkn.
> Just to know, does the Jenkins also complains about this?
Yes, Jenkins does report this e.g
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3359/4/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt
And I also tried to repro on Mac only. It was reproduced one time with
single test run (debug mode) but simple rerun is always failing with the same
issue.
> Again just asking is there a test-only fix possible, without going to Prod
files,
Sure thing, would be happy to fix this way only but given how replicas are
processed by BlockPoolSlice#getVolumeMap after Datanode is restarted, I think
it might be tough to achieve it for this particular test. At the same time, I
have also tried to keep source code changes as clear with descriptions as
possible to ensure devs understand that a particular flag is for test purpose
only.
FYI @LeonGao91 @Jing9 if you could also review, it would be great. Thanks
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 647925)
Time Spent: 2h 50m (was: 2h 40m)
> Flaky test TestFsDatasetImpl#testDnRestartWithHardLink
> --
>
> Key: HDFS-16213
> URL: https://issues.apache.org/jira/browse/HDFS-16213
> Project: Hadoop HDFS
> Issue Type: Bug
>Reporter: Viraj Jasani
>Assignee: Viraj Jasani
>Priority: Major
> Labels: pull-request-available
> Time Spent: 2h 50m
> Remaining Estimate: 0h
>
> Failure case:
> [here|https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3359/4/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt]
> {code:java}
> [ERROR]
> testDnRestartWithHardLink(org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.TestFsDatasetImpl)
> Time elapsed: 7.768 s <<< FAILURE![ERROR]
> testDnRestartWithHardLink(org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.TestFsDatasetImpl)
> Time elapsed: 7.768 s <<< FAILURE!java.lang.AssertionError at
> org.junit.Assert.fail(Assert.java:87) at
> org.junit.Assert.assertTrue(Assert.java:42) at
> org.junit.Assert.assertTrue(Assert.java:53) at
> org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.TestFsDatasetImpl.testDnRestartWithHardLink(TestFsDatasetImpl.java:1344)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498) at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59)
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56)
> at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> at
> org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:299)
> at
> org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:293)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266) at
> java.lang.Thread.run(Thread.java:748)
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDFS-16213) Flaky test TestFsDatasetImpl#testDnRestartWithHardLink
[
https://issues.apache.org/jira/browse/HDFS-16213?focusedWorklogId=647895=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-647895
]
ASF GitHub Bot logged work on HDFS-16213:
-
Author: ASF GitHub Bot
Created on: 08/Sep/21 11:22
Start Date: 08/Sep/21 11:22
Worklog Time Spent: 10m
Work Description: virajjasani edited a comment on pull request #3386:
URL: https://github.com/apache/hadoop/pull/3386#issuecomment-915135845
Thanks @ayushtkn.
> Just to know, does the Jenkins also complains about this?
Yes, Jenkins does report this e.g
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3359/4/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt
And I also tried to repro on Mac only. It was reproduced one time with
single test run (debug mode) but simple rerun is always failing with the same
issue.
> Again just asking is there a test-only fix possible, without going to Prod
files,
Sure thing, would be happy to fix this way only but given how replicas are
processed by BlockPoolSlice#getVolumeMap after Datanode is restarted, I think
it might be tough to achieve it for this particular test. At the same time, I
have also tried to keep source code changes as clear with descriptions as
possible to ensure devs understand that a particular flag is for test purpose
only.
FYI @LeonGao91, if you could also review, it would be great. Thanks
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 647895)
Time Spent: 2h 40m (was: 2.5h)
> Flaky test TestFsDatasetImpl#testDnRestartWithHardLink
> --
>
> Key: HDFS-16213
> URL: https://issues.apache.org/jira/browse/HDFS-16213
> Project: Hadoop HDFS
> Issue Type: Bug
>Reporter: Viraj Jasani
>Assignee: Viraj Jasani
>Priority: Major
> Labels: pull-request-available
> Time Spent: 2h 40m
> Remaining Estimate: 0h
>
> Failure case:
> [here|https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3359/4/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt]
> {code:java}
> [ERROR]
> testDnRestartWithHardLink(org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.TestFsDatasetImpl)
> Time elapsed: 7.768 s <<< FAILURE![ERROR]
> testDnRestartWithHardLink(org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.TestFsDatasetImpl)
> Time elapsed: 7.768 s <<< FAILURE!java.lang.AssertionError at
> org.junit.Assert.fail(Assert.java:87) at
> org.junit.Assert.assertTrue(Assert.java:42) at
> org.junit.Assert.assertTrue(Assert.java:53) at
> org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.TestFsDatasetImpl.testDnRestartWithHardLink(TestFsDatasetImpl.java:1344)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498) at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59)
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56)
> at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> at
> org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:299)
> at
> org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:293)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266) at
> java.lang.Thread.run(Thread.java:748)
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDFS-16213) Flaky test TestFsDatasetImpl#testDnRestartWithHardLink
[
https://issues.apache.org/jira/browse/HDFS-16213?focusedWorklogId=647887=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-647887
]
ASF GitHub Bot logged work on HDFS-16213:
-
Author: ASF GitHub Bot
Created on: 08/Sep/21 11:10
Start Date: 08/Sep/21 11:10
Worklog Time Spent: 10m
Work Description: virajjasani edited a comment on pull request #3386:
URL: https://github.com/apache/hadoop/pull/3386#issuecomment-915135845
Thanks @ayushtkn.
> Just to know, does the Jenkins also complains about this?
Yes, Jenkins does report this e.g
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3359/4/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt
And I also tried to repro on Mac only. It was reproduced one time with
single test run (debug mode) but simple rerun is always failing with the same
issue.
> Again just asking is there a test-only fix possible, without going to Prod
files,
Sure thing, would be happy to fix this way only but given how replicas are
built up by BlockPoolSlice instance after Datanode is restarted, I think it
might be tough to achieve it for this particular test. At the same time, I have
also tried to keep source code changes as clear with descriptions as possible
to ensure devs understand that a particular flag is for test purpose only.
FYI @LeonGao91, if you could also review, it would be great. Thanks
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 647887)
Time Spent: 2.5h (was: 2h 20m)
> Flaky test TestFsDatasetImpl#testDnRestartWithHardLink
> --
>
> Key: HDFS-16213
> URL: https://issues.apache.org/jira/browse/HDFS-16213
> Project: Hadoop HDFS
> Issue Type: Bug
>Reporter: Viraj Jasani
>Assignee: Viraj Jasani
>Priority: Major
> Labels: pull-request-available
> Time Spent: 2.5h
> Remaining Estimate: 0h
>
> Failure case:
> [here|https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3359/4/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt]
> {code:java}
> [ERROR]
> testDnRestartWithHardLink(org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.TestFsDatasetImpl)
> Time elapsed: 7.768 s <<< FAILURE![ERROR]
> testDnRestartWithHardLink(org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.TestFsDatasetImpl)
> Time elapsed: 7.768 s <<< FAILURE!java.lang.AssertionError at
> org.junit.Assert.fail(Assert.java:87) at
> org.junit.Assert.assertTrue(Assert.java:42) at
> org.junit.Assert.assertTrue(Assert.java:53) at
> org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.TestFsDatasetImpl.testDnRestartWithHardLink(TestFsDatasetImpl.java:1344)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498) at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59)
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56)
> at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> at
> org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:299)
> at
> org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:293)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266) at
> java.lang.Thread.run(Thread.java:748)
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDFS-16213) Flaky test TestFsDatasetImpl#testDnRestartWithHardLink
[
https://issues.apache.org/jira/browse/HDFS-16213?focusedWorklogId=647885=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-647885
]
ASF GitHub Bot logged work on HDFS-16213:
-
Author: ASF GitHub Bot
Created on: 08/Sep/21 11:09
Start Date: 08/Sep/21 11:09
Worklog Time Spent: 10m
Work Description: virajjasani edited a comment on pull request #3386:
URL: https://github.com/apache/hadoop/pull/3386#issuecomment-915135845
Thanks @ayushtkn.
> Just to know, does the Jenkins also complains about this?
Yes, Jenkins does report this e.g
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3359/4/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt
And I also tried to repro on Mac only. It was reproduced one time with
single test run (debug mode) but simple rerun is always failing with the same
issue.
> Again just asking is there a test-only fix possible, without going to Prod
files,
Sure thing, would be happy to fix this by tests only but given how replicas
are built up by BlockPoolSlice instance after Datanode is restarted, I think it
might be tough to achieve it. At the same time, I am trying to keep source code
changes as clear with descriptions as possible to ensure devs understand that a
particular flag is for test use only.
FYI @LeonGao91, if you could also review, it would be great. Thanks
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 647885)
Time Spent: 2h 20m (was: 2h 10m)
> Flaky test TestFsDatasetImpl#testDnRestartWithHardLink
> --
>
> Key: HDFS-16213
> URL: https://issues.apache.org/jira/browse/HDFS-16213
> Project: Hadoop HDFS
> Issue Type: Bug
>Reporter: Viraj Jasani
>Assignee: Viraj Jasani
>Priority: Major
> Labels: pull-request-available
> Time Spent: 2h 20m
> Remaining Estimate: 0h
>
> Failure case:
> [here|https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3359/4/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt]
> {code:java}
> [ERROR]
> testDnRestartWithHardLink(org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.TestFsDatasetImpl)
> Time elapsed: 7.768 s <<< FAILURE![ERROR]
> testDnRestartWithHardLink(org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.TestFsDatasetImpl)
> Time elapsed: 7.768 s <<< FAILURE!java.lang.AssertionError at
> org.junit.Assert.fail(Assert.java:87) at
> org.junit.Assert.assertTrue(Assert.java:42) at
> org.junit.Assert.assertTrue(Assert.java:53) at
> org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.TestFsDatasetImpl.testDnRestartWithHardLink(TestFsDatasetImpl.java:1344)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498) at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59)
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56)
> at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> at
> org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:299)
> at
> org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:293)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266) at
> java.lang.Thread.run(Thread.java:748)
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDFS-16213) Flaky test TestFsDatasetImpl#testDnRestartWithHardLink
[
https://issues.apache.org/jira/browse/HDFS-16213?focusedWorklogId=647884=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-647884
]
ASF GitHub Bot logged work on HDFS-16213:
-
Author: ASF GitHub Bot
Created on: 08/Sep/21 11:00
Start Date: 08/Sep/21 11:00
Worklog Time Spent: 10m
Work Description: virajjasani commented on pull request #3386:
URL: https://github.com/apache/hadoop/pull/3386#issuecomment-915135845
Thanks @ayushtkn.
> Just to know, does the Jenkins also complains about this?
Yes, Jenkins does report this e.g
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3359/4/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt
And I also tried to repro on Mac only. It was reproduced one time with
single test run (debug mode) but simple rerun is always failing with the same
issue.
> Again just asking is there a test-only fix possible, without going to Prod
files,
Sure thing, would be happy to fix this by tests only but given how replicas
are built up by BlockPoolSlice instance after Datanode is restarted, I think it
might be tough to achieve it.
FYI @LeonGao91, if you could also review, it would be great. Thanks
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 647884)
Time Spent: 2h 10m (was: 2h)
> Flaky test TestFsDatasetImpl#testDnRestartWithHardLink
> --
>
> Key: HDFS-16213
> URL: https://issues.apache.org/jira/browse/HDFS-16213
> Project: Hadoop HDFS
> Issue Type: Bug
>Reporter: Viraj Jasani
>Assignee: Viraj Jasani
>Priority: Major
> Labels: pull-request-available
> Time Spent: 2h 10m
> Remaining Estimate: 0h
>
> Failure case:
> [here|https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3359/4/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt]
> {code:java}
> [ERROR]
> testDnRestartWithHardLink(org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.TestFsDatasetImpl)
> Time elapsed: 7.768 s <<< FAILURE![ERROR]
> testDnRestartWithHardLink(org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.TestFsDatasetImpl)
> Time elapsed: 7.768 s <<< FAILURE!java.lang.AssertionError at
> org.junit.Assert.fail(Assert.java:87) at
> org.junit.Assert.assertTrue(Assert.java:42) at
> org.junit.Assert.assertTrue(Assert.java:53) at
> org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.TestFsDatasetImpl.testDnRestartWithHardLink(TestFsDatasetImpl.java:1344)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498) at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59)
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56)
> at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> at
> org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:299)
> at
> org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:293)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266) at
> java.lang.Thread.run(Thread.java:748)
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDFS-16213) Flaky test TestFsDatasetImpl#testDnRestartWithHardLink
[
https://issues.apache.org/jira/browse/HDFS-16213?focusedWorklogId=647882=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-647882
]
ASF GitHub Bot logged work on HDFS-16213:
-
Author: ASF GitHub Bot
Created on: 08/Sep/21 10:53
Start Date: 08/Sep/21 10:53
Worklog Time Spent: 10m
Work Description: ayushtkn commented on pull request #3386:
URL: https://github.com/apache/hadoop/pull/3386#issuecomment-915130973
Thanx @virajjasani for the PR. I too tried to repro this by running twice
couple of days back, but It didn't repro for me as well. Some OS specific? I am
on MacOs, can try on Ubuntu if you say so.
Just to know, does the Jenkins also complains about this? Since you said it
reproduces by re-runing the test. Again just asking is there a test-only fix
possible, without going to Prod files,
I think this is from some recent code, can you loop in the original
developer & the reviewer as well.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 647882)
Time Spent: 2h (was: 1h 50m)
> Flaky test TestFsDatasetImpl#testDnRestartWithHardLink
> --
>
> Key: HDFS-16213
> URL: https://issues.apache.org/jira/browse/HDFS-16213
> Project: Hadoop HDFS
> Issue Type: Bug
>Reporter: Viraj Jasani
>Assignee: Viraj Jasani
>Priority: Major
> Labels: pull-request-available
> Time Spent: 2h
> Remaining Estimate: 0h
>
> Failure case:
> [here|https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3359/4/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt]
> {code:java}
> [ERROR]
> testDnRestartWithHardLink(org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.TestFsDatasetImpl)
> Time elapsed: 7.768 s <<< FAILURE![ERROR]
> testDnRestartWithHardLink(org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.TestFsDatasetImpl)
> Time elapsed: 7.768 s <<< FAILURE!java.lang.AssertionError at
> org.junit.Assert.fail(Assert.java:87) at
> org.junit.Assert.assertTrue(Assert.java:42) at
> org.junit.Assert.assertTrue(Assert.java:53) at
> org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.TestFsDatasetImpl.testDnRestartWithHardLink(TestFsDatasetImpl.java:1344)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498) at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59)
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56)
> at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> at
> org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:299)
> at
> org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:293)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266) at
> java.lang.Thread.run(Thread.java:748)
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDFS-16213) Flaky test TestFsDatasetImpl#testDnRestartWithHardLink
[
https://issues.apache.org/jira/browse/HDFS-16213?focusedWorklogId=647815=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-647815
]
ASF GitHub Bot logged work on HDFS-16213:
-
Author: ASF GitHub Bot
Created on: 08/Sep/21 08:26
Start Date: 08/Sep/21 08:26
Worklog Time Spent: 10m
Work Description: virajjasani commented on pull request #3386:
URL: https://github.com/apache/hadoop/pull/3386#issuecomment-915028364
Also, in debug mode, I was able to reproduce the same flaky behaviour in
single run (only one time). FYI @ferhui
Thanks
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 647815)
Time Spent: 1h 50m (was: 1h 40m)
> Flaky test TestFsDatasetImpl#testDnRestartWithHardLink
> --
>
> Key: HDFS-16213
> URL: https://issues.apache.org/jira/browse/HDFS-16213
> Project: Hadoop HDFS
> Issue Type: Bug
>Reporter: Viraj Jasani
>Assignee: Viraj Jasani
>Priority: Major
> Labels: pull-request-available
> Time Spent: 1h 50m
> Remaining Estimate: 0h
>
> Failure case:
> [here|https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3359/4/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt]
> {code:java}
> [ERROR]
> testDnRestartWithHardLink(org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.TestFsDatasetImpl)
> Time elapsed: 7.768 s <<< FAILURE![ERROR]
> testDnRestartWithHardLink(org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.TestFsDatasetImpl)
> Time elapsed: 7.768 s <<< FAILURE!java.lang.AssertionError at
> org.junit.Assert.fail(Assert.java:87) at
> org.junit.Assert.assertTrue(Assert.java:42) at
> org.junit.Assert.assertTrue(Assert.java:53) at
> org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.TestFsDatasetImpl.testDnRestartWithHardLink(TestFsDatasetImpl.java:1344)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498) at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59)
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56)
> at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> at
> org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:299)
> at
> org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:293)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266) at
> java.lang.Thread.run(Thread.java:748)
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDFS-16213) Flaky test TestFsDatasetImpl#testDnRestartWithHardLink
[
https://issues.apache.org/jira/browse/HDFS-16213?focusedWorklogId=647748=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-647748
]
ASF GitHub Bot logged work on HDFS-16213:
-
Author: ASF GitHub Bot
Created on: 08/Sep/21 05:46
Start Date: 08/Sep/21 05:46
Worklog Time Spent: 10m
Work Description: virajjasani commented on pull request #3386:
URL: https://github.com/apache/hadoop/pull/3386#issuecomment-914935498
@ferhui did you run test code twice as part of same test? I can reproduce
this failure just by running
```
@Test
public void t1() throws Exception {
testDnRestartWithHardLink();
testDnRestartWithHardLink();
}
```
Ideally, based on how the test is written, the test code is supposed to be
idempotent but it is not because of the replica processing done by
`addReplicaThreadPool`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 647748)
Time Spent: 1h 40m (was: 1.5h)
> Flaky test TestFsDatasetImpl#testDnRestartWithHardLink
> --
>
> Key: HDFS-16213
> URL: https://issues.apache.org/jira/browse/HDFS-16213
> Project: Hadoop HDFS
> Issue Type: Bug
>Reporter: Viraj Jasani
>Assignee: Viraj Jasani
>Priority: Major
> Labels: pull-request-available
> Time Spent: 1h 40m
> Remaining Estimate: 0h
>
> Failure case:
> [here|https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3359/4/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt]
> {code:java}
> [ERROR]
> testDnRestartWithHardLink(org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.TestFsDatasetImpl)
> Time elapsed: 7.768 s <<< FAILURE![ERROR]
> testDnRestartWithHardLink(org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.TestFsDatasetImpl)
> Time elapsed: 7.768 s <<< FAILURE!java.lang.AssertionError at
> org.junit.Assert.fail(Assert.java:87) at
> org.junit.Assert.assertTrue(Assert.java:42) at
> org.junit.Assert.assertTrue(Assert.java:53) at
> org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.TestFsDatasetImpl.testDnRestartWithHardLink(TestFsDatasetImpl.java:1344)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498) at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59)
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56)
> at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> at
> org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:299)
> at
> org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:293)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266) at
> java.lang.Thread.run(Thread.java:748)
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDFS-16213) Flaky test TestFsDatasetImpl#testDnRestartWithHardLink
[
https://issues.apache.org/jira/browse/HDFS-16213?focusedWorklogId=647722=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-647722
]
ASF GitHub Bot logged work on HDFS-16213:
-
Author: ASF GitHub Bot
Created on: 08/Sep/21 04:37
Start Date: 08/Sep/21 04:37
Worklog Time Spent: 10m
Work Description: ferhui commented on pull request #3386:
URL: https://github.com/apache/hadoop/pull/3386#issuecomment-914905663
@virajjasani Thanks for contribution. I didn't reproduce it by running some
times.
Could you please introduce to make
TestFsDatasetImpl#testDnRestartWithHardLink fail by changing simple codes?
I want to dig it
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 647722)
Time Spent: 1.5h (was: 1h 20m)
> Flaky test TestFsDatasetImpl#testDnRestartWithHardLink
> --
>
> Key: HDFS-16213
> URL: https://issues.apache.org/jira/browse/HDFS-16213
> Project: Hadoop HDFS
> Issue Type: Bug
>Reporter: Viraj Jasani
>Assignee: Viraj Jasani
>Priority: Major
> Labels: pull-request-available
> Time Spent: 1.5h
> Remaining Estimate: 0h
>
> Failure case:
> [here|https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3359/4/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt]
> {code:java}
> [ERROR]
> testDnRestartWithHardLink(org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.TestFsDatasetImpl)
> Time elapsed: 7.768 s <<< FAILURE![ERROR]
> testDnRestartWithHardLink(org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.TestFsDatasetImpl)
> Time elapsed: 7.768 s <<< FAILURE!java.lang.AssertionError at
> org.junit.Assert.fail(Assert.java:87) at
> org.junit.Assert.assertTrue(Assert.java:42) at
> org.junit.Assert.assertTrue(Assert.java:53) at
> org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.TestFsDatasetImpl.testDnRestartWithHardLink(TestFsDatasetImpl.java:1344)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498) at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59)
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56)
> at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> at
> org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:299)
> at
> org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:293)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266) at
> java.lang.Thread.run(Thread.java:748)
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDFS-16213) Flaky test TestFsDatasetImpl#testDnRestartWithHardLink
[
https://issues.apache.org/jira/browse/HDFS-16213?focusedWorklogId=647299=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-647299
]
ASF GitHub Bot logged work on HDFS-16213:
-
Author: ASF GitHub Bot
Created on: 07/Sep/21 11:10
Start Date: 07/Sep/21 11:10
Worklog Time Spent: 10m
Work Description: hadoop-yetus commented on pull request #3386:
URL: https://github.com/apache/hadoop/pull/3386#issuecomment-914213445
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 43s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 2 new or modified test files. |
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 30m 56s | | trunk passed |
| +1 :green_heart: | compile | 1m 23s | | trunk passed with JDK
Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 |
| +1 :green_heart: | compile | 1m 16s | | trunk passed with JDK
Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| +1 :green_heart: | checkstyle | 1m 3s | | trunk passed |
| +1 :green_heart: | mvnsite | 1m 24s | | trunk passed |
| +1 :green_heart: | javadoc | 0m 58s | | trunk passed with JDK
Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 |
| +1 :green_heart: | javadoc | 1m 30s | | trunk passed with JDK
Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| +1 :green_heart: | spotbugs | 3m 8s | | trunk passed |
| +1 :green_heart: | shadedclient | 16m 23s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 1m 10s | | the patch passed |
| +1 :green_heart: | compile | 1m 16s | | the patch passed with JDK
Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 |
| +1 :green_heart: | javac | 1m 16s | | the patch passed |
| +1 :green_heart: | compile | 1m 7s | | the patch passed with JDK
Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| +1 :green_heart: | javac | 1m 7s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| +1 :green_heart: | checkstyle | 0m 54s | |
hadoop-hdfs-project/hadoop-hdfs: The patch generated 0 new + 241 unchanged - 3
fixed = 241 total (was 244) |
| +1 :green_heart: | mvnsite | 1m 14s | | the patch passed |
| +1 :green_heart: | javadoc | 0m 49s | | the patch passed with JDK
Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 |
| +1 :green_heart: | javadoc | 1m 22s | | the patch passed with JDK
Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| +1 :green_heart: | spotbugs | 3m 8s | | the patch passed |
| +1 :green_heart: | shadedclient | 16m 7s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| -1 :x: | unit | 248m 51s |
[/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3386/6/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt)
| hadoop-hdfs in the patch passed. |
| +1 :green_heart: | asflicense | 0m 44s | | The patch does not
generate ASF License warnings. |
| | | 333m 27s | | |
| Reason | Tests |
|---:|:--|
| Failed junit tests | hadoop.hdfs.server.mover.TestMover |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3386/6/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/3386 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient spotbugs checkstyle codespell |
| uname | Linux de7642c3d999 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6
11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 65c59d6601a9946d87cd49675516123ccc1a4a80 |
| Default Java | Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| Multi-JDK versions |
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
/usr/lib/jvm/java-8-openjdk-amd64:Private
Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| Test Results |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3386/6/testReport/ |
| Max. process+thread
[jira] [Work logged] (HDFS-16213) Flaky test TestFsDatasetImpl#testDnRestartWithHardLink
[
https://issues.apache.org/jira/browse/HDFS-16213?focusedWorklogId=646708=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-646708
]
ASF GitHub Bot logged work on HDFS-16213:
-
Author: ASF GitHub Bot
Created on: 05/Sep/21 16:57
Start Date: 05/Sep/21 16:57
Worklog Time Spent: 10m
Work Description: virajjasani commented on pull request #3386:
URL: https://github.com/apache/hadoop/pull/3386#issuecomment-913189856
@aajisaka @ayushtkn @jojochuang @tasanuma could you please take a look?
Thanks
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 646708)
Time Spent: 1h 10m (was: 1h)
> Flaky test TestFsDatasetImpl#testDnRestartWithHardLink
> --
>
> Key: HDFS-16213
> URL: https://issues.apache.org/jira/browse/HDFS-16213
> Project: Hadoop HDFS
> Issue Type: Bug
>Reporter: Viraj Jasani
>Assignee: Viraj Jasani
>Priority: Major
> Labels: pull-request-available
> Time Spent: 1h 10m
> Remaining Estimate: 0h
>
> Failure case:
> [here|https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3359/4/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt]
> {code:java}
> [ERROR]
> testDnRestartWithHardLink(org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.TestFsDatasetImpl)
> Time elapsed: 7.768 s <<< FAILURE![ERROR]
> testDnRestartWithHardLink(org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.TestFsDatasetImpl)
> Time elapsed: 7.768 s <<< FAILURE!java.lang.AssertionError at
> org.junit.Assert.fail(Assert.java:87) at
> org.junit.Assert.assertTrue(Assert.java:42) at
> org.junit.Assert.assertTrue(Assert.java:53) at
> org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.TestFsDatasetImpl.testDnRestartWithHardLink(TestFsDatasetImpl.java:1344)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498) at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59)
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56)
> at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> at
> org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:299)
> at
> org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:293)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266) at
> java.lang.Thread.run(Thread.java:748)
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDFS-16213) Flaky test TestFsDatasetImpl#testDnRestartWithHardLink
[
https://issues.apache.org/jira/browse/HDFS-16213?focusedWorklogId=646707=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-646707
]
ASF GitHub Bot logged work on HDFS-16213:
-
Author: ASF GitHub Bot
Created on: 05/Sep/21 16:41
Start Date: 05/Sep/21 16:41
Worklog Time Spent: 10m
Work Description: hadoop-yetus commented on pull request #3386:
URL: https://github.com/apache/hadoop/pull/3386#issuecomment-913187495
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 1m 25s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 1s | | codespell was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 2 new or modified test files. |
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 32m 26s | | trunk passed |
| +1 :green_heart: | compile | 1m 23s | | trunk passed with JDK
Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 |
| +1 :green_heart: | compile | 1m 18s | | trunk passed with JDK
Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| +1 :green_heart: | checkstyle | 1m 1s | | trunk passed |
| +1 :green_heart: | mvnsite | 1m 25s | | trunk passed |
| +1 :green_heart: | javadoc | 0m 57s | | trunk passed with JDK
Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 |
| +1 :green_heart: | javadoc | 1m 31s | | trunk passed with JDK
Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| +1 :green_heart: | spotbugs | 3m 15s | | trunk passed |
| +1 :green_heart: | shadedclient | 16m 37s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 1m 15s | | the patch passed |
| +1 :green_heart: | compile | 1m 15s | | the patch passed with JDK
Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 |
| +1 :green_heart: | javac | 1m 15s | | the patch passed |
| +1 :green_heart: | compile | 1m 8s | | the patch passed with JDK
Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| +1 :green_heart: | javac | 1m 8s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| +1 :green_heart: | checkstyle | 0m 52s | |
hadoop-hdfs-project/hadoop-hdfs: The patch generated 0 new + 240 unchanged - 3
fixed = 240 total (was 243) |
| +1 :green_heart: | mvnsite | 1m 18s | | the patch passed |
| +1 :green_heart: | javadoc | 0m 47s | | the patch passed with JDK
Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 |
| +1 :green_heart: | javadoc | 1m 23s | | the patch passed with JDK
Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| +1 :green_heart: | spotbugs | 3m 17s | | the patch passed |
| +1 :green_heart: | shadedclient | 17m 12s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| -1 :x: | unit | 462m 11s |
[/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3386/2/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt)
| hadoop-hdfs in the patch passed. |
| +1 :green_heart: | asflicense | 0m 45s | | The patch does not
generate ASF License warnings. |
| | | 550m 26s | | |
| Reason | Tests |
|---:|:--|
| Failed junit tests | hadoop.hdfs.TestViewDistributedFileSystemContract |
| | hadoop.hdfs.TestSnapshotCommands |
| | hadoop.hdfs.server.namenode.ha.TestSeveralNameNodes |
| | hadoop.fs.viewfs.TestViewFSOverloadSchemeWithMountTableConfigInHDFS |
| | hadoop.hdfs.server.diskbalancer.command.TestDiskBalancerCommand |
| | hadoop.hdfs.TestHDFSFileSystemContract |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3386/2/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/3386 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient spotbugs checkstyle codespell |
| uname | Linux 15d85754f4af 4.15.0-65-generic #74-Ubuntu SMP Tue Sep 17
17:06:04 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / ae4fbd19d641d748971f976aa6c3a7773d10902d |
| Default Java | Private
[jira] [Work logged] (HDFS-16213) Flaky test TestFsDatasetImpl#testDnRestartWithHardLink
[
https://issues.apache.org/jira/browse/HDFS-16213?focusedWorklogId=646704=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-646704
]
ASF GitHub Bot logged work on HDFS-16213:
-
Author: ASF GitHub Bot
Created on: 05/Sep/21 15:53
Start Date: 05/Sep/21 15:53
Worklog Time Spent: 10m
Work Description: hadoop-yetus commented on pull request #3386:
URL: https://github.com/apache/hadoop/pull/3386#issuecomment-913179678
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 1m 21s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 2 new or modified test files. |
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 35m 14s | | trunk passed |
| +1 :green_heart: | compile | 1m 34s | | trunk passed with JDK
Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 |
| +1 :green_heart: | compile | 1m 24s | | trunk passed with JDK
Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| +1 :green_heart: | checkstyle | 1m 1s | | trunk passed |
| +1 :green_heart: | mvnsite | 1m 30s | | trunk passed |
| +1 :green_heart: | javadoc | 1m 0s | | trunk passed with JDK
Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 |
| +1 :green_heart: | javadoc | 1m 29s | | trunk passed with JDK
Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| +1 :green_heart: | spotbugs | 3m 34s | | trunk passed |
| +1 :green_heart: | shadedclient | 19m 30s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 1m 25s | | the patch passed |
| +1 :green_heart: | compile | 1m 28s | | the patch passed with JDK
Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 |
| +1 :green_heart: | javac | 1m 28s | | the patch passed |
| +1 :green_heart: | compile | 1m 21s | | the patch passed with JDK
Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| +1 :green_heart: | javac | 1m 21s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| +1 :green_heart: | checkstyle | 0m 57s | |
hadoop-hdfs-project/hadoop-hdfs: The patch generated 0 new + 240 unchanged - 3
fixed = 240 total (was 243) |
| +1 :green_heart: | mvnsite | 1m 28s | | the patch passed |
| +1 :green_heart: | javadoc | 0m 54s | | the patch passed with JDK
Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 |
| +1 :green_heart: | javadoc | 1m 26s | | the patch passed with JDK
Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| +1 :green_heart: | spotbugs | 3m 45s | | the patch passed |
| +1 :green_heart: | shadedclient | 19m 22s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| -1 :x: | unit | 404m 24s |
[/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3386/3/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt)
| hadoop-hdfs in the patch passed. |
| +1 :green_heart: | asflicense | 0m 37s | | The patch does not
generate ASF License warnings. |
| | | 501m 40s | | |
| Reason | Tests |
|---:|:--|
| Failed junit tests | hadoop.hdfs.web.TestWebHdfsFileSystemContract |
| | hadoop.hdfs.TestLeaseRecovery |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3386/3/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/3386 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient spotbugs checkstyle codespell |
| uname | Linux 4849c85c7f25 4.15.0-147-generic #151-Ubuntu SMP Fri Jun 18
19:21:19 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / ae4fbd19d641d748971f976aa6c3a7773d10902d |
| Default Java | Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| Multi-JDK versions |
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
/usr/lib/jvm/java-8-openjdk-amd64:Private
Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| Test Results |
[jira] [Work logged] (HDFS-16213) Flaky test TestFsDatasetImpl#testDnRestartWithHardLink
[
https://issues.apache.org/jira/browse/HDFS-16213?focusedWorklogId=646703=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-646703
]
ASF GitHub Bot logged work on HDFS-16213:
-
Author: ASF GitHub Bot
Created on: 05/Sep/21 15:49
Start Date: 05/Sep/21 15:49
Worklog Time Spent: 10m
Work Description: hadoop-yetus commented on pull request #3386:
URL: https://github.com/apache/hadoop/pull/3386#issuecomment-913179175
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 1m 12s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 1s | | codespell was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 2 new or modified test files. |
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 35m 28s | | trunk passed |
| +1 :green_heart: | compile | 1m 32s | | trunk passed with JDK
Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 |
| +1 :green_heart: | compile | 1m 22s | | trunk passed with JDK
Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| +1 :green_heart: | checkstyle | 1m 2s | | trunk passed |
| +1 :green_heart: | mvnsite | 1m 30s | | trunk passed |
| +1 :green_heart: | javadoc | 0m 59s | | trunk passed with JDK
Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 |
| +1 :green_heart: | javadoc | 1m 30s | | trunk passed with JDK
Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| +1 :green_heart: | spotbugs | 3m 37s | | trunk passed |
| +1 :green_heart: | shadedclient | 19m 23s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 1m 26s | | the patch passed |
| +1 :green_heart: | compile | 1m 29s | | the patch passed with JDK
Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 |
| +1 :green_heart: | javac | 1m 29s | | the patch passed |
| +1 :green_heart: | compile | 1m 19s | | the patch passed with JDK
Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| +1 :green_heart: | javac | 1m 19s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| +1 :green_heart: | checkstyle | 0m 57s | |
hadoop-hdfs-project/hadoop-hdfs: The patch generated 0 new + 240 unchanged - 3
fixed = 240 total (was 243) |
| +1 :green_heart: | mvnsite | 1m 26s | | the patch passed |
| +1 :green_heart: | javadoc | 0m 55s | | the patch passed with JDK
Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 |
| +1 :green_heart: | javadoc | 1m 25s | | the patch passed with JDK
Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| +1 :green_heart: | spotbugs | 3m 49s | | the patch passed |
| +1 :green_heart: | shadedclient | 19m 27s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| -1 :x: | unit | 400m 56s |
[/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3386/4/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt)
| hadoop-hdfs in the patch passed. |
| +1 :green_heart: | asflicense | 0m 47s | | The patch does not
generate ASF License warnings. |
| | | 498m 19s | | |
| Reason | Tests |
|---:|:--|
| Failed junit tests |
hadoop.hdfs.server.blockmanagement.TestBlockTokenWithDFSStriped |
| | hadoop.hdfs.web.TestWebHdfsFileSystemContract |
| | hadoop.hdfs.TestReconstructStripedFileWithRandomECPolicy |
| | hadoop.hdfs.TestHDFSFileSystemContract |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3386/4/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/3386 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient spotbugs checkstyle codespell |
| uname | Linux 36891b3c852a 4.15.0-147-generic #151-Ubuntu SMP Fri Jun 18
19:21:19 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / ae4fbd19d641d748971f976aa6c3a7773d10902d |
| Default Java | Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| Multi-JDK versions |
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
[jira] [Work logged] (HDFS-16213) Flaky test TestFsDatasetImpl#testDnRestartWithHardLink
[
https://issues.apache.org/jira/browse/HDFS-16213?focusedWorklogId=646697=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-646697
]
ASF GitHub Bot logged work on HDFS-16213:
-
Author: ASF GitHub Bot
Created on: 05/Sep/21 14:21
Start Date: 05/Sep/21 14:21
Worklog Time Spent: 10m
Work Description: hadoop-yetus commented on pull request #3386:
URL: https://github.com/apache/hadoop/pull/3386#issuecomment-913163730
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 1m 2s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 1s | | codespell was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 2 new or modified test files. |
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 33m 5s | | trunk passed |
| +1 :green_heart: | compile | 1m 23s | | trunk passed with JDK
Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 |
| +1 :green_heart: | compile | 1m 14s | | trunk passed with JDK
Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| +1 :green_heart: | checkstyle | 1m 0s | | trunk passed |
| +1 :green_heart: | mvnsite | 1m 24s | | trunk passed |
| +1 :green_heart: | javadoc | 0m 55s | | trunk passed with JDK
Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 |
| +1 :green_heart: | javadoc | 1m 27s | | trunk passed with JDK
Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| +1 :green_heart: | spotbugs | 3m 19s | | trunk passed |
| +1 :green_heart: | shadedclient | 18m 36s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 1m 17s | | the patch passed |
| +1 :green_heart: | compile | 1m 17s | | the patch passed with JDK
Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 |
| +1 :green_heart: | javac | 1m 17s | | the patch passed |
| +1 :green_heart: | compile | 1m 8s | | the patch passed with JDK
Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| +1 :green_heart: | javac | 1m 8s | | the patch passed |
| +1 :green_heart: | blanks | 0m 1s | | The patch has no blanks
issues. |
| +1 :green_heart: | checkstyle | 0m 55s | |
hadoop-hdfs-project/hadoop-hdfs: The patch generated 0 new + 240 unchanged - 3
fixed = 240 total (was 243) |
| +1 :green_heart: | mvnsite | 1m 16s | | the patch passed |
| +1 :green_heart: | javadoc | 0m 47s | | the patch passed with JDK
Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 |
| +1 :green_heart: | javadoc | 1m 22s | | the patch passed with JDK
Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| +1 :green_heart: | spotbugs | 3m 24s | | the patch passed |
| +1 :green_heart: | shadedclient | 18m 58s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 318m 38s | | hadoop-hdfs in the patch
passed. |
| +1 :green_heart: | asflicense | 0m 38s | | The patch does not
generate ASF License warnings. |
| | | 410m 27s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3386/5/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/3386 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient spotbugs checkstyle codespell |
| uname | Linux a4c0da14ca37 4.15.0-147-generic #151-Ubuntu SMP Fri Jun 18
19:21:19 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / ae4fbd19d641d748971f976aa6c3a7773d10902d |
| Default Java | Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| Multi-JDK versions |
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
/usr/lib/jvm/java-8-openjdk-amd64:Private
Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| Test Results |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3386/5/testReport/ |
| Max. process+thread count | 2698 (vs. ulimit of 5500) |
| modules | C: hadoop-hdfs-project/hadoop-hdfs U:
hadoop-hdfs-project/hadoop-hdfs |
| Console output |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3386/5/console |
| versions | git=2.25.1 maven=3.6.3 spotbugs=4.2.2 |
[jira] [Work logged] (HDFS-16213) Flaky test TestFsDatasetImpl#testDnRestartWithHardLink
[
https://issues.apache.org/jira/browse/HDFS-16213?focusedWorklogId=646693=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-646693
]
ASF GitHub Bot logged work on HDFS-16213:
-
Author: ASF GitHub Bot
Created on: 05/Sep/21 12:45
Start Date: 05/Sep/21 12:45
Worklog Time Spent: 10m
Work Description: hadoop-yetus commented on pull request #3386:
URL: https://github.com/apache/hadoop/pull/3386#issuecomment-913148446
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 42s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 1s | | codespell was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 2 new or modified test files. |
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 30m 45s | | trunk passed |
| +1 :green_heart: | compile | 1m 22s | | trunk passed with JDK
Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 |
| +1 :green_heart: | compile | 1m 14s | | trunk passed with JDK
Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| +1 :green_heart: | checkstyle | 1m 0s | | trunk passed |
| +1 :green_heart: | mvnsite | 1m 25s | | trunk passed |
| +1 :green_heart: | javadoc | 0m 57s | | trunk passed with JDK
Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 |
| +1 :green_heart: | javadoc | 1m 27s | | trunk passed with JDK
Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| +1 :green_heart: | spotbugs | 3m 7s | | trunk passed |
| +1 :green_heart: | shadedclient | 16m 22s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 1m 12s | | the patch passed |
| +1 :green_heart: | compile | 1m 13s | | the patch passed with JDK
Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 |
| +1 :green_heart: | javac | 1m 13s | | the patch passed |
| +1 :green_heart: | compile | 1m 5s | | the patch passed with JDK
Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| +1 :green_heart: | javac | 1m 5s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| +1 :green_heart: | checkstyle | 0m 53s | |
hadoop-hdfs-project/hadoop-hdfs: The patch generated 0 new + 240 unchanged - 3
fixed = 240 total (was 243) |
| +1 :green_heart: | mvnsite | 1m 13s | | the patch passed |
| +1 :green_heart: | javadoc | 0m 47s | | the patch passed with JDK
Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 |
| +1 :green_heart: | javadoc | 1m 22s | | the patch passed with JDK
Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| +1 :green_heart: | spotbugs | 3m 11s | | the patch passed |
| +1 :green_heart: | shadedclient | 16m 2s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| -1 :x: | unit | 231m 2s |
[/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3386/1/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt)
| hadoop-hdfs in the patch passed. |
| +1 :green_heart: | asflicense | 0m 47s | | The patch does not
generate ASF License warnings. |
| | | 315m 6s | | |
| Reason | Tests |
|---:|:--|
| Failed junit tests |
hadoop.hdfs.server.balancer.TestBalancerWithHANameNodes |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3386/1/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/3386 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient spotbugs checkstyle codespell |
| uname | Linux 67947e04a1b9 4.15.0-151-generic #157-Ubuntu SMP Fri Jul 9
23:07:57 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / ae4fbd19d641d748971f976aa6c3a7773d10902d |
| Default Java | Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| Multi-JDK versions |
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
/usr/lib/jvm/java-8-openjdk-amd64:Private
Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| Test Results |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3386/1/testReport/ |
|
[jira] [Work logged] (HDFS-16213) Flaky test TestFsDatasetImpl#testDnRestartWithHardLink
[
https://issues.apache.org/jira/browse/HDFS-16213?focusedWorklogId=646682=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-646682
]
ASF GitHub Bot logged work on HDFS-16213:
-
Author: ASF GitHub Bot
Created on: 05/Sep/21 07:29
Start Date: 05/Sep/21 07:29
Worklog Time Spent: 10m
Work Description: virajjasani opened a new pull request #3386:
URL: https://github.com/apache/hadoop/pull/3386
### Description of PR
TestFsDatasetImpl#testDnRestartWithHardLink is flapper:
```
[ERROR]
testDnRestartWithHardLink(org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.TestFsDatasetImpl)
Time elapsed: 7.768 s <<< FAILURE!
java.lang.AssertionError
at org.junit.Assert.fail(Assert.java:87)
at org.junit.Assert.assertTrue(Assert.java:42)
at org.junit.Assert.assertTrue(Assert.java:53)
at
org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.TestFsDatasetImpl.testDnRestartWithHardLink(TestFsDatasetImpl.java:1344)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at
org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59)
at
org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
at
org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56)
at
org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
at
org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:299)
at
org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:293)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.lang.Thread.run(Thread.java:748)
```
### How was this patch tested?
Unit testing. The current flaky behaviour is easy to reproduce by running
the test code twice as part of same test.
The resolution is to disable the detection as well as deletion of duplicate
finalized replica by BlockPoolSlice instance.
When Datanode comes up, BPServiceActors handshakes to Namenode and tries to
initialize Block pool and in the process, it tries to get VolumeMap using
BlockPoolSlice instance. While doing so, reading replicas from cache fails and
hence, the thread tries to add Finalized and RBW replicas to
addReplicaThreadPool fork-join pool in order to build the map. This process
also tries to identify if there exists any duplicate replica. For this
particular test, sometimes this process can detect duplicate replica on /data2
while processing finalized replica of /data1. Hence, before we can confirm
newReplicaInfo.getBlockURI() exists, finalized replica on /data2 might get
deleted (rare and flaky case). Although the probability for the thread
processing the identification and deletion of duplicate finalized replica to be
faster than main thread is less, it cannot be avoided. Hence, we disable adding
Finalized and RBW replicas to addReplicaThreadPool in BlockPoolSlice here and
re-enable it only after we confirm the existence of newReplicaInfo on "/data2"
ARCHIVE storage.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 646682)
Remaining Estimate: 0h
Time Spent: 10m
> Flaky test TestFsDatasetImpl#testDnRestartWithHardLink
> --
>
> Key: HDFS-16213
> URL: https://issues.apache.org/jira/browse/HDFS-16213
> Project: Hadoop HDFS
> Issue Type: Bug
>Reporter: Viraj Jasani
>Assignee: Viraj Jasani
>Priority: Major
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Failure case:
> [here|https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3359/4/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt]
> {code:java}
> [ERROR]
> testDnRestartWithHardLink(org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.TestFsDatasetImpl)
> Time elapsed: 7.768 s <<< FAILURE![ERROR]
> testDnRestartWithHardLink(org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.TestFsDatasetImpl)
> Time elapsed: 7.768 s <<< FAILURE!java.lang.AssertionError at
> org.junit.Assert.fail(Assert.java:87) at
>
