[
https://issues.apache.org/jira/browse/HDFS-16740?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17621282#comment-17621282
]
ASF GitHub Bot commented on HDFS-16740:
---------------------------------------
steveloughran commented on code in PR #4835:
URL: https://github.com/apache/hadoop/pull/4835#discussion_r1000928173
##########
hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestRollingUpgrade.java:
##########
@@ -83,37 +83,18 @@ public static void runCmd(DFSAdmin dfsadmin, boolean
success,
}
@Rule
- public TemporaryFolder folder = new TemporaryFolder();
-
- /**
- * Create a default HDFS configuration which has test-specific data
directories. This is
- * intended to protect against interactions between test runs that might
corrupt results. Each
- * test run's data is automatically cleaned-up by JUnit.
- *
- * @return a default configuration with test-specific data directories
- */
- public Configuration getHdfsConfiguration() throws IOException {
- Configuration conf = new HdfsConfiguration();
Review Comment:
retain this, but just return the new config. allows for changes later
##########
hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/qjournal/MiniJournalCluster.java:
##########
@@ -64,7 +65,17 @@ public static class Builder {
public Builder(Configuration conf) {
this.conf = conf;
}
-
+
+ public Builder(Configuration conf, TemporaryFolder baseDir) {
Review Comment:
i don't want to add junit dependencies here. better to take a File ref and
pass it in when used
##########
hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/MiniDFSCluster.java:
##########
@@ -240,6 +241,10 @@ public Builder(Configuration conf) {
}
}
+ public Builder(Configuration conf, TemporaryFolder baseDir) {
Review Comment:
i don't want to add junit dependencies here; we don't know where else it is
used. and test dependencies don't get exported by maven. the code will need to
be given baseDir.getRoot()
> Mini cluster test flakiness
> ---------------------------
>
> Key: HDFS-16740
> URL: https://issues.apache.org/jira/browse/HDFS-16740
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: hdfs, test
> Affects Versions: 3.4.0, 3.3.5
> Reporter: Steve Vaughan
> Assignee: Steve Vaughan
> Priority: Major
> Labels: pull-request-available
>
> Mini clusters used during HDFS unit tests are reporting test failures that do
> not appear to be directly related to submitted changes. The failures are the
> result of either interactions between tests run in parallel, or tests which
> share common disk space for tests. In all cases, the tests can be run
> individually serially without any errors. Addressing this issue will
> simplify future submissions by eliminating the confusion introduced by these
> unrelated test failures.
> We can apply lessons recently from TestRollingUpgrade, which was recently
> patched to unblock a recent submission. The fixes involved changing the HDFS
> configuration to use temporary disk space for each individual tests, and
> using try-with-resources to ensure that clusters were shutdown cleanly.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
