[
https://issues.apache.org/jira/browse/HDFS-16740?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17624080#comment-17624080
]
ASF GitHub Bot commented on HDFS-16740:
---------------------------------------
xinglin commented on code in PR #4835:
URL: https://github.com/apache/hadoop/pull/4835#discussion_r1005061516
##########
hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestDecommissionWithStriped.java:
##########
@@ -117,10 +121,8 @@ protected Configuration createConfiguration() {
public void setup() throws IOException {
conf = createConfiguration();
// Set up the hosts/exclude files.
- localFileSys = FileSystem.getLocal(conf);
- Path workingDir = localFileSys.getWorkingDirectory();
- decommissionDir = new Path(workingDir,
- PathUtils.getTestDirName(getClass()) + "/work-dir/decommission");
+ localFileSys = FileSystem.get(baseDir.getRoot().toURI(), conf);
Review Comment:
Will FileSystem.get() return a LocalFileSystem object here? what scheme does
baseDir.getRoot().toURI() return?
##########
hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestRollingUpgrade.java:
##########
@@ -83,37 +83,18 @@ public static void runCmd(DFSAdmin dfsadmin, boolean
success,
}
@Rule
- public TemporaryFolder folder = new TemporaryFolder();
-
- /**
- * Create a default HDFS configuration which has test-specific data
directories. This is
- * intended to protect against interactions between test runs that might
corrupt results. Each
- * test run's data is automatically cleaned-up by JUnit.
- *
- * @return a default configuration with test-specific data directories
- */
- public Configuration getHdfsConfiguration() throws IOException {
- Configuration conf = new HdfsConfiguration();
-
- // Override the file system locations with test-specific temporary folders
- conf.set(DFSConfigKeys.DFS_NAMENODE_NAME_DIR_KEY,
Review Comment:
Why is the case that we no longer need to set these properties anymore? Is
it because now NAME_DIR/DATA_DIR/CHECKPOINT_DIR will all defaulted to be within
the new baseDir?
> Mini cluster test flakiness
> ---------------------------
>
> Key: HDFS-16740
> URL: https://issues.apache.org/jira/browse/HDFS-16740
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: hdfs, test
> Affects Versions: 3.4.0, 3.3.5
> Reporter: Steve Vaughan
> Assignee: Steve Vaughan
> Priority: Major
> Labels: pull-request-available
>
> Mini clusters used during HDFS unit tests are reporting test failures that do
> not appear to be directly related to submitted changes. The failures are the
> result of either interactions between tests run in parallel, or tests which
> share common disk space for tests. In all cases, the tests can be run
> individually serially without any errors. Addressing this issue will
> simplify future submissions by eliminating the confusion introduced by these
> unrelated test failures.
> We can apply lessons recently from TestRollingUpgrade, which was recently
> patched to unblock a recent submission. The fixes involved changing the HDFS
> configuration to use temporary disk space for each individual tests, and
> using try-with-resources to ensure that clusters were shutdown cleanly.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
