[
https://issues.apache.org/jira/browse/MAPREDUCE-7475?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17842623#comment-17842623
]
ASF GitHub Bot commented on MAPREDUCE-7475:
-------------------------------------------
steveloughran commented on code in PR #6785:
URL: https://github.com/apache/hadoop/pull/6785#discussion_r1586258741
##########
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapreduce/TestNewCombinerGrouping.java:
##########
@@ -105,7 +106,8 @@ public int compare(Text o1, Text o2) {
@Test
public void testCombiner() throws Exception {
- if (!new File(TEST_ROOT_DIR).mkdirs()) {
+ File testDir = new File(TEST_ROOT_DIR);
Review Comment:
best to do the fully delete here, so mkdirs always works, even if test was
stopped partway through
##########
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapreduce/TestNewCombinerGrouping.java:
##########
@@ -174,6 +176,9 @@ public void testCombiner() throws Exception {
} else {
Assert.fail("Job failed");
}
+ if (testDir.exists()) {
Review Comment:
this doesn't get invoked if the job failed, and as there's no cleanup before
L109, no recovery.
> 2 tests are non-idempotent (passes in the first run but fails in repeated
> runs in the same JVM)
> -----------------------------------------------------------------------------------------------
>
> Key: MAPREDUCE-7475
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-7475
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Environment: Ubuntu 22.04, Java 17
> Reporter: Kaiyao Ke
> Priority: Major
> Labels: pull-request-available
> Original Estimate: 10m
> Remaining Estimate: 10m
>
> 2 tests are not idempotent and fails upon repeated execution within the same
> JVM instance due to self-induced state pollution. Specifically, these tests
> try to make the directory TEST_ROOT_DIR and write to it. The tests do not
> clean up (remove) the directory after execution. Therefore, in the second
> execution, TEST_ROOT_DIR would already exist and the exception `Could not
> create test dir` would be thrown. Below are the 2 non-idempotent tests:
> * org.apache.hadoop.mapred.TestOldCombinerGrouping.testCombiner
> * org.apache.hadoop.mapreduce.TestNewCombinerGrouping.testCombiner
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
