joseluisll opened a new pull request, #8659:
URL: https://github.com/apache/hadoop/pull/8659
### Description of PR
Shuts down 67 leaked mini-cluster instances across 42 test files in
hadoop-hdfs-project, and fixes an assertion race in `TestFsVolumeList`.
A leaked cluster keeps its NameNode and DataNode threads, heap and ports
alive
under the remaining tests of the class, so one real failure turns into
several
bogus ones. The three categories are cluster never shut down (22), shut down
only on the happy path so it leaks precisely when the test fails (36), and
held
in a field with no teardown (9). The fixes are `try`-with-resources,
`shutdown()`
moved into `finally`, or an `@AfterEach`.
Two things a reviewer should look at:
`TestFsVolumeList` also gets an unrelated fix, without which it stays red.
Its
`testAddRplicaProcessorForAddingReplicaInMap` compares the configured pool
size
against `BlockPoolSlice#getAddReplicaForkPoolSize()`, which returned
`ForkJoinPool#getPoolSize()` — threads currently started, which the pool
grows
lazily — giving its `expected: <5> but was: <4>` failures. The
`@VisibleForTesting` getter now returns `getParallelism()`. This is the only
`src/main` change in the PR and the method has no production callers.
`TestNNWithQJM#testNewNamenodeTakesOverWriter` had its shutdown commented out
on purpose: the test leaves its NameNode fenced behind a second cluster
holding
the journal quorum, so closing the edit log terminates the JVM with "Could
not
sync enough journals to persistent storage". It is now shut down inside a
`try`/`catch` that tolerates that, so the rest of the class no longer runs
underneath a live fenced NameNode. This is the only judgement call in the
diff.
`TestFetchImage` is the one survey case deliberately left alone — same fenced
shutdown problem, but with a single `@Test` there is no later test to
protect,
so adding teardown would only break a class that passes today.
The remaining survey candidates are itemised in the JIRA: 4 need no fix by
design, 4 were scan false positives, and 11 are outside hadoop-hdfs-project
and
tracked separately.
### How was this patch tested?
Every touched test class was run on Ubuntu 24.04 with JDK
`17.0.19+10-1-24.04.2-Ubuntu`, the same JDK and OS as the precommit agents:
| Module | Classes | Tests | Result |
|---|---|---|---|
| hadoop-hdfs | 36 | 367 | green |
| hadoop-hdfs-rbf | 4 | 22 | green |
| hadoop-hdfs-nfs | 1 | 1 | green |
### For code changes:
- [x] Does the title of this PR start with the corresponding JIRA issue id?
- [ ] Object storage: N/A
- [x] If adding new dependencies … — no new dependencies
- [x] If applicable, have you updated the `LICENSE`… — N/A
### AI Tooling
Contains content generated by Claude Code.
- [x] The PR includes the phrase "Contains content generated by Claude Code"
- [x] My use of AI contributions follows the ASF legal policy
https://www.apache.org/legal/generative-tooling.html
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
