yew1eb opened a new pull request, #3882: URL: https://github.com/apache/celeborn/pull/3882
### What changes were proposed in this pull request? Stop the LifecycleManager instances leaked by spark-it client suites: - `LifecycleManagerSuite` "CELEBORN-1258" test: add `lifecycleManager.stop()` at the end of the test. - `ChangePartitionManagerUpdateWorkersSuite` "verify partition location storage info" test: add `lifecycleManager.stop()` at the end of the test. - `LifecycleManagerReserveSlotsSuite` (its only test): add `lifecycleManager.stop()` at the end of the test. - `ChangePartitionManagerUpdateWorkersSuite.afterEach()` now calls `super.afterEach()` before shutting down the mini cluster, so the managers created by `WithShuffleClientSuite`'s base tests (via `prepareService()`) are stopped. ### Why are the changes needed? Several spark-it suites create a `LifecycleManager` but never `stop()` it. Each leaked manager's `ApplicationHeartbeater` keeps heartbeating the already-stopped master for the rest of the test run, producing ~1200 ERROR stack traces — about a third of the CI test log — which drowns out real failures. The `afterEach()` case is the largest source: `ChangePartitionManagerUpdateWorkersSuite` overrides `afterEach()` to shut down the mini cluster without calling `super`, so every base test's manager leaks and errors every ~20s until JVM exit. The cleanup follows the pattern already used by the other client suites (e.g. `LifecycleManagerCommitFilesSuite`), which stop the manager at the end of each test. JIRA: [CELEBORN-2487](https://issues.apache.org/jira/browse/CELEBORN-2487) ### Does this PR resolve a correctness bug? - [ ] Yes ### Does this PR introduce _any_ user-facing change? - [ ] Yes ### How was this patch tested? Ran the three affected suites with Spark 4.2: ``` build/mvn -Pspark-4.2,google-mirror -pl tests/spark-it test \ -DwildcardSuites=org.apache.celeborn.tests.client.LifecycleManagerSuite,org.apache.celeborn.tests.client.LifecycleManagerReserveSlotsSuite,org.apache.celeborn.tests.client.ChangePartitionManagerUpdateWorkersSuite ``` 17 tests pass, and the `ApplicationHeartbeater: AskSync HeartbeatFromApplication failed` / `MasterClient: Send rpc with failure` ERROR stacks are gone (before the fix they were reproduced every ~20s until JVM exit). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
