yew1eb opened a new pull request, #3817: URL: https://github.com/apache/celeborn/pull/3817
### What changes were proposed in this pull request? Test-only changes to cut CI wall time: 1. **DataPushQueueSuiteJ**: reduce `numPartitions` from 1,000,000 to 30,000. 2. **QuotaManagerSuite**: reduce case1/case2 scale from 1000 users × 1000 applications to 100 × 100 (and the inner check loop accordingly). 3. **RatisMasterStatusSystemSuiteJ** (plus its SSL and gRPC variants): add an `awaitCondition(BooleanSupplier)` helper (100ms poll interval, 30s upper bound, falls back to the original assertion on timeout) and convert 38 fixed `Thread.sleep` calls — including the hard 15s leader-election wait — to condition polling. One negative assertion that must observe state *not* changing keeps a (reduced) fixed wait. 4. **spark-it**: stop leaked `LifecycleManager`/`ShuffleClient` instances in `LifecycleManagerReserveSlotsSuite`, `LifecycleManagerSuite` and `ChangePartitionManagerUpdateWorkersSuite`. The key fix is adding the missing `super.afterEach()` in `ChangePartitionManagerUpdateWorkersSuite`: its parent `WithShuffleClientSuite` creates LifecycleManagers via `prepareService()` and relies on the trait's `afterEach` to stop them, so the override leaked 4 LifecycleManagers permanently. ### Why are the changes needed? Several tests dominate CI time due to artificial data scale, fixed sleeps, and leaked LifecycleManagers whose app-heartbeat threads keep flooding `AskSync HeartbeatFromApplication failed` for the rest of the forked JVM (forkMode=once), stealing CPU from subsequent suites. Measured locally: - DataPushQueueSuiteJ: 28min → ~56s - QuotaManagerSuite: ~11min → ~3s - RatisMasterStatusSystemSuiteJ: 3×180s → 3×~53s - Full spark-it run: BUILD SUCCESS, `HeartbeatFromApplication failed` drops from a continuous storm to 6 benign shutdown-race occurrences - Full sbt test run: 25m26s ### Does this PR resolve a correctness bug? - [ ] Yes ### Does this PR introduce _any_ user-facing change? - [ ] Yes ### How was this patch tested? - All modified suites pass locally: DataPushQueueSuiteJ 2/2, QuotaManagerSuite 7/7, RatisMasterStatusSystemSuiteJ + SSL + gRPC variants all green - Full `tests/spark-it` run (`-Pspark-3.5`): BUILD SUCCESS - Full sbt test run passes (two pre-existing macOS-local environment failures unrelated to this change: EPOLL/KQUEUE config test and an SSL data-writer test) - `spotless:check` passes for all touched modules -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
