wang-jiahua commented on PR #10448: URL: https://github.com/apache/rocketmq/pull/10448#issuecomment-4796706256
Thanks for the suggestion! We considered the approach of reusing the single-topic delete path with a `RateLimiter`, but ran benchmarks to compare both approaches. **Why batch persist instead of loop + RateLimiter:** The single-topic delete path (`deleteTopicConfig` / `deleteSubscriptionGroupConfig`) calls `persist()` each time, which performs 3 `fsync` operations (file write + 2 directory fsyncs). Looping N times means 3N fsync calls, which is the main bottleneck. The batch approach removes all entries from memory first, then calls `persist()` once — reducing 3N fsync calls to 3. **Server benchmark results (50 topics/groups, with Producer+Consumer load):** | Metric | Loop delete (50 topics) | Batch delete (50 topics) | |--------|------------------------|-------------------------| | Delete time | 19,455ms | 110ms (**177x**) | | Producer TPS | 139k (-22%) | 171k (0%) | | Producer P99 RT | 6ms | 5ms | | Consumer TPS | 141k (-21%) | 176k (0%) | | GC delta | +34 | 0 | | Metric | Loop delete (50 groups) | Batch delete (50 groups) | |--------|------------------------|-------------------------| | Delete time | 221,128ms | 501ms (**441x**) | | Producer TPS | 166k (-3%) | 171k (0%) | | Consumer Max RT | 511ms (+119%) | 289ms (0%) | | GC delta | +1,108 | +408 (-63%) | **Unit test (persist call count):** 100 loop persists = 100ms, 1 batch persist = 1ms (**97x**). **Regarding `RateLimiter`:** The loop approach with `RateLimiter.create(10.0)` would take ~5 seconds for 50 topics (at 10/s), still 45x slower than batch (110ms). For 50 groups it would take ~5 seconds vs 501ms batch. The batch approach is faster and has zero TPS/RT impact. **Regarding regression tests:** Added unit tests in the PR covering batch delete with valid entries, empty list, and non-existent entries. Would appreciate your feedback on this approach! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
