wyxxxcat opened a new pull request, #69675: URL: https://github.com/apache/doris/pull/69675
### What problem does this PR solve? Problem Summary: Rowset recycling uses fixed worker concurrency even when MetaService or FDB is under pressure, so background recycling continues competing for resources without reducing its concurrency. Add an AdaptiveThreadPool that uses the existing MetaService stress decision to halve concurrency under pressure and gradually restore it with randomized increments after pressure subsides. Keep concurrency within configured bounds and limit queue capacity to twice the current concurrency to apply backpressure to submitters. Use the adaptive pool for regular, versioned, and temporary rowset recycling. Enable it by default, with a configuration switch to retain the fixed-size pool. Add configurable adjustment intervals, minimum concurrency, expansion steps, and a concurrency metric. ### Release note Rowset recycling now automatically reduces concurrency when MetaService or FDB is under pressure and gradually restores it afterward. Set enable_recycler_adaptive_thread_pool=false to use fixed concurrency; the setting takes effect when a new worker pool is created. ### Check List (For Author) - Test: Unit tests added for pool lifecycle, queue backpressure, submission timeouts, exception handling, concurrency adjustment and bounds, adjustment intervals, metrics, and both worker pool modes. Test execution status has not been verified. - Behavior changed: Yes. Adaptive concurrency is enabled by default for rowset recycling. - Does this need documentation: Yes. Document the new configuration options and concurrency metric; documentation is not included here. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
