github-actions[bot] commented on code in PR #67781:
URL: https://github.com/apache/doris/pull/67781#discussion_r3986977907
##########
be/src/storage/storage_engine.cpp:
##########
@@ -158,6 +158,23 @@ void
BaseStorageEngine::_start_adaptive_thread_controller() {
config::max_flush_thread_num_per_cpu,
config::min_flush_thread_num_per_cpu);
}
+
+ // Keep the two delete bitmap queues independent, while reusing the flush
+ // policy and per-CPU thread limits.
+ auto add_delete_bitmap_pool = [this](const std::string& name,
+ CalcDeleteBitmapExecutor* executor) {
+ if (executor) {
+ auto* pool = executor->thread_pool();
+ _adaptive_thread_controller.add(name, {pool},
Review Comment:
[P1] Keep the mutable adaptive settings effective for these new groups.
`add()` copies the current max/min values into `PoolGroup`, but
`config::update_config()` only resizes the memtable/workload-group pools for a
max change and never updates either delete-bitmap pool/group (nor handles the
min here). For example, after starting at 4 threads/CPU, a supported runtime
change to 1 leaves both delete-bitmap pools and subsequent timer clamps at the
old 4 threads/CPU. Starting with adaptive mode disabled and then enabling the
mutable flag also leaves these groups unregistered. Please wire these
pools/controller bounds into runtime reconfiguration and cover both limit
changes and disabled-to-enabled activation.
##########
be/src/storage/delete/calc_delete_bitmap_executor.cpp:
##########
@@ -95,8 +98,17 @@ Status CalcDeleteBitmapToken::wait() {
}
void CalcDeleteBitmapExecutor::init(const std::string& name, int max_threads) {
+ int min_threads = 1;
+ if (config::enable_adaptive_flush_threads) {
+ int num_cpus = std::thread::hardware_concurrency();
Review Comment:
[P1] Size these pools from Doris's effective CPU count.
`std::thread::hardware_concurrency()` can expose the host's online CPUs and
bypasses the cgroup quota/cpuset and `num_cores` override that
`CpuInfo::num_cores()` applies. On a 64-CPU host with an 8-CPU BE limit, the
defaults initialize *each* new pool at min=32/max=256 (and the controller
recomputes the same bounds), rather than min=4/max=32; the two pools can then
retain 64 workers and grow to 512 while only 8 CPUs are available. Please use
the cgroup-aware count consistently for initialization and controller bounds,
and add a limited-CPU test.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
