weizuo93 opened a new issue #5779: URL: https://github.com/apache/incubator-doris/issues/5779
Now, the maximum and minimum number of threads (`max_thread_num` and `min_thread_num`) in the thread pool are initialized once and cannot be changed at runtime. Sometimes, we need to adjust `max_threads` or `min_threads`dynamically according to machine load. （For example, we need to adjust compaction thread number according to memory usage or cpu load sometimes） The current mechanism of thread pool is as follows: (1) When the thread pool is initialized, multiple threads (which we call `permanent threads`) are created based on the `min_thread_num`. (2) When more tasks need to be executed and `permanent threads` cannot meet the demand, new threads are created which we call `non-permanent threads`. The total number of threads will not exceed `max_thread_num`, that is: ``` permanent_threads_num + non-permanent_threads_num <= max_thread_num ``` (3)When the number of tasks reduced and there are idle threads, idle `non-permanent threads`will be released but idle `permanent threads`will never be released even if there is no task in `task queue`. We can easily adjust `max_thread_num`and `min_thread_num`at runtime. New `max_thread_num` can work well, but sometimes new`min_thread_num`is hard to work because `permanent threads`, which have been created when initialization, can not be released. So can we modify the thread pool model as follows: (1) When the thread pool is initialized, multiple threads (which we call `initial threads`) are created based on the `min_thread_num`. (2) When more tasks need to be executed and current threads cannot meet the demand, new threads are created which we call `subsequent threads`. The total number of thread will not exceed `max_thread_num` , that is: ``` initial_threads_num + subsequent_threads_num <= max_thread_num ``` (3) When the number of tasks reduced and there are idle threads, both `initial threads`and `subsequent threads`could be released so long as total number of thread is not less than `min_thread_num`. In this model, `initial threads` and `subsequent threads`will have equal status for thread pool. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
