Yukang-Lian opened a new pull request, #49883:
URL: https://github.com/apache/doris/pull/49883
Pick #48759
This PR introduces a delay strategy for cumulative compaction tasks that
protects the thread pool from becoming overwhelmed by resource-intensive
operations.
The new `should_delay_submission()` function implements a delay strategy
that:
- Only activates when the thread pool size meets a configurable minimum
threshold
- Checks if the thread pool is near capacity (0-1 threads remaining)
- Evaluates whether the compaction task is resource-intensive based on input
size and row count
When these conditions are met, the function returns true to indicate the
submission should be delayed, preventing potential thread pool exhaustion and
system performance degradation.
The implementation includes three configurable thresholds:
- `min_threads_for_cumu_delay_strategy`: Minimum thread pool size to
activate the strategy
- `cumu_delay_strategy_size`: Maximum input size threshold (in bytes)
- `cumu_delay_strategy_row_num`: Maximum input row count threshold
Appropriate warning logs are generated when a resource-intensive task is
delayed, including the tablet ID and resource metrics for easier
troubleshooting.
### What problem does this PR solve?
Issue Number: close #xxx
Related PR: #xxx
Problem Summary:
### Release note
None
### Check List (For Author)
- Test <!-- At least one of them must be included. -->
- [ ] Regression test
- [ ] Unit Test
- [ ] Manual test (add detailed scripts or steps below)
- [ ] No need to test or manual test. Explain why:
- [ ] This is a refactor/code format and no logic has been changed.
- [ ] Previous test can cover this change.
- [ ] No code files have been changed.
- [ ] Other reason <!-- Add your reason? -->
- Behavior changed:
- [ ] No.
- [ ] Yes. <!-- Explain the behavior change -->
- Does this need documentation?
- [ ] No.
- [ ] Yes. <!-- Add document PR link here. eg:
https://github.com/apache/doris-website/pull/1214 -->
### Check List (For Reviewer who merge this PR)
- [ ] Confirm the release note
- [ ] Confirm test cases
- [ ] Confirm document
- [ ] Add branch pick label <!-- Add branch pick label that this PR should
merge into -->
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
