mrhhsg opened a new pull request, #67845:
URL: https://github.com/apache/doris/pull/67845
### What problem does this PR solve?
Issue Number: None
Related PR: #62485
Problem Summary:
When a query is paused because of `PROCESS_MEMORY_EXCEEDED` and it has no
revocable memory, `WorkloadGroupMgr::handle_single_query_()` cancelled it
immediately if the process was still above the soft memory limit. The
memory maintenance thread runs every 50ms by default, so a query that just
finished spilling everything it could was cancelled about 50-100ms later,
even when it only used a few dozen MiB and cancelling it did not relieve
the process memory pressure at all.
The timeout fallback added by #62485 in `handle_process_memory_exceeded_()`
only applies when the workload group uses less than its min memory limit.
With the default `min_memory_percent = 0` that branch is never reached,
and the query goes straight to `handle_single_query_()`, which cancels it
at once.
This PR makes the process memory branch of `handle_single_query_()` keep
the query paused until it has waited `spill_in_paused_queue_timeout_ms`,
re-checking the soft memory limit every round so the query resumes as soon
as other queries release memory. The query is cancelled only after the
timeout. Memory gc still cancels queries when the process reaches the hard
memory limit, so the wait is bounded.
The cancellation message is also fixed: it claimed to "disable reserve
memory and resume" the query while it actually cancelled it.
### Release note
Queries paused under process memory pressure without revocable memory now
wait up to `spill_in_paused_queue_timeout_ms` for memory to be released
before being cancelled.
### Check List (For Author)
- Test
- [ ] Regression test
- [x] Unit Test
- [ ] Manual test (add detailed scripts or steps below)
- [ ] No need to test or manual test. Explain why:
- Behavior changed:
- [ ] No.
- [x] Yes. Queries paused due to process memory exceeded are kept paused
until `spill_in_paused_queue_timeout_ms` instead of being cancelled immediately
when no memory can be revoked.
- Does this need documentation?
- [x] No.
- [ ] Yes.
https://claude.ai/code/session_014c64WZPFui9AyvVMiseceY
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
