matanper opened a new issue, #70452: URL: https://github.com/apache/doris/issues/70452
### Search before asking - [x] I had searched in the [issues](https://github.com/apache/doris/issues?q=is%3Aissue) and found no similar issues. ### Version 4.1.4. master (`9fb3e3a2a13`) has the same code. ### Description With memory reservation enabled (`enable_reserve_memory`), a pipeline task reserves the sum of `get_reserve_mem_size()` over its operators before it runs. `LocalExchangeSourceOperatorX` doesn't override it, so it gets the default from `OperatorX::get_reserve_mem_size` (`be/src/exec/operator/operator.h:916` on master), which is `state->minimum_operator_memory_required_bytes()`. A plain pass-through local exchange source doesn't need that memory. `PassthroughExchanger::get_block` (`be/src/exec/exchange/local_exchanger.cpp:328`) takes a block already allocated by the sink side and hands it over with `swap` (`:344`). The source allocates nothing new. Because of this, a small pipeline such as `LOCAL_EXCHANGE_OPERATOR(PASSTHROUGH) -> HASH_JOIN_OPERATOR` reserves twice the per-operator minimum, half of it for the exchange source alone. With a larger `minimum_operator_memory_required_kb`, that is tens of MiB for a lookup that uses very little memory. When the process is near its soft memory limit, the oversized reservation fails and the query is paused (`PROCESS_MEMORY_EXCEEDED`). It stays paused until memory recovers or the paused-query handling gets to it, even though the memory it actually needs would have fit. We see small point lookups paused this way for seconds under memory pressure. ### Solution Override `get_reserve_mem_size` in `LocalExchangeSourceOperatorX`. Return 0 for a `PASSTHROUGH` exchange with no projection (`has_output_row_desc()` is false) and no limit (`_limit == -1`). Otherwise keep the default, because projections and limit truncation can allocate. Other exchange types keep their reservation, since they copy or build blocks: - hash/bucket shuffle; - broadcast; - pass-to-one; - adaptive pass-through, which can split or copy. The pipeline then reserves only what its other operators need. We have a patch and can open a PR. ### How to Reproduce? 1. Enable memory reservation and set `minimum_operator_memory_required_kb` so that two operators' worth (2 × the minimum) does not fit in the remaining process headroom but one operator's worth does. For example, set the process soft limit to current usage + 48 MiB, with a 32000 KiB minimum. 2. Run a query whose plan contains a `LOCAL_EXCHANGE_OPERATOR(PASSTHROUGH)` source feeding one other operator, e.g. a hash join probe after a pass-through local exchange. 3. The pipeline's reservation of 2 × the minimum fails and the query pauses with `PROCESS_MEMORY_EXCEEDED`, although only one operator needs memory. In a BE unit test that uses the real `GlobalMemoryArbitrator` with 48 MiB of headroom and a 32000 KiB minimum: - current code: the 62.5 MiB reservation (two operators) fails; - patched: 31.25 MiB is reserved and succeeds. The test also checks that a pass-through source with a projection or a limit, and the copying exchange types, keep their reservation. ### Anything Else? This only changes how much memory is requested up front. Memory that is actually allocated is still tracked and limited as before. ### Are you willing to submit PR? - [x] Yes I am willing to submit a PR! ### Code of Conduct - [x] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
