Tim Armstrong created IMPALA-7297:
-------------------------------------
Summary: Reject reservation increase if it would exceed soft limit
Key: IMPALA-7297
URL: https://issues.apache.org/jira/browse/IMPALA-7297
Project: IMPALA
Issue Type: Improvement
Components: Backend
Reporter: Tim Armstrong
Assignee: Tim Armstrong
Currently the MemTracker only has a hard limit, which is applied to any memory
increase.
One problem we have with the interaction between reserved and non-reserved
memory is that the reserved memory is capped to grow up to 80% of the query
memory limit unless it hits the query memory limit. If non-reserved memory
grows to > 20% of the query memory limit and reserved memory grows until it
hits the hard limit, then we almost always end up with a query failure because
there's no headroom for additional non-reserved allocations.
We could reduce query failures if reservation increases were rejected earlier
to preserve some headroom for non-reserved allocations above 20% of the query
memory limit.
The proposal is that reservation increases (aside from the initial reservation)
should be rejected if they exceed (x * limit). We could determine x with some
experimentation, but 5-10% might be a good value.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
