[
https://issues.apache.org/jira/browse/IMPALA-13083?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17848917#comment-17848917
]
ASF subversion and git services commented on IMPALA-13083:
----------------------------------------------------------
Commit 98739a84557a209e05694abd79f62f7f7daf8777 in impala's branch
refs/heads/master from Riza Suminto
[ https://gitbox.apache.org/repos/asf?p=impala.git;h=98739a845 ]
IMPALA-13083: Clarify REASON_MEM_LIMIT_TOO_LOW_FOR_RESERVATION
This patch improves REASON_MEM_LIMIT_TOO_LOW_FOR_RESERVATION error
message by saying the specific configuration that must be adjusted such
that the query can pass the Admission Control. New fields
'per_backend_mem_to_admit_source' and
'coord_backend_mem_to_admit_source' of type MemLimitSourcePB are added
into QuerySchedulePB. These fields explain what limiting factor drives
final numbers at 'per_backend_mem_to_admit' and
'coord_backend_mem_to_admit' respectively. In turn, Admission Control
will use this information to compose a more informative error message
that the user can act upon. The new error message pattern also
explicitly mentions "Per Host Min Memory Reservation" as a place to look
at to investigate memory reservations scheduled for each backend node.
Updated documentation with examples of query rejection by Admission
Control and how to read the error message.
Testing:
- Add BE tests at admission-controller-test.cc
- Adjust and pass affected EE tests
Change-Id: I1ef7fb7e7a194b2036c2948639a06c392590bf66
Reviewed-on: http://gerrit.cloudera.org:8080/21436
Reviewed-by: Impala Public Jenkins <impala-public-jenk...@cloudera.com>
Tested-by: Impala Public Jenkins <impala-public-jenk...@cloudera.com>
> Clarify REASON_MEM_LIMIT_TOO_LOW_FOR_RESERVATION error message
> --------------------------------------------------------------
>
> Key: IMPALA-13083
> URL: https://issues.apache.org/jira/browse/IMPALA-13083
> Project: IMPALA
> Issue Type: Improvement
> Components: Distributed Exec
> Reporter: Riza Suminto
> Assignee: Riza Suminto
> Priority: Major
>
> REASON_MEM_LIMIT_TOO_LOW_FOR_RESERVATION error message is too vague for
> user/administrator to make necessary adjustment to run query that is rejected
> by admission-controller.
> {code:java}
> const string REASON_MEM_LIMIT_TOO_LOW_FOR_RESERVATION =
> "minimum memory reservation is greater than memory available to the query
> for buffer "
> "reservations. Memory reservation needed given the current plan: $0.
> Adjust either "
> "the mem_limit or the pool config (max-query-mem-limit,
> min-query-mem-limit) for the "
> "query to allow the query memory limit to be at least $1. Note that
> changing the "
> "mem_limit may also change the plan. See the query profile for more
> information "
> "about the per-node memory requirements.";
> {code}
> There are many config and options that directly and indirectly clamp
> schedule.per_backend_mem_limit() and schedule.per_backend_mem_to_admit().
> [https://github.com/apache/impala/blob/3b35ddc8ca7b0e540fc16c413a170a25e164462b/be/src/scheduling/schedule-state.cc#L262-L361]
> Ideally, this error message should clearly mention which query option / llama
> config / backend flag that influence per_backend_mem_limit decision so that
> user can make directly make adjustment on that config. It should also clearly
> mention 'Per Host Min Memory Reservation' info string at query profile
> instead of just 'per-node memory requirements'.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
