[
https://issues.apache.org/jira/browse/IMPALA-10860?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17757872#comment-17757872
]
ASF subversion and git services commented on IMPALA-10860:
----------------------------------------------------------
Commit 05bc485851a255b1b00bdb1a1086ba5527877768 in impala's branch
refs/heads/master from Abhishek Rawat
[ https://gitbox.apache.org/repos/asf?p=impala.git;h=05bc48585 ]
IMPALA-10860: Allow setting mem_limit for coordinators
Added support for MEM_LIMIT_COORDINATORS query option. This is
similar to exisiting MEM_LIMIT_EXECUTORS, but applies to coordinators.
There are cases where Planner generates inaccurate estimates for
coordinator fragments and would be good to be able to set mem limit
just for the coordinator, since a query's memory requirement on
coordinator tends to be much lower compared to that on executors.
If MEM_LIMIT is set, then MEM_LIMIT_COORDINATORS is ignored.
Also updated the documentation for the new query option.
Testing:
- Added new custom cluster tests which validates MEM_LIMIT_COORDINATORS
applies only on coordinator. The test also validates that both
MEM_LIMIT_EXECUTORS and MEM_LIMIT_COORDINATORS can be set together.
- Built docs and made sure that the new changes have proper formatting.
Change-Id: I2dfc9a735e82dce2fd903bdaf6bc2e46e982ef8c
Reviewed-on: http://gerrit.cloudera.org:8080/20378
Reviewed-by: Impala Public Jenkins <[email protected]>
Tested-by: Impala Public Jenkins <[email protected]>
> Allow setting separate mem_limit for coordinators
> -------------------------------------------------
>
> Key: IMPALA-10860
> URL: https://issues.apache.org/jira/browse/IMPALA-10860
> Project: IMPALA
> Issue Type: Task
> Reporter: Abhishek Rawat
> Assignee: Abhishek Rawat
> Priority: Major
>
> The mem_limit query option applies to all impala coordinators and executors.
> This may not be ideal for dedicated coordinators, since they generally need
> less memory per query and having same memory limit will reduce system wide
> query concurrency.
> We could add new query options:
>
> {code:java}
> mem_limit_coordinators
> {code}
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
