[
https://issues.apache.org/jira/browse/DRILL-6513?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16517681#comment-16517681
]
ASF GitHub Bot commented on DRILL-6513:
---------------------------------------
rhou1 commented on a change in pull request #1329: DRILL-6513: Max query memory
per node set to Drillbit's max direrct memory
URL: https://github.com/apache/drill/pull/1329#discussion_r196626074
##########
File path:
exec/java-exec/src/main/java/org/apache/drill/exec/ExecConstants.java
##########
@@ -440,7 +441,7 @@ private ExecConstants() {
* DEFAULT: 2048 MB
*/
public static final String MAX_QUERY_MEMORY_PER_NODE_KEY =
"planner.memory.max_query_memory_per_node";
- public static final LongValidator MAX_QUERY_MEMORY_PER_NODE = new
RangeLongValidator(MAX_QUERY_MEMORY_PER_NODE_KEY, 1024 * 1024, Long.MAX_VALUE);
+ public static final LongValidator MAX_QUERY_MEMORY_PER_NODE = new
RangeLongValidator(MAX_QUERY_MEMORY_PER_NODE_KEY, 1024 * 1024,
DrillConfig.getMaxDirectMemory());
Review comment:
Yes, this change is not perfect by any means. It is just a step towards a
better solution.
A better solution is to have a global Resource Manager determine the right
setting, where the Resource Manager knows what each node is capable of. But we
don't have that yet.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
> Drill should only allow valid values when users set
> planner.memory.max_query_memory_per_node
> --------------------------------------------------------------------------------------------
>
> Key: DRILL-6513
> URL: https://issues.apache.org/jira/browse/DRILL-6513
> Project: Apache Drill
> Issue Type: Bug
> Components: Query Planning & Optimization
> Reporter: salim achouche
> Assignee: salim achouche
> Priority: Major
> Labels: pull-request-available
> Fix For: 1.14.0
>
>
> The "planner.memory.max_query_memory_per_node" configuration can be currently
> set to values higher than the Drillbit Direct Memory configuration. The goal
> of this Jira is to fail queries with such an erroneous configuration to avoid
> runtime OOM.
> NOTE - The current semantic of the maximum query memory per node
> configuration is that the end user has computed valid values especially
> knowing the current Drill limitations. Such values have to account for
> Netty's overhead (memory pools), shared pools (e.g., network exchanges), and
> concurrent query execution. This Jira should not be used to also cover such
> use-cases. The Drill Resource Management feature has the means to automate
> query quotas and the associated validation. We should create another Jira
> requesting the enhanced validations contracts under the umbrella of the
> Resource Management feature.
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
