Ben-Zvi commented on a change in pull request #1351: DRILL-6543: Disable Hash
Join fallback, add percent_reserved_allowance_from_direct
URL: https://github.com/apache/drill/pull/1351#discussion_r199985944
##########
File path:
exec/java-exec/src/main/java/org/apache/drill/exec/util/MemoryAllocationUtilities.java
##########
@@ -138,16 +139,36 @@ public static long computeOperatorMemory(OptionSet
optionManager, long maxAllocP
@VisibleForTesting
public static long computeQueryMemory(DrillConfig config, OptionSet
optionManager, long directMemory) {
+ // Get the options
+ double percent_per_query =
optionManager.getOption(ExecConstants.PERCENT_MEMORY_PER_QUERY);
+ long max_query_per_node =
optionManager.getOption(ExecConstants.MAX_QUERY_MEMORY_PER_NODE);
+ double percent_allowance =
optionManager.getOption(ExecConstants.PERCENT_RESERVED_ALLOWANCE_FROM_DIRECT);
+
+ // verify that the allowance is kept
+ if ( percent_per_query + percent_allowance > 1.0 ) {
Review comment:
The code in the PR "enforces" keeping the allowance (out of the direct
memory); any user settings that violates this limit returns an error
(unfortunately can only be done when a query (using buffered ops) is executed).
This mainly serves as a reminder (for the common user that does not read
the documentation). It does not help with concurrent query execution (though
users using concurrent are usually more sophisticated, so may know about the
allowance).
The example suggested above applies the *allowance* onto the final
computed memory, which is not the intention. For example, Direct Memory of
20GB, and Mem Per Query of 4GB -- then the code about would subtract 25% and
the Mem Per Query would be only 3GB (and confuse the user).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
