ilooner commented on a change in pull request #1351: DRILL-6543: Disable Hash
Join fallback, add percent_reserved_allowance_from_direct
URL: https://github.com/apache/drill/pull/1351#discussion_r199913761
##########
File path:
exec/java-exec/src/main/java/org/apache/drill/exec/util/MemoryAllocationUtilities.java
##########
@@ -138,16 +139,36 @@ public static long computeOperatorMemory(OptionSet
optionManager, long maxAllocP
@VisibleForTesting
public static long computeQueryMemory(DrillConfig config, OptionSet
optionManager, long directMemory) {
+ // Get the options
+ double percent_per_query =
optionManager.getOption(ExecConstants.PERCENT_MEMORY_PER_QUERY);
+ long max_query_per_node =
optionManager.getOption(ExecConstants.MAX_QUERY_MEMORY_PER_NODE);
+ double percent_allowance =
optionManager.getOption(ExecConstants.PERCENT_RESERVED_ALLOWANCE_FROM_DIRECT);
+
+ // verify that the allowance is kept
+ if ( percent_per_query + percent_allowance > 1.0 ) {
Review comment:
Why do we need to make sure these add to one? Couldn't we just reserve our
allowance from the previously computed maxAllocPerNode value. For example
```
double percentAllowance =
optionManager.getOption(ExecConstants.PERCENT_RESERVED_ALLOWANCE_FROM_DIRECT);
// Memory computed as a percent of total memory.
long perQueryMemory = Math.round(directMemory *
optionManager.getOption(ExecConstants.PERCENT_MEMORY_PER_QUERY));
// But, must allow at least the amount given explicitly for
// backward compatibility.
perQueryMemory = Math.max(perQueryMemory,
optionManager.getOption(ExecConstants.MAX_QUERY_MEMORY_PER_NODE));
// Compute again as either the total direct memory, or the
// configured maximum top-level allocation (10 GB).
long maxAllocPerNode = Math.min(directMemory,
config.getLong(RootAllocatorFactory.TOP_LEVEL_MAX_ALLOC));
// Final amount per node per query is the minimum of these two.
maxAllocPerNode = Math.min(maxAllocPerNode, perQueryMemory);
// Deduct non buffered allowance
return Math.round(maxAllocPerNode * (1.0 - percentAllowance));
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
