[
https://issues.apache.org/jira/browse/DRILL-5547?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16083196#comment-16083196
]
ASF GitHub Bot commented on DRILL-5547:
---------------------------------------
Github user paul-rogers commented on a diff in the pull request:
https://github.com/apache/drill/pull/868#discussion_r126825276
--- Diff:
exec/java-exec/src/main/java/org/apache/drill/exec/planner/fragment/SimpleParallelizer.java
---
@@ -72,7 +72,16 @@ public SimpleParallelizer(QueryContext context) {
OptionManager optionManager = context.getOptions();
long sliceTarget =
optionManager.getOption(ExecConstants.SLICE_TARGET).num_val;
this.parallelizationThreshold = sliceTarget > 0 ? sliceTarget : 1;
- this.maxWidthPerNode =
optionManager.getOption(ExecConstants.MAX_WIDTH_PER_NODE_KEY).num_val.intValue();
+ // If the max_width is not set in the validator compute from the cpu
load average
+ long maxWidth =
optionManager.getOption(ExecConstants.MAX_WIDTH_PER_NODE);
+ if(maxWidth == 0) {
+ Double cpu_load_average =
optionManager.getOption(ExecConstants.CPU_LOAD_AVERAGE);
+ this.maxWidthPerNode = (int)
Math.ceil(Runtime.getRuntime().availableProcessors() * cpu_load_average);
--- End diff --
Probably should:
* round result, not take the ceiling
* clamp the number at 1 and the number of processors
```
Math.max(1, Math.min( availProc, Math.round(... * ...) ) )
```
We didn't do this before, but we should have...
This is true if the value was set by a property, so the clamping should be
done regardless of whether the value is retrieved or computed. (Gracefully
handle the case where someone sets the value to 0 or 1 million, say.)
> Drill config options and session options do not work as intended
> ----------------------------------------------------------------
>
> Key: DRILL-5547
> URL: https://issues.apache.org/jira/browse/DRILL-5547
> Project: Apache Drill
> Issue Type: Bug
> Components: Server
> Affects Versions: 1.10.0
> Reporter: Karthikeyan Manivannan
> Assignee: Venkata Jyothsna Donapati
> Fix For: Future
>
>
> In Drill, session options should take precedence over config options. But
> several of these session options are assigned hard-coded default values when
> the option validators are initialized. Because of this config options will
> never be read and honored even if the user did not specify the session
> option.
> ClassCompilerSelector.JAVA_COMPILER_VALIDATOR uses CompilerPolicy.DEFAULT as
> the default value. This default value gets into the session options map via
> the initialization of validators in SystemOptionManager.
> Now any piece of code that tries to check if a session option is set will
> never see a null, so it will always use that value and never try to look into
> the config options. For example, in the following piece of code from
> ClassCompilerSelector (), the policy will never be read from the config file.
> policy = CompilerPolicy.valueOf((value != null) ?
> value.string_val.toUpperCase() :
> config.getString(JAVA_COMPILER_CONFIG).toUpperCase());
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
