[
https://issues.apache.org/jira/browse/DRILL-7147?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16807259#comment-16807259
]
Paul Rogers commented on DRILL-7147:
------------------------------------
This is an easy misunderstanding. If we imagine that files contain lines of the
form:
{noformat}
drill-env.sh:
export FOO="ABC"
distrib-env.sh:
export FOO="DEF"
{noformat}
If this is how things worked, then the change suggested here would be correct.
However, this is *not* how the variables are supposed to be set. In order to
create the hierarchy explained in the comments, the correct form, explained in
{{drill-env.sh}} is:
{noformat}
drill-env.sh:
export FOO=${FOO:-"ABC"}
distrib-env.sh:
export FOO=${FOO:-"DEF"}
{noformat}
In this new form, we want to source the top-most levels of the hierarchy before
the bottom-most levels. Consider:
* If {{FOO}} is set in the environment, it must take precedence, which it can
do only using the form explained above. (There is no way to, say, source
{{distrib-env.sh}} before the environment.)
* If {{FOO}} is set in drill-env.sh (but not in the environment), then it must
take precedence, which it does using the above form.
* And so on.
All this said, it is likely that there is some particular variable that
triggered this issue. Suggestion: track that variable down. Did it not follow
the pattern above? For example, it appears that there are some entries in
{{drill-env.sh}} for which the comments are wrong:
{code:bash}
# Maximum amount of direct memory to allocate to the Drillbit in the format
# supported by -XX:MaxDirectMemorySize. Default is 8G.
#export DRILL_MAX_DIRECT_MEMORY=${DRILL_MAX_DIRECT_MEMORY:-"8G"}
# Native library path passed to Java. Note: use this form instead
# of the old form of DRILLBIT_JAVA_OPTS="-Djava.library.path=<dir>"
# The old form is not compatible with Drill-on-YARN.
# export DRILL_JAVA_LIB_PATH="<lib1>:<lib2>"
# Value for the code cache size for the Drillbit. Because the Drillbit generates
# code, it benefits from a large cache. Default is 1G.
#export DRILLBIT_CODE_CACHE_SIZE=${DRILLBIT_CODE_CACHE_SIZE:-"1G"}
# Provide a customized host name for when the default mechanism is not accurate
#export DRILL_HOST_NAME=`hostname`
# Base name for Drill log files. Files are named ${DRILL_LOG_NAME}.out, etc.
# DRILL_LOG_NAME="drillbit"
{code}
In the above, {{DRILL_MAX_DIRECT_MEMORY}} is correct, {{DRILL_JAVA_LIB_PATH}},
{{DRILL_HOST_NAME}} and {{DRILL_LOG_NAME}} are wrong. They should be:
{code:bash}
# Native library path passed to Java. Note: use this form instead
# of the old form of DRILLBIT_JAVA_OPTS="-Djava.library.path=<dir>"
# The old form is not compatible with Drill-on-YARN.
# export
DRILL_JAVA_LIB_PATH="$DRILL_JAVA_LIB_PATH${DRILL_JAVA_LIB_PATH:=;}<lib1>:<lib2>"
# Provide a customized host name for when the default mechanism is not accurate
#export DRILL_HOST_NAME=${DRILL_HOST_NAME:-`hostname`}
# Base name for Drill log files. Files are named ${DRILL_LOG_NAME}.out, etc.
# DRILL_LOG_NAME=${DRILL_LOG_NAME:-"drillbit"}
{code}
> Source order of "drill-env.sh" and "distrib-env.sh" should be swapped
> ---------------------------------------------------------------------
>
> Key: DRILL-7147
> URL: https://issues.apache.org/jira/browse/DRILL-7147
> Project: Apache Drill
> Issue Type: Bug
> Components: Execution - Flow
> Affects Versions: 1.15.0
> Reporter: Hao Zhu
> Assignee: Abhishek Girish
> Priority: Minor
> Fix For: 1.16.0
>
>
> In bin/drill-config.sh, the description of the source order is:
> {code:java}
> # Variables may be set in one of four places:
> #
> # Environment (per run)
> # drill-env.sh (per site)
> # distrib-env.sh (per distribution)
> # drill-config.sh (this file, Drill defaults)
> #
> # Properties "inherit" from items lower on the list, and may be "overridden"
> by items
> # higher on the list. In the environment, just set the variable:
> {code}
> However actually bin/drill-config.sh sources drill-env.sh firstly, and then
> distrib-env.sh.
> {code:java}
> drillEnv="$DRILL_CONF_DIR/drill-env.sh"
> if [ -r "$drillEnv" ]; then
> . "$drillEnv"
> fi
> ...
> distribEnv="$DRILL_CONF_DIR/distrib-env.sh"
> if [ -r "$distribEnv" ]; then
> . "$distribEnv"
> else
> distribEnv="$DRILL_HOME/conf/distrib-env.sh"
> if [ -r "$distribEnv" ]; then
> . "$distribEnv"
> fi
> fi
> {code}
> We need to swap the source order of drill-env.sh and distrib-env.sh.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
