[
https://issues.apache.org/jira/browse/DRILL-4587?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16405591#comment-16405591
]
Paul Rogers commented on DRILL-4587:
------------------------------------
For item 1: there is no overlap between {{drill-env.sh}} (bootstrap) and
{{drill-override.conf}} (startup) properties. Properties are in
{{drill-override.conf}} unless they must be set before Java starts.
Better to say we have three systems:
1. {{drill-env.sh}} and environment.
2. {{drill-override.conf}}, and -Dprop=value
3. System/session properties set in SQL.
There is some confusion in the original description around systems 2 and 3. In
general, a property is settable ONLY in the startup property system OR the
system/session properties set in SQL.
Now, it is true that the defaults for system/session properties exist in our
internal startup {{drill-module.conf}} system. But, this is ONLY for our own
use. Users should NEVER set properties this way. Instead, use {{ALTER SYSTEM}}
so that the property is set in ZK and propagated to all Drillbits.
As a result, the three systems are completely independent.
To revise: the first item 3 ({{$DRILL_HOME/conf/drill-env.sh}}), the user can
(should!) also use a site directory:
{code:java}
drillbit.sh --site $DRILL_SITE
{code}
Where the config files are in the separate site directory. [Drill-on-YARN
explains the site
directory|https://github.com/apache/drill/blob/master/drill-yarn/USAGE.md].
To document the variables, see {{drill-env.sh}} which has explanations for all
supported variables (or at least those as of 1.8. If others were added later,
then comments should be added.)
The following is technically correct, but represents unsupported (legacy) usage:
bq. Drill startup propeties can be set in a number of locations. Those listed
later take precedence over those listed earlier.
bq. 1. Drill-override.conf as identified by DRILL_CONF_DIR or its default.
Actually, {{drill-override.conf}} as determined by {{$DRILL_CONF_DIR}}, the
{{--site site/dir}} option, or the default location of {{$DRILL_HOME/conf}}.
But, note that the properties here are *distinct* from those described below,
so they do not overlap. See note above. Grouping them together actually adds to
confusion.
bq. 2. Set in the environment using DRILL_JAVA_OPTS or DRILL_DRILLBIT_JAVA_OPTS.
Not supported. Use these options ONLY for additional JVM options.
{{drill-config.sh}} does some fancy processing of options that cannot be done
if the options are hidden in one of the above.
bq. 3. Set in drill-env.sh using the above two variables.
Preferred solution.
bq. 4. Set on the drillbit.sh command line as explained above. (Drill 1.7 and
later.)
See the confusion? The command line only works for the values in item 1 above,
not for those in 3.
5. As environment variables.
This is the ultimate override for items from 4. DoY uses this to pass values
from the DoY config file, though YARN, into the YARN Node Manager and on into
Drill.
Suggestion: reword the description (or submit a revised note) that separates
the three systems, then documents the alternatives for each.
> Document Drillbit launch options
> --------------------------------
>
> Key: DRILL-4587
> URL: https://issues.apache.org/jira/browse/DRILL-4587
> Project: Apache Drill
> Issue Type: Improvement
> Components: Documentation
> Reporter: Paul Rogers
> Assignee: Bridget Bevens
> Priority: Major
> Fix For: 1.14.0
>
>
> Drill provides the drillbit.sh script to launch Drill. When Drill is run in
> production environments, or when managed by a tool such as Mesos or YARN,
> customers have many options to customize the launch options. We should
> document this information as below.
> The user can configure Drill launch in one of four ways, depending on their
> needs.
> 1. Using the properties in drill-override.conf. Sets only startup and runtime
> properties. All drillbits should use a copy of the file so that properties
> set here apply to all drill bits and to client applications.
> 2. By setting environment variables prior to launching Drill. See the list
> below. Use this to customize properties per drill-bit, such as for setting
> port numbers. This option is useful when launching Drill from a tool such as
> Mesos or YARN.
> 3. By setting environment variables in $DRILL_HOME/conf/drill-env.sh. See the
> list below. This script is intended to be unique to each node and is another
> way to customize properties for this one node.
> 4. In Drill 1.7 and later, the administrator can set Drill configuration
> options directly on the launch command as shown below. This option is also
> useful when launching Drill from a tool such as YARN or Mesos. Options are of
> the form:
> $ drillbit.sh start -Dvariable=value
> For example, to control the HTTP port:
> $ drillbit.sh start -Ddrill.exec.http.port=8099
> Properties are of three types.
> 1. Launch-only properties: those that can be set only through environment
> variables (such as JAVA_HOME.)
> 2. Drill startup properties which can be set in the locations detailed below.
> 3. Drill runtime properties which are set in drill-override.conf also via SQL.
> Drill startup propeties can be set in a number of locations. Those listed
> later take precedence over those listed earlier.
> 1. Drill-override.conf as identified by DRILL_CONF_DIR or its default.
> 2. Set in the environment using DRILL_JAVA_OPTS or DRILL_DRILLBIT_JAVA_OPTS.
> 3. Set in drill-env.sh using the above two variables.
> 4. Set on the drill.bit command line as explained above. (Drill 1.7 and
> later.)
> You can see the actual set of properties used (from items 2-3 above) by using
> the "debug" command (Drill 1.7 or later):
> $ drillbit.sh debug
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
