Paul Rogers created DRILL-4591:
----------------------------------
Summary: Extend config system with site, node property files
Key: DRILL-4591
URL: https://issues.apache.org/jira/browse/DRILL-4591
Project: Apache Drill
Issue Type: Improvement
Reporter: Paul Rogers
Priority: Minor
Today Drill provides the drill-override.conf file to set Drill properties, and
the drill-env.sh file to provide custom launch properties. Today, most users
seem to have a copy of DRILL_HOME per node, and thus they copy these two files
per-node.
The result is that the two files act as both the overall "site" configuration
(for all nodes) and the "per-node" configuration for that one node.
The approach works, but is awkward. Ideally, provide the option to have two
files: one for the site, one for the node.
drill-override.conf, site-env.sh: For the site
drill-node.conf, drill-env.sh: For the particular node
The improvement becomes more important if a user employs NFS, MapR FS or YARN
to automatically deploy the site-wide files. In that case, the site files
cannot also act as per-node files.
Note that, above, site-env.sh is new. The existing file is supposed to be
per-node, so the new file is site-wide.
By contrast, the drill-node.conf per-node configuration is new as the existing
file is supposed to be site-wide.
The site-wide site-env.sh might, say, point to custom locations for log, conf,
pid or other file locations, or set custom JVM options.
Similarly, the per-node drill-node.conf might customize the per-node memory or
other resources if one node has a different amount of physical memory or other
resources.
The existing files are identified with the DRILL_CONF_DIR environment variable.
Add a new DRILL_SITE_CONF_DIR variable to point to the site-wide files.
When loading, per-node files are loaded after the site-wide files (so that
per-node values override site-wide values.)
As described, the change should be fully backward compatible. The change will
also work with a YARN solution that "localizes" (copies to the local node) the
site-wide files.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
