Paul Rogers created DRILL-4893:
----------------------------------
Summary: Define in Drill-override.conf a common DFS root directory
Key: DRILL-4893
URL: https://issues.apache.org/jira/browse/DRILL-4893
Project: Apache Drill
Issue Type: Improvement
Reporter: Paul Rogers
Priority: Minor
Drill-on-YARN and Dynamic UDFs are two Drill features that write files to the
distributed file system (DFS) such as HDFS. These files are configuration-style
files (Drill software for DoY, UDF jars for Dynamic UDFs.) Likely other Drill
features will also use DFS for storage, perhaps for logs, metadata, etc.
At present, each feature defines its own DFS storage path. An administrator who
wants to define a DFS location must know of, and modify, each of these
locations.
The proposal here is to define a single Drill root directory property. Then,
define all other DFS locations relative to the root. This allows the admin to
relocate all Drill files by changing a single property. (Of course, this change
should be made at install time, not after Drill has written files, unless the
files themselves are moved.)
For example (using made-up property names):
{code}
drill.dfs: “/user/drill”
drill.yarn.dfs: “${drill.dfs}/doy”
drill.exec.udf.directory.base: “${drill.dfs}/udf”
drill.exec.udf.directory.staging: “${drill.exec.udf.directory.base}/staging”
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
