Paul Rogers created DRILL-6521:
----------------------------------
Summary: When using Hadoop configs, allow dfs connection to be
unset
Key: DRILL-6521
URL: https://issues.apache.org/jira/browse/DRILL-6521
Project: Apache Drill
Issue Type: Improvement
Affects Versions: 1.13.0
Reporter: Paul Rogers
Drill usually works stand-alone. In a Hadoop HDFS environment, the docs. say to
set the HDFS configuration in the storage plugin config.
In a MapR installation, MaprFS settings are applied automatically.
However, when Drill runs on an existing HDFS cluster, one must often provide
more than the simple HDFS URL. Particularly in a secure cluster, other
configuration settings are also needed. At present, these must be copied out of
the HDFS config files into the Drill storage plugin config, and the two must be
updated in tandem. Clearly less than ideal.
Drill does allow the user to add Hadoop configs to the class path. (Though, it
looks like in recent releases the previous {{HADOOP_CONF}} setting has been
removed.) The user can edit {{drill-env.sh}} to add the Hadoop class path to
{{EXTN_CLASSPATH}}. (But see DRILL-6520.)
This is all good, but Drill still requires that the "dfs" storage plugin config
contain a connection. Omit the connection and we get:
{noformat}
Please retry: Error while creating/ updating storage : The value of property
fs.defaultFS must not be null
{noformat}
Would expect to be able to omit this property if the value is provided by the
standard Hadoop {{core-site.xml}} file (and to do so without Drill crashing,
per DRILL-6520).
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
