Paul Rogers created DRILL-6520:
----------------------------------
Summary: Setting fs.defaultFS in core-site.xml causes startup crash
Key: DRILL-6520
URL: https://issues.apache.org/jira/browse/DRILL-6520
Project: Apache Drill
Issue Type: Bug
Affects Versions: 1.13.0
Reporter: Paul Rogers
In a Hadoop HDFS environment, the docs. say to set the HDFS configuration in
the storage plugin config.
However, when Drill runs on an existing HDFS cluster, one must often provide
more than the simple HDFS URL. Particularly in a secure cluster, other
configuration settings are also needed. At present, these must be copied out of
the HDFS config files into the Drill storage plugin config, and the two must be
updated in tandem. Clearly less than ideal.
In experimenting with better integration, I added my HDFS {{core-site.xml}}
file to the class path, hoping Drill would pick up the settings from the Hadoop
config files that I had placed on the class path. Instead, Drill failed to
start.
{noformat}
Exception in thread "main" java.lang.NullPointerException
at
org.apache.drill.exec.coord.zk.ZKClusterCoordinator.update(ZKClusterCoordinator.java:218)
at org.apache.drill.exec.server.Drillbit.close(Drillbit.java:228)
at org.apache.drill.exec.server.Drillbit.start(Drillbit.java:401)
at org.apache.drill.exec.server.Drillbit.start(Drillbit.java:372)
at org.apache.drill.exec.server.Drillbit.main(Drillbit.java:368)
{noformat}
Modify the {{core-site.xml}} file to comment out the {{fs.defaultFS}} property.
{noformat}
<configuration>
<!--
<property>
<name>fs.defaultFS</name>
<value>hdfs://myNameNode:8020</value>
</property>
-->
</configuration>
{noformat}
Now Drill starts fine.
The "myNameNode" is not the real name. But, the actual value here is identical
to the value set in the dfs "connection" property, which works fine by itself
without {{core-site.xml}}.
Expected that Drill would allow the common Hadoop pattern of setting
{{fs.defaultFS}} in {{core-site.xml}}. At the very least, Drill should not
crash when this (or any other) property is set in {{core-site.xml}}.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
