arvindshmicrosoft commented on pull request #1101:
URL: https://github.com/apache/fluo/pull/1101#issuecomment-656386328
For context, the original issue encountered (and which is fixed by this PR)
is that when Fluo is configured to use a highly-available (HA) HDFS namespace
as the DFS root, fluo-env.sh does not add the Hadoop conf folder to the
classpath. This leads to a error as shown below. In this case,
`muchoshacluster` was the HA HDFS namespace configured.
```
2020-07-09 22:30:39,322 [client.FluoAdminImpl] ERROR: Failed to create DFS
directory hdfs://muchoshacluster/fluo/webindex/lib/accumulo
2020-07-09 22:30:39,425 [zookeeper.ZooCache] WARN : Unhandled: WatchedEvent
state:Closed type:None path:null
Exception in thread "main" java.lang.IllegalStateException:
java.lang.IllegalArgumentException: java.net.UnknownHostException:
muchoshacluster
at
org.apache.fluo.core.client.FluoAdminImpl.copyJarsToDfs(FluoAdminImpl.java:393)
at
org.apache.fluo.core.client.FluoAdminImpl.initialize(FluoAdminImpl.java:178)
...
Caused by: java.lang.IllegalArgumentException:
java.net.UnknownHostException: muchoshacluster
at
org.apache.hadoop.security.SecurityUtil.buildTokenService(SecurityUtil.java:447)
at
org.apache.hadoop.hdfs.NameNodeProxiesClient.createProxyWithClientProtocol(NameNodeProxiesClient.java:139)
...
at
org.apache.fluo.core.client.FluoAdminImpl.copyJarsToDfs(FluoAdminImpl.java:382)
... 5 more
Caused by: java.net.UnknownHostException: muchoshacluster
```
Regular Hadoop commands worked fine on that node because Hadoop sets up its
classpath to correctly load hdfs-site.xml from the conf folder.
Also, the Hadoop native library is not on the system library load path,
leading to the below warning:
```
2020-07-09 22:30:39,076 [util.NativeCodeLoader] WARN : Unable to load
native-hadoop library for your platform... using builtin-java classes where
applicable
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
