Github user bbende commented on the issue:
https://github.com/apache/nifi/pull/1075
Tried this out on against an HDP 2.4 sandbox and got an exception that
boiled down to:
```
Caused by: java.io.IOException: No FileSystem for scheme: hdfs
at
org.apache.hadoop.fs.FileSystem.getFileSystemClass(FileSystem.java:2584)
~[hadoop-common-2.6.2.jar:na]
at
org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:2591)
~[hadoop-common-2.6.2.jar:na]
at org.apache.hadoop.fs.FileSystem.access$200(FileSystem.java:91)
~[hadoop-common-2.6.2.jar:na]
at
org.apache.hadoop.fs.FileSystem$Cache.getInternal(FileSystem.java:2630)
~[hadoop-common-2.6.2.jar:na]
at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:2612)
~[hadoop-common-2.6.2.jar:na]
at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:370)
~[hadoop-common-2.6.2.jar:na]
at org.apache.hadoop.fs.Path.getFileSystem(Path.java:296)
~[hadoop-common-2.6.2.jar:na]
at
org.apache.hadoop.hive.ql.io.orc.OrcRecordUpdater.<init>(OrcRecordUpdater.java:221)
~[hive-exec-1.2.1.jar:1.2.1]
at
org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat.getRecordUpdater(OrcOutputFormat.java:292)
~[hive-exec-1.2.1.jar:1.2.1]
at
org.apache.hive.hcatalog.streaming.AbstractRecordWriter.createRecordUpdater(AbstractRecordWriter.java:141)
~[hive-hcatalog-streaming-1.2.1.jar:1.2.1]
at
org.apache.hive.hcatalog.streaming.AbstractRecordWriter.newBatch(AbstractRecordWriter.java:121)
~[hive-hcatalog-streaming-1.2.1.jar:1.2.1]
... 10 common frames omitted
```
Comparing the JARs that get included with Hadoop Libraries NAR vs the Hive
NAR there appear to be some differences.
Hadoop Libraries NAR:
hadoop-annotations-2.6.2.jar
hadoop-auth-2.6.2.jar
hadoop-client-2.6.2.jar
hadoop-common-2.6.2.jar
hadoop-hdfs-2.6.2.jar
hadoop-mapreduce-client-app-2.6.2.jar
hadoop-mapreduce-client-common-2.6.2.jar
hadoop-mapreduce-client-core-2.6.2.jar
hadoop-mapreduce-client-jobclient-2.6.2.jar
hadoop-mapreduce-client-shuffle-2.6.2.jar
hadoop-yarn-api-2.6.2.jar
hadoop-yarn-client-2.6.2.jar
hadoop-yarn-common-2.6.2.jar
hadoop-yarn-server-common-2.6.2.jar
Hive NAR:
hadoop-annotations-2.6.2.jar
hadoop-auth-2.6.2.jar
hadoop-common-2.6.2.jar
hadoop-mapreduce-client-core-2.6.2.jar
hadoop-yarn-api-2.6.2.jar
hadoop-yarn-common-2.6.2.jar
hadoop-yarn-server-applicationhistoryservice-2.6.0.jar
hadoop-yarn-server-common-2.6.0.jar
hadoop-yarn-server-resourcemanager-2.6.0.jar
hadoop-yarn-server-web-proxy-2.6.0.jar
I think the Hive NAR at least needs the hadoop-hdfs jar, but not sure what
else.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
