Hi Peter/Andre,
I cherrypicked the commit and added the dependency on:
<dependency>
<groupId>org.apache.orc</groupId>
<artifactId>orc-core</artifactId>
<version>1.1.0</version>
</dependency>
(tried also with version 1.2.0) but stil no luck, as I keep on getting
compilation errors:
[ERROR]
/nifi/nifi-nar-bundles/nifi-hive-bundle/nifi-hive-processors/src/main/java/org/apache/hadoop/hive/ql/io/orc/OrcFlowFileWriter.java:[200,20]
error: cannot find symbol
[ERROR] symbol: variable OrcUtils
[ERROR] location: class OrcFlowFileWriter
[ERROR]
/nifi/nifi-nar-bundles/nifi-hive-bundle/nifi-hive-processors/src/main/java/org/apache/hadoop/hive/ql/io/orc/OrcFlowFileWriter.java:[203,20]
error: cannot find symbol
[ERROR] symbol: variable OrcUtils
[ERROR] location: class OrcFlowFileWriter
[ERROR]
/nifi/nifi-nar-bundles/nifi-hive-bundle/nifi-hive-processors/src/main/java/org/apache/hadoop/hive/ql/io/orc/OrcFlowFileWriter.java:[313,45]
error: cannot find symbol
[ERROR] symbol: method loadClass(String)
[ERROR] location: class JavaUtils
[ERROR]
/nifi/nifi-nar-bundles/nifi-hive-bundle/nifi-hive-processors/src/main/java/org/apache/hadoop/hive/ql/io/orc/OrcFlowFileWriter.java:[494,21]
error: an enum switch case label must be the unqualified name of an
enumeration constant
[ERROR]
/nifi/nifi-nar-bundles/nifi-hive-bundle/nifi-hive-processors/src/main/java/org/apache/hadoop/hive/ql/io/orc/OrcFlowFileWriter.java:[689,43]
error: cannot find symbol
[ERROR] symbol: variable BloomFilter
[ERROR] location: class OrcProto
[ERROR]
/nifi/nifi-nar-bundles/nifi-hive-bundle/nifi-hive-processors/src/main/java/org/apache/hadoop/hive/ql/io/orc/OrcFlowFileWriter.java:[690,43]
error: cannot find symbol
[ERROR] symbol: variable BloomFilterIndex
[ERROR] location: class OrcProto
[ERROR]
/nifi/nifi-nar-bundles/nifi-hive-bundle/nifi-hive-processors/src/main/java/org/apache/hadoop/hive/ql/io/orc/OrcFlowFileWriter.java:[691,87]
error: cannot find symbol
[ERROR] symbol: variable BLOOM_FILTER
[ERROR] location: class Kind
[ERROR]
/nifi/nifi-nar-bundles/nifi-hive-bundle/nifi-hive-processors/src/main/java/org/apache/hadoop/hive/ql/io/orc/OrcFlowFileWriter.java:[692,34]
error: cannot find symbol
[ERROR] symbol: class BloomFilterIO
[ERROR] location: class TreeWriter
[ERROR]
/nifi/nifi-nar-bundles/nifi-hive-bundle/nifi-hive-processors/src/main/java/org/apache/hadoop/hive/ql/io/orc/OrcFlowFileWriter.java:[726,52]
error: incompatible types: PositionedOutputStream cannot be converted to
OutStream
[ERROR]
/nifi/nifi-nar-bundles/nifi-hive-bundle/nifi-hive-processors/src/main/java/org/apache/hadoop/hive/ql/io/orc/OrcFlowFileWriter.java:[728,50]
error: incompatible types: PositionedOutputStream cannot be converted to
OutStream
[ERROR]
/nifi/nifi-nar-bundles/nifi-hive-bundle/nifi-hive-processors/src/main/java/org/apache/hadoop/hive/ql/io/orc/OrcFlowFileWriter.java:[804,19]
error: cannot find symbol
[ERROR] symbol: method setWriterTimezone(String)
[ERROR] location: variable builder of type Builder
[ERROR]
/nifi/nifi-nar-bundles/nifi-hive-bundle/nifi-hive-processors/src/main/java/org/apache/hadoop/hive/ql/io/orc/NiFiOrcUtils.java:[450,32]
error: cannot find symbol
do you have any idea?
Thanks,
Yari
On 8 September 2016 at 20:42, Peter Wicks (pwicks) <[email protected]>
wrote:
> Also, ORC File support was pulled out into its own library on the HIVE
> side.
>
> If you are willing to compile and run your own version you might need to
> include orc-core as a MVN dependency: https://mvnrepository.com/
> artifact/org.apache.orc/orc-core/1.2.0.
>
>
>
>
>
> *From:* Andre [mailto:[email protected]]
> *Sent:* Thursday, September 08, 2016 4:51 AM
> *To:* [email protected]
> *Subject:* Re: Nifi 1.0.0 compatibility with Hive 1.1.0
>
>
>
> Yari,
>
>
>
> Is there any chance you can cherry pick commit
> 80224e3e5ed7ee7b09c4985a920a7fa393bff26c and try again?
>
>
>
> Post 1.0.0 there have been some changes to streamline compilation using
> vendor provided libraries.
>
>
>
> Cheers
>
>
>
> On Thu, Sep 8, 2016 at 8:44 PM, Yari Marchetti <
> [email protected]> wrote:
>
> Hello,
>
> I'd like to use Nifi 1.0.0 with Hive 1.1.0 (on CDH 5.5.2) but after some
> investigation I realised that the hive-jdbc driver included in Nifi is
> incompatible with the Hive version we're using (1.1.0 on CDH 5.5.2) as I'm
> getting the error:
>
>
>
> org.apache.hive.jdbc.HiveConnection Error opening session
>
> org.apache.thrift.TApplicationException: Required field 'client_protocol'
> is unset! Struct:TOpenSessionReq(client_protocol:null,
> configuration:{use:database=unifieddata})
>
>
>
> So I just tried to recompile Nifi using the Cloudera profile 5.5.2 but
> compilation is failing:
>
>
>
> [ERROR] Failed to execute goal org.apache.maven.plugins:
> maven-compiler-plugin:3.2:compile (default-compile) on project
> nifi-hive-processors: Compilation failure: Compilation failure:
>
> [ERROR] /home/matteo/git/nifi/nifi-nar-bundles/nifi-hive-bundle/
> nifi-hive-processors/src/main/java/org/apache/hadoop/hive/
> ql/io/orc/NiFiOrcUtils.java:[26,43] error: package
> org.apache.hadoop.hive.ql.io.filters does not exist
>
> [ERROR] /home/matteo/git/nifi/nifi-nar-bundles/nifi-hive-bundle/
> nifi-hive-processors/src/main/java/org/apache/hadoop/hive/
> ql/io/orc/OrcFlowFileWriter.java:[45,43] error: package
> org.apache.hadoop.hive.ql.io.filters does not exist
>
> [ERROR] /home/matteo/git/nifi/nifi-nar-bundles/nifi-hive-bundle/
> nifi-hive-processors/src/main/java/org/apache/hadoop/hive/
> ql/io/orc/OrcFlowFileWriter.java:[643,24] error: cannot find symbol
>
> [ERROR] symbol: class BloomFilterIO
>
> [ERROR] location: class TreeWriter
>
> [ERROR] /home/matteo/git/nifi/nifi-nar-bundles/nifi-hive-bundle/
> nifi-hive-processors/src/main/java/org/apache/hadoop/hive/
> ql/io/orc/OrcFlowFileWriter.java:[645,30] error: cannot find symbol
>
> [ERROR] symbol: class BloomFilterIndex
>
> [ERROR] location: class OrcProto
>
> [ERROR] /home/matteo/git/nifi/nifi-nar-bundles/nifi-hive-bundle/
> nifi-hive-processors/src/main/java/org/apache/hadoop/hive/
> ql/io/orc/OrcFlowFileWriter.java:[646,30] error: cannot find symbol
>
> [ERROR] symbol: class BloomFilter
>
> [ERROR] location: class OrcProto
>
> [ERROR] /home/matteo/git/nifi/nifi-nar-bundles/nifi-hive-bundle/
> nifi-hive-processors/src/main/java/org/apache/hadoop/hive/
> ql/io/orc/NiFiOrcUtils.java:[450,32] error: cannot find symbol
>
> [ERROR] symbol: variable BloomFilterIO
>
> [ERROR] location: class NiFiOrcUtils
>
> [ERROR] /home/matteo/git/nifi/nifi-nar-bundles/nifi-hive-bundle/
> nifi-hive-processors/src/main/java/org/apache/hadoop/hive/
> ql/io/orc/OrcFlowFileWriter.java:[200,20] error: cannot find symbol
>
> [ERROR] symbol: variable OrcUtils
>
> [ERROR] location: class OrcFlowFileWriter
>
>
>
>
>
> Is there any way to get Nifi to work with Hive 1.1.0 and CDH 5.5.2?
>
>
>
> Thanks,
>
> Yari
>
>
>
