Ben, NiFi doesn't currently have components that support Hive 2, although some work has been done on that front [1]. However there are Hive 3 components including a PutORC processor that is kind of a mashup between the old Hive 1-based ConvertAvroToORC and PutHDFS/PutParquet, it does the conversion from any format (using a supplied Reader) and writes it out to the specified location. The choice was made to have the conversion and writing done at the same time as you can't do much with ORC in NiFi (transformations, e.g.).
PutORC is part of the Hive 3 NAR, which due to size constraints is not included with the Apache NiFi distribution. However we do build and publish the NAR for each release, so you can download it separately and drop it into your lib/ folder. You can find the Hive 3 NAR for the latest release at [2]. If you have a use case for which PutORC is not sufficient, please explain and/or vote for the Jira case that covers adding ORC reader/writer services [3]. Regards, Matt [1] https://issues.apache.org/jira/browse/NIFI-6456 [2] https://repository.apache.org/content/repositories/releases/org/apache/nifi/nifi-hive3-nar/1.11.4/ [3] https://issues.apache.org/jira/browse/NIFI-4186 On Thu, Mar 26, 2020 at 12:23 PM Benjamin Garrett <[email protected]> wrote: > > Hi, > > Regarding ConvertAvroToORC, I found this link which describes > incompatibilities with later version of hive. > https://community.cloudera.com/t5/Support-Questions/hive-2-3-return-NULLS-using-NIFI-convertAvroToORC/td-p/175279 > > Are there plans to support newer versions of hive with orc output from Nifi? > > I couldn't find any jira ticket related to this topic. I did find this open > ticket regarding the RecordPath/RecordReader pattern and orc: > https://issues.apache.org/jira/browse/NIFI-4186 > > Thank you! > Ben
