Mike, For your Maven command in #1, those hive.* properties are for the Hive 1 NAR (PutHiveStreaming, e.g.), you can override the hive3.* versions of them if you want to set a particular version (they default to Apache Hive 3.0.0).
For #2, I didn't see anything in the new Hive Streaming API that accepts just username and password, rather it tries to get the information from the UserGroupInformation which I believe needs Kerberos. If Kerberos is not enabled, I believe it uses the "user.name" system property for username, but I still don't see where authorization is happening (even behind the scenes). I assume it has to do with being authenticated and passing the username to Ranger, but I'm not sure. Having said that, there may be some properties you could set in hive-site.xml to specify a username/password. I'll ask around and follow up if I find out anything. Regards, Matt On Mon, Jul 30, 2018 at 10:40 AM Michal Tomaszewski <michal.tomaszew...@cca.pl> wrote: > > Hi Matt, > > #1 > Thank you very much!!! > PutHive3Streaming works after compilation :) > > BTW: compilation with options > mvn -T C2.0 clean install -Phortonworks -Dhive.version=3.1.0.3.0.0.0-1634 > -Dhive.hadoop.version=3.1.0.3.0.0.0-1634 -Dhadoop.version=3.1.0.3.0.0.0-1634 > -Pinclude-hive3 -DskipTests -e > throws errors like > [ERROR] symbol: class MemoryManager > [ERROR] location: class NiFiOrcUtils > [ERROR] > /home/compiler/nifi/nifi-master/nifi/nifi-nar-bundles/nifi-hive-bundle/nifi-hive-processors/src/main/java/org/apache/hadoop/hive/ql/io/orc/NiFiOrcUtils.java:[478,32] > error: cannot find symbol > > so I compiled using: > mvn -T C2.0 clean install -Phortonworks -Pinclude-hive3 -DskipTests -e > I hope its proper command. > > #2 > Please advice how to make PutHive3Streaming use specified username and > password for authorization when contacting with hive? > Currently all requests are made as anonymous. > I have ranger for authorization, but cluster is not kerberised. > > In HiveConnectionPool (e.g. for HiveQL) there is a place for username and > password for query that works as required. In PutHive3Streaming there is no > place to put credentials. > > Regards, > Mike > > > -----Original Message----- > > From: Matt Burgess <mattyb...@apache.org> > > Sent: Monday, July 30, 2018 2:45 PM > > To: users@nifi.apache.org > > Subject: Re: PutHiveStreaming + Hive 3.0 > > > > Mike, > > > > That error usually indicates a Thrift version mismatch, which in this case > > is > > pretty much expected since PutHiveStreaming uses version 1.2.1 and HDP 3.0 > > uses 3.0.0+. As of NiFi 1.7.0 you can add the "-Pinclude-hive3" profile in > > your > > Maven build and it will add a full set of Hive 3-compatible processors to > > the > > assembly, or you can just go into nifi-nar-bundles/nifi-hive-bundle and > > build > > from there, there will be a nifi-hive3-nar created in > > nifi-hive3-nar/target. The > > processors include SelectHive3QL, PutHive3QL, PutHive3Streaming, and > > PutORC, as well as a Hive3ConnectionPool for the JDBC processors > > (Select/Put). > > > > Regards, > > Matt > > > > On Mon, Jul 30, 2018 at 8:18 AM Michal Tomaszewski > > <michal.tomaszew...@cca.pl> wrote: > > > > > > Hi Pierre, > > > nifi-app.log enclosed. > > > > > > Regards, > > > Mike > > > > > > > > > From: Pierre Villard <pierre.villard...@gmail.com> > > > Sent: Monday, July 30, 2018 11:40 AM > > > To: users@nifi.apache.org > > > Subject: Re: PutHiveStreaming + Hive 3.0 > > > > > > Hi Mike, > > > > > > By any chance, could you share the full stack trace from nifi-app.log? > > > > > > Thanks, > > > Pierre > > > > > > 2018-07-30 11:19 GMT+02:00 Michal Tomaszewski > > <mailto:michal.tomaszew...@cca.pl>: > > > Hello, > > > Is PutHiveStreaming processor working with Hive 3.0? > > > We installed Hortonworks HDP 3.0, compiled newest NiFi 1.8 snapshot and > > got errors: > > > Failed connecting to Hive endpoint table: hivetest3 at > > > thrift://servername. > > > NiFi has access to all site.xml's. Configuration is exactly the same as > > > with > > previous Hive server, that worked properly. > > > > > > Regards, > > > Mike > ________________________________________ Uwaga: Treść niniejszej wiadomości > może być poufna i objęta zakazem jej ujawniania. Jeśli czytelnik tej > wiadomości nie jest jej zamierzonym adresatem, pracownikiem lub pośrednikiem > upoważnionym do jej przekazania adresatowi, informujemy że wszelkie > rozprowadzanie, rozpowszechnianie lub powielanie niniejszej wiadomości jest > zabronione. Jeśli otrzymałeś tę wiadomość omyłkowo, proszę bezzwłocznie > odesłać ją nadawcy, a samą wiadomość usunąć z komputera. Dziękujemy. > ________________________________ Note: The information contained in this > message may be privileged and confidential and protected from disclosure. If > the reader of this message is not the intended recipient, or an employee or > agent responsible for delivering this message to the intended recipient, you > are hereby notified that any dissemination, distribution or copying of this > communication is strictly prohibited.If you have received this communication > in error, please notify the sender immediately by replying to the message and > deleting it from your computer. Thank you. ________________________________
