hi, we are using livy to connect with spark, could you check livy related documents,
Otherwise file a ticket for us. Thanks, William On Fri, Aug 23, 2019 at 2:11 PM < [email protected]> wrote: > Hello, > > No the authentication in the cluster is MapR-SASL. > > Regards > > -----Original Message----- > From: Qian Wang <[email protected]> > Sent: Thursday, August 22, 2019 7:04 PM > To: [email protected] > Cc: zxBCN De_La_Fuente_Diaz,Alvaro (IT EDS) EXTERNAL < > [email protected]> > Subject: RE: Connect Griffin to Hive secured metastore > > Hi, > > Do you have kerberos auth? If you have, you need config the > > livy.need.kerberos=true > #if livy need kerberos is false then don't need set following two > properties livy.server.auth.kerberos.principal=livy/kerberos.principal > livy.server.auth.kerberos.keytab=/path/to/livy/keytab/file > Best, > Qian > > On Aug 22, 2019, 4:04 AM -0700, > [email protected], wrote: > > Hi Qian, > > > > Thank you very much for your help, we changed the connection to Hive > Metadata to Hive JDBC and now we are able to get Hive Metadata. > > But now we have a problem with Livy authorization, the problem is that > we do not know how to configure user and password for Livy in Griffin, does > anybody know how to do it. > > The error we are getting is the following: > > > > 2019-08-22 10:50:00.830 INFO 83698 --- [ryBean_Worker-2] > > o.a.g.c.j.LivyTaskSubmitHelper [230] : Post To Livy URI is: > > https://inhas68625.eu.boehringer.com:8998/batches > > 2019-08-22 10:50:00.830 INFO 83698 --- [ryBean_Worker-2] > > o.a.g.c.j.LivyTaskSubmitHelper [232] : Need Kerberos:false > > 2019-08-22 10:50:00.830 INFO 83698 --- [ryBean_Worker-2] > > o.a.g.c.j.LivyTaskSubmitHelper [244] : The livy server doesn't need > > Kerberos Authentication > > 2019-08-22 10:50:01.462 ERROR 83698 --- [ryBean_Worker-2] > o.a.g.c.j.SparkSubmitJob [116] : Post spark task ERROR. > > > > org.springframework.web.client.HttpClientErrorException: 401 > > Authentication required at > > org.springframework.web.client.DefaultResponseErrorHandler.handleError > > (DefaultResponseErrorHandler.java:91) > > ~[spring-web-4.3.6.RELEASE.jar!/:4.3.6.RELEASE] > > at > > org.springframework.web.client.RestTemplate.handleResponse(RestTemplat > > e.java:700) ~[spring-web-4.3.6.RELEASE.jar!/:4.3.6.RELEASE] > > at > > org.springframework.web.client.RestTemplate.doExecute(RestTemplate.jav > > a:653) ~[spring-web-4.3.6.RELEASE.jar!/:4.3.6.RELEASE] > > at > > org.springframework.web.client.RestTemplate.execute(RestTemplate.java: > > 613) ~[spring-web-4.3.6.RELEASE.jar!/:4.3.6.RELEASE] > > at > > org.springframework.web.client.RestTemplate.postForObject(RestTemplate > > .java:380) ~[spring-web-4.3.6.RELEASE.jar!/:4.3.6.RELEASE] > > at > > org.apache.griffin.core.job.LivyTaskSubmitHelper.postToLivy(LivyTaskSu > > bmitHelper.java:248) ~[classes!/:0.6.0-SNAPSHOT] at > > org.apache.griffin.core.job.SparkSubmitJob.post2Livy(SparkSubmitJob.ja > > va:131) ~[classes!/:0.6.0-SNAPSHOT] at > > org.apache.griffin.core.job.SparkSubmitJob.post2LivyWithRetry(SparkSub > > mitJob.java:224) ~[classes!/:0.6.0-SNAPSHOT] at > > org.apache.griffin.core.job.SparkSubmitJob.saveJobInstance(SparkSubmit > > Job.java:213) ~[classes!/:0.6.0-SNAPSHOT] at > > org.apache.griffin.core.job.SparkSubmitJob.execute(SparkSubmitJob.java > > :113) [classes!/:0.6.0-SNAPSHOT] at > > org.quartz.core.JobRunShell.run(JobRunShell.java:202) > > [quartz-2.2.2.jar!/:?] at > > org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.ja > > va:573) [quartz-2.2.2.jar!/:?] > > > > Waiting for your answer. > > > > Regards > > > > -----Original Message----- > > From: Qian Wang <[email protected]> > > Sent: Wednesday, August 21, 2019 7:17 PM > > To: [email protected] > > Subject: Re: Connect Griffin to Hive secured metastore > > > > Hi, > > > > You have an alternative method to get Hive Metadata by using Hive JDBC. > If you want to use JDBC, you need change > org.apache.griffin.core.metastore.hive.HiveMetaStoreController: > > @Autowired > > @Qualifier(value = "jdbcSvc") > > private HiveMetaStoreService hiveMetaStoreService; Also, if your Hive is > Authenticated by Kerberos, you need setup application.properties: > > #Hive jdbc > > hive.jdbc.className=org.apache.hive.jdbc.HiveDriver > > hive.jdbc.url=jdbc:hive2://localhost:10000/ #your Hive url > hive.need.kerberos=true # if you need Kerberos Auth hive.keytab.user= > [email protected] hive.keytab.path=/path/to/keytab/file #here is absolute path > Hopefully can answer your question. > > > > Best, > > Eric > > On Aug 21, 2019, 7:52 AM -0700, > [email protected], wrote: > > > Hello, > > > > > > We are trying to connect Griffin to a secured Hive metastore, does > anybody know how to configure Griffin for this connection? We are getting > authorization errors in the metastore. > > > > > > Waiting for your answer. > > > > > > Regards >
