Hi Qian, Thank you very much for your help, we changed the connection to Hive Metadata to Hive JDBC and now we are able to get Hive Metadata. But now we have a problem with Livy authorization, the problem is that we do not know how to configure user and password for Livy in Griffin, does anybody know how to do it. The error we are getting is the following:
2019-08-22 10:50:00.830 INFO 83698 --- [ryBean_Worker-2] o.a.g.c.j.LivyTaskSubmitHelper [230] : Post To Livy URI is: https://inhas68625.eu.boehringer.com:8998/batches 2019-08-22 10:50:00.830 INFO 83698 --- [ryBean_Worker-2] o.a.g.c.j.LivyTaskSubmitHelper [232] : Need Kerberos:false 2019-08-22 10:50:00.830 INFO 83698 --- [ryBean_Worker-2] o.a.g.c.j.LivyTaskSubmitHelper [244] : The livy server doesn't need Kerberos Authentication 2019-08-22 10:50:01.462 ERROR 83698 --- [ryBean_Worker-2] o.a.g.c.j.SparkSubmitJob [116] : Post spark task ERROR. org.springframework.web.client.HttpClientErrorException: 401 Authentication required at org.springframework.web.client.DefaultResponseErrorHandler.handleError(DefaultResponseErrorHandler.java:91) ~[spring-web-4.3.6.RELEASE.jar!/:4.3.6.RELEASE] at org.springframework.web.client.RestTemplate.handleResponse(RestTemplate.java:700) ~[spring-web-4.3.6.RELEASE.jar!/:4.3.6.RELEASE] at org.springframework.web.client.RestTemplate.doExecute(RestTemplate.java:653) ~[spring-web-4.3.6.RELEASE.jar!/:4.3.6.RELEASE] at org.springframework.web.client.RestTemplate.execute(RestTemplate.java:613) ~[spring-web-4.3.6.RELEASE.jar!/:4.3.6.RELEASE] at org.springframework.web.client.RestTemplate.postForObject(RestTemplate.java:380) ~[spring-web-4.3.6.RELEASE.jar!/:4.3.6.RELEASE] at org.apache.griffin.core.job.LivyTaskSubmitHelper.postToLivy(LivyTaskSubmitHelper.java:248) ~[classes!/:0.6.0-SNAPSHOT] at org.apache.griffin.core.job.SparkSubmitJob.post2Livy(SparkSubmitJob.java:131) ~[classes!/:0.6.0-SNAPSHOT] at org.apache.griffin.core.job.SparkSubmitJob.post2LivyWithRetry(SparkSubmitJob.java:224) ~[classes!/:0.6.0-SNAPSHOT] at org.apache.griffin.core.job.SparkSubmitJob.saveJobInstance(SparkSubmitJob.java:213) ~[classes!/:0.6.0-SNAPSHOT] at org.apache.griffin.core.job.SparkSubmitJob.execute(SparkSubmitJob.java:113) [classes!/:0.6.0-SNAPSHOT] at org.quartz.core.JobRunShell.run(JobRunShell.java:202) [quartz-2.2.2.jar!/:?] at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:573) [quartz-2.2.2.jar!/:?] Waiting for your answer. Regards -----Original Message----- From: Qian Wang <[email protected]> Sent: Wednesday, August 21, 2019 7:17 PM To: [email protected] Subject: Re: Connect Griffin to Hive secured metastore Hi, You have an alternative method to get Hive Metadata by using Hive JDBC. If you want to use JDBC, you need change org.apache.griffin.core.metastore.hive.HiveMetaStoreController: @Autowired @Qualifier(value = "jdbcSvc") private HiveMetaStoreService hiveMetaStoreService; Also, if your Hive is Authenticated by Kerberos, you need setup application.properties: #Hive jdbc hive.jdbc.className=org.apache.hive.jdbc.HiveDriver hive.jdbc.url=jdbc:hive2://localhost:10000/ #your Hive url hive.need.kerberos=true # if you need Kerberos Auth [email protected] hive.keytab.path=/path/to/keytab/file #here is absolute path Hopefully can answer your question. Best, Eric On Aug 21, 2019, 7:52 AM -0700, [email protected], wrote: > Hello, > > We are trying to connect Griffin to a secured Hive metastore, does anybody > know how to configure Griffin for this connection? We are getting > authorization errors in the metastore. > > Waiting for your answer. > > Regards
