Thanks Jeff, Getting back to this now. I will try your suggestion when I get my environment running again. I did run into something today that I missed earlier and now I have some questions:
I noticed a setting in livy-client.conf.template for livy.rsc.proxy-user:: # The user that should be impersonated when requesting a Livy session # livy.rsc.proxy-user = What effect would that property have when I'm doing user impersonation? Is it necessary? Also, I see this at the top of livy-client.conf.template. Does that mean I should put the file in the 'conf' folder or some other folder accessible to the spark executors, like repl_2.10-jars ? # Configurations for a Livy Client, any configurations set here will override any # livy or spark-default configurations. # # Before a Livy Client is able to load these configurations the folder containing # this file must be added to the application classpath # ________________________________ From: Jeff Zhang <zjf...@gmail.com> Sent: Tuesday, July 10, 2018 5:27:52 PM To: user@livy.incubator.apache.org Subject: Re: user impersonation in Livy not working Make sure you enable hiveContext. set livy.repl.enable-hive-context to true in livy.conf Harsch, Tim <tim.har...@teradata.com<mailto:tim.har...@teradata.com>>于2018年7月10日周二 下午11:53写道： Hi, I have been unable to get proxyUsers to work with Livy. I seem to have Kerberos configured correctly, as evidenced by the last beeline command. When I start a livy session the server log will usually say something like: “owner: kylo, proxyUser: Some(dladmin)” so it seems the server should know what to do. But no matter what I try I always get “org.apache.spark.sql.AnalysisException: Table not found” # Ambari config [/var/folders/_n/k2nq_2fx5t5__fcxqdl2tc6j_p8_px/T/com.microsoft.Outlook/WebArchiveCopyPasteTempFiles/p1426] # Livy Configuration livy.impersonation.enabled = true livy.server.auth.type = kerberos livy.server.launch.kerberos.principal = kylo/sandbox.kylo.io@KYLO livy.server.launch.kerberos.keytab = /etc/security/keytabs/kylo.keytab livy.server.auth.kerberos.principal = HTTP/sandbox.kylo.io@KYLO livy.server.auth.kerberos.keytab = /etc/security/keytabs/spnego.service.keytab livy.server.access_control.enabled = true livy.server.access_control.users = kylo,dladmin livy.server.access_control.modify-users = kylo livy.superusers = kylo # Curl connect kinit -kt /etc/security/keytabs/kylo.keytab kylo/sandbox.kylo.io@KYLO klist curl --negotiate -u : -s -X POST --data '{"kind": "spark", "proxyUser": "dladmin"}' -H "Content-Type: application/json" sandbox.kylo.io:8998/sessions<http://sandbox.kylo.io:8998/sessions> curl --negotiate -u : -s -X GET sandbox.kylo.io:8998/sessions<http://sandbox.kylo.io:8998/sessions> * NOTE: verify proxyUser="dladmin" in response. it will be null if impersonation is not enabled # submit this code to Livy: sqlContext.sql("select * from default.d4").show() # Response: { "code": "sqlContext.sql(\"select * from default.d4\").show()", "id": 0, "output": { "ename": "Error", "evalue": "org.apache.spark.sql.AnalysisException: Table not found: `default`.`d4`;", .... # Can "dladmin" see table data? [root@sandbox more]# beeline -u "jdbc:hive2://localhost:10000/;principal=hive/sandbox.kylo.io@KYLO;;hive.server2.proxy.user=dladmin" Connecting to jdbc:hive2://localhost:10000/;principal=hive/sandbox.kylo.io@KYLO;;hive.server2.proxy.user=dladmin Connected to: Apache Hive (version 1.2.1000.2.5.6.0-40) Driver: Hive JDBC (version 1.2.1000.2.5.6.0-40) Transaction isolation: TRANSACTION_REPEATABLE_READ Beeline version 1.2.1000.2.5.6.0-40 by Apache Hive 0: jdbc:hive2://localhost:10000/> select * from default.d4; +--------+--+ | d4.d4 | +--------+--+ | d4 | +--------+--+ 1 row selected (0.132 seconds)
