[
https://issues.apache.org/jira/browse/LIVY-692?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16947289#comment-16947289
]
Wing Yew Poon commented on LIVY-692:
------------------------------------
To clarify, the problem occurs when I am using binary mode (the default for
livy.server.thrift.transport.mode). When I use http mode, then I am able to
connect using
{noformat}
beeline -u
'jdbc:hive2://<livy_server>:<thrift_port>/<db>;principal=HTTP/<livy_server>@<REALM>;transportMode=http;httpPath=cliservice'
{noformat}
and am able to run queries.
However, in that case, there are a slew of exceptions in the Livy server log,
like
{noformat}
2019-10-08 18:02:05,926 INFO
org.apache.livy.thriftserver.cli.ThriftHttpServlet: Could not validate cookie
sent, will try to generate a new cookie
2019-10-08 18:02:05,926 INFO
org.apache.livy.thriftserver.cli.ThriftHttpServlet: Failed to authenticate with
http/_HOST kerberos principal, trying with livy/_HOST kerberos principal
2019-10-08 18:02:05,926 ERROR
org.apache.livy.thriftserver.cli.ThriftHttpServlet: Failed to authenticate with
livy/_HOST kerberos principal
2019-10-08 18:02:05,926 ERROR
org.apache.livy.thriftserver.cli.ThriftHttpServlet: Error:
org.apache.hive.service.auth.HttpAuthenticationException:
java.lang.reflect.UndeclaredThrowableException
at
org.apache.livy.thriftserver.cli.ThriftHttpServlet.doKerberosAuth(ThriftHttpServlet.scala:314)
at
org.apache.livy.thriftserver.cli.ThriftHttpServlet.doPost(ThriftHttpServlet.scala:111)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:707)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:790)
at
org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:848)
at
org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:584)
at
org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:224)
at
org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1180)
at
org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:512)
at
org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)
at
org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1112)
at
org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
at
org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:134)
at org.eclipse.jetty.server.Server.handle(Server.java:539)
at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:333)
at
org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:251)
at
org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:283)
at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:108)
at
org.eclipse.jetty.io.SelectChannelEndPoint$2.run(SelectChannelEndPoint.java:93)
at
org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.executeProduceConsume(ExecuteProduceConsume.java:303)
at
org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceConsume(ExecuteProduceConsume.java:148)
at
org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(ExecuteProduceConsume.java:136)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.reflect.UndeclaredThrowableException
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1748)
at
org.apache.livy.thriftserver.cli.ThriftHttpServlet.doKerberosAuth(ThriftHttpServlet.scala:310)
... 24 more
Caused by: org.apache.hive.service.auth.HttpAuthenticationException:
Authorization header received from the client is empty.
at
org.apache.livy.thriftserver.cli.ThriftHttpServlet$.getAuthHeader(ThriftHttpServlet.scala:407)
at
org.apache.livy.thriftserver.cli.HttpKerberosServerAction.run(ThriftHttpServlet.scala:461)
at
org.apache.livy.thriftserver.cli.HttpKerberosServerAction.run(ThriftHttpServlet.scala:430)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1730)
... 25 more
{noformat}
These exceptions seem to be an unrelated issue. I also see them when using http
mode with my patch.
> Thrift server Kerberos authentication does not use configured principal
> -----------------------------------------------------------------------
>
> Key: LIVY-692
> URL: https://issues.apache.org/jira/browse/LIVY-692
> Project: Livy
> Issue Type: Bug
> Components: Thriftserver
> Affects Versions: 0.6.0
> Reporter: Wing Yew Poon
> Priority: Major
> Time Spent: 10m
> Remaining Estimate: 0h
>
> I tried to use the Thrift server in Livy 0.6. I have a secure cluster, and I
> set
> {noformat}
> livy.server.thrift.authentication=KERBEROS
> {noformat}
> on the server side. I also have livy.server.auth.kerberos.keytab and
> livy.server.auth.kerberos.principal configured. In particular, in my system,
> I have
> {noformat}
> livy.server.auth.kerberos.principal=HTTP/<livy_server>@<REALM>
> {noformat}
> When I try to connect to the Thrift server using the beeline shipped with
> Hive, with
> {noformat}
> beeline -u
> 'jdbc:hive2://<livy_server>:<thrift_port>/<db>;principal=HTTP/<livy_server>@<REALM>'
> {noformat}
> it fails with
> {noformat}
> Error: Could not open client transport with JDBC Uri: ... :: Peer indicated
> failure: GSS initiate failed (state=08S01,code=0)
> {noformat}
> On the server side, the log shows
> {noformat}
> 2019-10-03 15:21:09,536 ERROR org.apache.thrift.transport.TSaslTransport:
> SASL negotiation failure
> javax.security.sasl.SaslException: GSS initiate failed [Caused by
> GSSException: Failure unspecified at GSS-API level (Mechanism level: Invalid
> argument (400) - Cannot find key of appropriate type to decrypt AP REP - DES3
> CBC mode with SHA1-KD)]
> at
> com.sun.security.sasl.gsskerb.GssKrb5Server.evaluateResponse(GssKrb5Server.java:199)
> at
> org.apache.thrift.transport.TSaslTransport$SaslParticipant.evaluateChallengeOrResponse(TSaslTransport.java:539)
> at
> org.apache.thrift.transport.TSaslTransport.open(TSaslTransport.java:283)
> at
> org.apache.thrift.transport.TSaslServerTransport.open(TSaslServerTransport.java:41)
> at
> org.apache.thrift.transport.TSaslServerTransport$Factory.getTransport(TSaslServerTransport.java:216)
> at
> org.apache.livy.thriftserver.auth.TUGIAssumingTransportFactory$$anon$1.run(AuthBridgeServer.scala:143)
> at
> org.apache.livy.thriftserver.auth.TUGIAssumingTransportFactory$$anon$1.run(AuthBridgeServer.scala:142)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:360)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1710)
> at
> org.apache.livy.thriftserver.auth.TUGIAssumingTransportFactory.getTransport(AuthBridgeServer.scala:142)
> at
> org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:269)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748)
> Caused by: GSSException: Failure unspecified at GSS-API level (Mechanism
> level: Invalid argument (400) - Cannot find key of appropriate type to
> decrypt AP REP - DES3 CBC mode with SHA1-KD)
> at
> sun.security.jgss.krb5.Krb5Context.acceptSecContext(Krb5Context.java:856)
> at
> sun.security.jgss.GSSContextImpl.acceptSecContext(GSSContextImpl.java:342)
> at
> sun.security.jgss.GSSContextImpl.acceptSecContext(GSSContextImpl.java:285)
> at
> com.sun.security.sasl.gsskerb.GssKrb5Server.evaluateResponse(GssKrb5Server.java:167)
> ... 14 more
> Caused by: KrbException: Invalid argument (400) - Cannot find key of
> appropriate type to decrypt AP REP - DES3 CBC mode with SHA1-KD
> at sun.security.krb5.KrbApReq.authenticate(KrbApReq.java:278)
> at sun.security.krb5.KrbApReq.<init>(KrbApReq.java:149)
> at
> sun.security.jgss.krb5.InitSecContextToken.<init>(InitSecContextToken.java:108)
> at
> sun.security.jgss.krb5.Krb5Context.acceptSecContext(Krb5Context.java:829)
> ... 17 more
> {noformat}
> On debugging, I found that the UGI in the TUGIAssumingTransportFactory is not
> that of the configured HTTP principal but the UGI of the service principal
> (livy).
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
