[
https://issues.apache.org/jira/browse/STORM-1596?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15178562#comment-15178562
]
ASF GitHub Bot commented on STORM-1596:
---------------------------------------
Github user asfgit closed the pull request at:
https://github.com/apache/storm/pull/1177
> Multiple Subject sharing Kerberos TGT - causes services to fail
> ---------------------------------------------------------------
>
> Key: STORM-1596
> URL: https://issues.apache.org/jira/browse/STORM-1596
> Project: Apache Storm
> Issue Type: Bug
> Affects Versions: 0.10.0, 1.0.0, 0.10.1, 2.0.0
> Reporter: Kishor Patil
> Assignee: Kishor Patil
> Priority: Critical
>
> With multiple threads accessing same {{Subject}}, it can cause
> {{ServiceTicket}} in use be by one thread be destroyed by another thread.
> Running BasicDRPCTopology with high parallelism in secure cluster would
> reproduce the issue.
> Here is sample log from such a scenarios:
> {code}
> 2016-01-20 15:52:26.904 o.a.t.t.TSaslTransport [ERROR] SASL negotiation
> failure
> javax.security.sasl.SaslException: GSS initiate failed
> at
> com.sun.security.sasl.gsskerb.GssKrb5Client.evaluateChallenge(GssKrb5Client.java:211)
> ~[?:1.8.0_40]
> at
> org.apache.thrift7.transport.TSaslClientTransport.handleSaslStartMessage(TSaslClientTransport.java:94)
> ~[storm-core-0.10.1.y.jar:0.10.1.y]
> at
> org.apache.thrift7.transport.TSaslTransport.open(TSaslTransport.java:271)
> [storm-core-0.10.1.y.jar:0.10.1.y]
> at
> org.apache.thrift7.transport.TSaslClientTransport.open(TSaslClientTransport.java:37)
> [storm-core-0.10.1.y.jar:0.10.1.y]
> at
> backtype.storm.security.auth.kerberos.KerberosSaslTransportPlugin$1.run(KerberosSaslTransportPlugin.java:195)
> [storm-core-0.10.1.y.jar:0.10.1.y]
> at
> backtype.storm.security.auth.kerberos.KerberosSaslTransportPlugin$1.run(KerberosSaslTransportPlugin.java:191)
> [storm-core-0.10.1.y.jar:0.10.1.y]
> at java.security.AccessController.doPrivileged(Native Method)
> ~[?:1.8.0_40]
> at javax.security.auth.Subject.doAs(Subject.java:422) [?:1.8.0_40]
> at
> backtype.storm.security.auth.kerberos.KerberosSaslTransportPlugin.connect(KerberosSaslTransportPlugin.java:190)
> [storm-core-0.10.1.y.jar:0.10.1.y]
> at
> backtype.storm.security.auth.TBackoffConnect.doConnectWithRetry(TBackoffConnect.java:54)
> [storm-core-0.10.1.y.jar:0.10.1.y]
> at
> backtype.storm.security.auth.ThriftClient.reconnect(ThriftClient.java:109)
> [storm-core-0.10.1.y.jar:0.10.1.y]
> at
> backtype.storm.drpc.DRPCInvocationsClient.reconnectClient(DRPCInvocationsClient.java:57)
> [storm-core-0.10.1.y.jar:0.10.1.y]
> at
> backtype.storm.drpc.ReturnResults.reconnectClient(ReturnResults.java:113)
> [storm-core-0.10.1.y.jar:0.10.1.y]
> at backtype.storm.drpc.ReturnResults.execute(ReturnResults.java:103)
> [storm-core-0.10.1.y.jar:0.10.1.y]
> at
> backtype.storm.daemon.executor$fn__6377$tuple_action_fn__6379.invoke(executor.clj:689)
> [storm-core-0.10.1.y.jar:0.10.1.y]
> at
> backtype.storm.daemon.executor$mk_task_receiver$fn__6301.invoke(executor.clj:448)
> [storm-core-0.10.1.y.jar:0.10.1.y]
> at
> backtype.storm.disruptor$clojure_handler$reify__6018.onEvent(disruptor.clj:40)
> [storm-core-0.10.1.y.jar:0.10.1.y]
> at
> backtype.storm.utils.DisruptorQueue.consumeBatchToCursor(DisruptorQueue.java:437)
> [storm-core-0.10.1.y.jar:0.10.1.y]
> at
> backtype.storm.utils.DisruptorQueue.consumeBatchWhenAvailable(DisruptorQueue.java:416)
> [storm-core-0.10.1.y.jar:0.10.1.y]
> at
> backtype.storm.disruptor$consume_batch_when_available.invoke(disruptor.clj:73)
> [storm-core-0.10.1.y.jar:0.10.1.y]
> at
> backtype.storm.daemon.executor$fn__6377$fn__6390$fn__6441.invoke(executor.clj:801)
> [storm-core-0.10.1.y.jar:0.10.1.y]
> at backtype.storm.util$async_loop$fn__742.invoke(util.clj:482)
> [storm-core-0.10.1.y.jar:0.10.1.y]
> at clojure.lang.AFn.run(AFn.java:22) [clojure-1.6.0.jar:?]
> at java.lang.Thread.run(Thread.java:745) [?:1.8.0_40]
> Caused by: org.ietf.jgss.GSSException: No valid credentials provided
> (Mechanism level: The ticket isn't for us (35) - BAD TGS SERVER NAME)
> at
> sun.security.jgss.krb5.Krb5Context.initSecContext(Krb5Context.java:770)
> ~[?:1.8.0_40]
> at
> sun.security.jgss.GSSContextImpl.initSecContext(GSSContextImpl.java:248)
> ~[?:1.8.0_40]
> at
> sun.security.jgss.GSSContextImpl.initSecContext(GSSContextImpl.java:179)
> ~[?:1.8.0_40]
> at
> com.sun.security.sasl.gsskerb.GssKrb5Client.evaluateChallenge(GssKrb5Client.java:192)
> ~[?:1.8.0_40]
> ... 23 more
> Caused by: sun.security.krb5.KrbException: The ticket isn't for us (35) - BAD
> TGS SERVER NAME
> at sun.security.krb5.KrbTgsRep.<init>(KrbTgsRep.java:73) ~[?:1.8.0_40]
> at sun.security.krb5.KrbTgsReq.getReply(KrbTgsReq.java:259)
> ~[?:1.8.0_40]
> at sun.security.krb5.KrbTgsReq.sendAndGetCreds(KrbTgsReq.java:270)
> ~[?:1.8.0_40]
> at
> sun.security.krb5.internal.CredentialsUtil.serviceCreds(CredentialsUtil.java:302)
> ~[?:1.8.0_40]
> at
> sun.security.krb5.internal.CredentialsUtil.acquireServiceCreds(CredentialsUtil.java:120)
> ~[?:1.8.0_40]
> at
> sun.security.krb5.Credentials.acquireServiceCreds(Credentials.java:458)
> ~[?:1.8.0_40]
> at
> sun.security.jgss.krb5.Krb5Context.initSecContext(Krb5Context.java:693)
> ~[?:1.8.0_40]
> at
> sun.security.jgss.GSSContextImpl.initSecContext(GSSContextImpl.java:248)
> ~[?:1.8.0_40]
> at
> sun.security.jgss.GSSContextImpl.initSecContext(GSSContextImpl.java:179)
> ~[?:1.8.0_40]
> at
> com.sun.security.sasl.gsskerb.GssKrb5Client.evaluateChallenge(GssKrb5Client.java:192)
> ~[?:1.8.0_40]
> ... 23 more
> Caused by: sun.security.krb5.Asn1Exception: Identifier doesn't match expected
> value (906)
> at sun.security.krb5.internal.KDCRep.init(KDCRep.java:140)
> ~[?:1.8.0_40]
> at sun.security.krb5.internal.TGSRep.init(TGSRep.java:65)
> ~[?:1.8.0_40]
> at sun.security.krb5.internal.TGSRep.<init>(TGSRep.java:60)
> ~[?:1.8.0_40]
> at sun.security.krb5.KrbTgsRep.<init>(KrbTgsRep.java:55) ~[?:1.8.0_40]
> at sun.security.krb5.KrbTgsReq.getReply(KrbTgsReq.java:259)
> ~[?:1.8.0_40]
> at sun.security.krb5.KrbTgsReq.sendAndGetCreds(KrbTgsReq.java:270)
> ~[?:1.8.0_40]
> at
> sun.security.krb5.internal.CredentialsUtil.serviceCreds(CredentialsUtil.java:302)
> ~[?:1.8.0_40]
> at
> sun.security.krb5.internal.CredentialsUtil.acquireServiceCreds(CredentialsUtil.java:120)
> ~[?:1.8.0_40]
> at
> sun.security.krb5.Credentials.acquireServiceCreds(Credentials.java:458)
> ~[?:1.8.0_40]
> at
> sun.security.jgss.krb5.Krb5Context.initSecContext(Krb5Context.java:693)
> ~[?:1.8.0_40]
> at
> sun.security.jgss.GSSContextImpl.initSecContext(GSSContextImpl.java:248)
> ~[?:1.8.0_40]
> at
> sun.security.jgss.GSSContextImpl.initSecContext(GSSContextImpl.java:179)
> ~[?:1.8.0_40]
> at
> com.sun.security.sasl.gsskerb.GssKrb5Client.evaluateChallenge(GssKrb5Client.java:192)
> ~[?:1.8.0_40]
> ... 23 more
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
