[
https://issues.apache.org/jira/browse/AMBARI-17258?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Dmitry Lysnichenko updated AMBARI-17258:
----------------------------------------
Component/s: ambari-server
> Storm service check failed after Ambari upgrade
> -----------------------------------------------
>
> Key: AMBARI-17258
> URL: https://issues.apache.org/jira/browse/AMBARI-17258
> Project: Ambari
> Issue Type: Bug
> Components: ambari-server
> Reporter: Dmitry Lysnichenko
> Assignee: Dmitry Lysnichenko
> Attachments: AMBARI-17258.patch
>
>
> Marked this as blocker since the issue affected many EU End to End test runs
> that involved running a service check as part of pre-check condition for EU
> ambari-server-2.4.0.0-665.x86_64
> ambari-server --hash
> *Steps*
> # Deploy HDP-2.3.2.0 cluster with Ambari 2.1.2 (secure, non-HA cluster)
> # Upgrade Ambari to 2.4.0.0-665
> # Stop and Start all services
> # Run service check for Storm
> *Result*
> Failure as below:
> {code}
> stderr: /var/lib/ambari-agent/data/errors-729.txt
> Traceback (most recent call last):
> File
> "/var/lib/ambari-agent/cache/common-services/STORM/0.9.1/package/scripts/service_check.py",
> line 79, in <module>
> ServiceCheck().execute()
> File
> "/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py",
> line 257, in execute
> method(env)
> File
> "/var/lib/ambari-agent/cache/common-services/STORM/0.9.1/package/scripts/service_check.py",
> line 70, in service_check
> user=params.storm_user
> File "/usr/lib/python2.6/site-packages/resource_management/core/base.py",
> line 155, in __init__
> self.env.run()
> File
> "/usr/lib/python2.6/site-packages/resource_management/core/environment.py",
> line 160, in run
> self.run_action(resource, action)
> File
> "/usr/lib/python2.6/site-packages/resource_management/core/environment.py",
> line 124, in run_action
> provider_action()
> File
> "/usr/lib/python2.6/site-packages/resource_management/core/providers/system.py",
> line 273, in action_run
> tries=self.resource.tries, try_sleep=self.resource.try_sleep)
> File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py",
> line 70, in inner
> result = function(command, **kwargs)
> File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py",
> line 92, in checked_call
> tries=tries, try_sleep=try_sleep)
> File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py",
> line 140, in _call_wrapper
> result = _call(command, **kwargs_copy)
> File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py",
> line 293, in _call
> raise Fail(err_msg)
> resource_management.core.exceptions.Fail: Execution of 'storm jar
> /tmp/wordCount.jar storm.starter.WordCountTopology
> WordCountid16ac396c_date241516' returned 1. 951 [main] INFO b.s.u.Utils -
> Using defaults.yaml from resources
> 1088 [main] INFO b.s.u.Utils - Using storm.yaml from resources
> 1184 [main] INFO b.s.u.Utils - Using defaults.yaml from resources
> 1240 [main] INFO b.s.u.Utils - Using storm.yaml from resources
> 1285 [main] INFO b.s.StormSubmitter - Generated ZooKeeper secret payload for
> MD5-digest: -7463484230184273671:-7128399429695564111
> 1287 [main] INFO b.s.s.a.AuthUtils - Got AutoCreds []
> 1292 [main] WARN b.s.u.NimbusClient - Using deprecated config nimbus.host
> for backward compatibility. Please update your storm.yaml so it only has
> config nimbus.seeds
> 1352 [main] INFO b.s.u.StormBoundedExponentialBackoffRetry - The
> baseSleepTimeMs [2000] the maxSleepTimeMs [60000] the maxRetries [5]
> 1615 [main] INFO o.a.s.z.Login - successfully logged in.
> 1650 [main] ERROR o.a.t.t.TSaslTransport - SASL negotiation failure
> javax.security.sasl.SaslException: GSS initiate failed
> at
> com.sun.security.sasl.gsskerb.GssKrb5Client.evaluateChallenge(GssKrb5Client.java:212)
> ~[?:1.7.0_67]
> at
> org.apache.thrift7.transport.TSaslClientTransport.handleSaslStartMessage(TSaslClientTransport.java:94)
> ~[storm-core-0.10.0.2.3.2.0-2950.jar:0.10.0.2.3.2.0-2950]
> at org.apache.thrift7.transport.TSaslTransport.open(TSaslTransport.java:271)
> [storm-core-0.10.0.2.3.2.0-2950.jar:0.10.0.2.3.2.0-2950]
> at
> org.apache.thrift7.transport.TSaslClientTransport.open(TSaslClientTransport.java:37)
> [storm-core-0.10.0.2.3.2.0-2950.jar:0.10.0.2.3.2.0-2950]
> at
> backtype.storm.security.auth.kerberos.KerberosSaslTransportPlugin$1.run(KerberosSaslTransportPlugin.java:145)
> [storm-core-0.10.0.2.3.2.0-2950.jar:0.10.0.2.3.2.0-2950]
> at
> backtype.storm.security.auth.kerberos.KerberosSaslTransportPlugin$1.run(KerberosSaslTransportPlugin.java:141)
> [storm-core-0.10.0.2.3.2.0-2950.jar:0.10.0.2.3.2.0-2950]
> at java.security.AccessController.doPrivileged(Native Method) ~[?:1.7.0_67]
> at javax.security.auth.Subject.doAs(Subject.java:415) [?:1.7.0_67]
> at
> backtype.storm.security.auth.kerberos.KerberosSaslTransportPlugin.connect(KerberosSaslTransportPlugin.java:140)
> [storm-core-0.10.0.2.3.2.0-2950.jar:0.10.0.2.3.2.0-2950]
> at
> backtype.storm.security.auth.TBackoffConnect.doConnectWithRetry(TBackoffConnect.java:48)
> [storm-core-0.10.0.2.3.2.0-2950.jar:0.10.0.2.3.2.0-2950]
> at backtype.storm.security.auth.ThriftClient.reconnect(ThriftClient.java:103)
> [storm-core-0.10.0.2.3.2.0-2950.jar:0.10.0.2.3.2.0-2950]
> at backtype.storm.security.auth.ThriftClient.<init>(ThriftClient.java:72)
> [storm-core-0.10.0.2.3.2.0-2950.jar:0.10.0.2.3.2.0-2950]
> at backtype.storm.utils.NimbusClient.<init>(NimbusClient.java:104)
> [storm-core-0.10.0.2.3.2.0-2950.jar:0.10.0.2.3.2.0-2950]
> at backtype.storm.utils.NimbusClient.<init>(NimbusClient.java:95)
> [storm-core-0.10.0.2.3.2.0-2950.jar:0.10.0.2.3.2.0-2950]
> at
> backtype.storm.utils.NimbusClient.getConfiguredClientAs(NimbusClient.java:67)
> [storm-core-0.10.0.2.3.2.0-2950.jar:0.10.0.2.3.2.0-2950]
> at backtype.storm.StormSubmitter.submitTopologyAs(StormSubmitter.java:225)
> [storm-core-0.10.0.2.3.2.0-2950.jar:0.10.0.2.3.2.0-2950]
> at backtype.storm.StormSubmitter.submitTopology(StormSubmitter.java:271)
> [storm-core-0.10.0.2.3.2.0-2950.jar:0.10.0.2.3.2.0-2950]
> at backtype.storm.StormSubmitter.submitTopology(StormSubmitter.java:157)
> [storm-core-0.10.0.2.3.2.0-2950.jar:0.10.0.2.3.2.0-2950]
> at storm.starter.WordCountTopology.main(WordCountTopology.java:77)
> [wordCount.jar:?]
> Caused by: org.ietf.jgss.GSSException: No valid credentials provided
> (Mechanism level: Server not found in Kerberos database (7) - UNKNOWN_SERVER)
> at sun.security.jgss.krb5.Krb5Context.initSecContext(Krb5Context.java:710)
> ~[?:1.7.0_67]
> at sun.security.jgss.GSSContextImpl.initSecContext(GSSContextImpl.java:248)
> ~[?:1.7.0_67]
> at sun.security.jgss.GSSContextImpl.initSecContext(GSSContextImpl.java:179)
> ~[?:1.7.0_67]
> at
> com.sun.security.sasl.gsskerb.GssKrb5Client.evaluateChallenge(GssKrb5Client.java:193)
> ~[?:1.7.0_67]
> ... 18 more
> Caused by: sun.security.krb5.KrbException: Server not found in Kerberos
> database (7) - UNKNOWN_SERVER
> at sun.security.krb5.KrbTgsRep.<init>(KrbTgsRep.java:73) ~[?:1.7.0_67]
> at sun.security.krb5.KrbTgsReq.getReply(KrbTgsReq.java:192) ~[?:1.7.0_67]
> at sun.security.krb5.KrbTgsReq.sendAndGetCreds(KrbTgsReq.java:203)
> ~[?:1.7.0_67]
> at
> sun.security.krb5.internal.CredentialsUtil.serviceCreds(CredentialsUtil.java:309)
> ~[?:1.7.0_67]
> at
> sun.security.krb5.internal.CredentialsUtil.acquireServiceCreds(CredentialsUtil.java:115)
> ~[?:1.7.0_67]
> at sun.security.krb5.Credentials.acquireServiceCreds(Credentials.java:454)
> ~[?:1.7.0_67]
> at sun.security.jgss.krb5.Krb5Context.initSecContext(Krb5Context.java:641)
> ~[?:1.7.0_67]
> at sun.security.jgss.GSSContextImpl.initSecContext(GSSContextImpl.java:248)
> ~[?:1.7.0_67]
> at sun.security.jgss.GSSContextImpl.initSecContext(GSSContextImpl.java:179)
> ~[?:1.7.0_67]
> at
> com.sun.security.sasl.gsskerb.GssKrb5Client.evaluateChallenge(GssKrb5Client.java:193)
> ~[?:1.7.0_67]
> ... 18 more
> Caused by: sun.security.krb5.Asn1Exception: Identifier doesn't match expected
> value (906)
> at sun.security.krb5.internal.KDCRep.init(KDCRep.java:143) ~[?:1.7.0_67]
> at sun.security.krb5.internal.TGSRep.init(TGSRep.java:66) ~[?:1.7.0_67]
> at sun.security.krb5.internal.TGSRep.<init>(TGSRep.java:61) ~[?:1.7.0_67]
> at sun.security.krb5.KrbTgsRep.<init>(KrbTgsRep.java:55) ~[?:1.7.0_67]
> at sun.security.krb5.KrbTgsReq.getReply(KrbTgsReq.java:192) ~[?:1.7.0_67]
> at sun.security.krb5.KrbTgsReq.sendAndGetCreds(KrbTgsReq.java:203)
> ~[?:1.7.0_67]
> at
> sun.security.krb5.internal.CredentialsUtil.serviceCreds(CredentialsUtil.java:309)
> ~[?:1.7.0_67]
> at
> sun.security.krb5.internal.CredentialsUtil.acquireServiceCreds(CredentialsUtil.java:115)
> ~[?:1.7.0_67]
> at sun.security.krb5.Credentials.acquireServiceCreds(Credentials.java:454)
> ~[?:1.7.0_67]
> at sun.security.jgss.krb5.Krb5Context.initSecContext(Krb5Context.java:641)
> ~[?:1.7.0_67]
> at sun.security.jgss.GSSContextImpl.initSecContext(GSSContextImpl.java:248)
> ~[?:1.7.0_67]
> at sun.security.jgss.GSSContextImpl.initSecContext(GSSContextImpl.java:179)
> ~[?:1.7.0_67]
> at
> com.sun.security.sasl.gsskerb.GssKrb5Client.evaluateChallenge(GssKrb5Client.java:193)
> ~[?:1.7.0_67]
> ... 18 more
> 1666 [main] ERROR b.s.s.a.k.KerberosSaslTransportPlugin - Client failed to
> open SaslClientTransport to interact with a server during session initiation:
> org.apache.thrift7.transport.TTransportException: GSS initiate failed
> org.apache.thrift7.transport.TTransportException: GSS initiate failed
> at
> org.apache.thrift7.transport.TSaslTransport.sendAndThrowMessage(TSaslTransport.java:232)
> ~[storm-core-0.10.0.2.3.2.0-2950.jar:0.10.0.2.3.2.0-2950]
> at org.apache.thrift7.transport.TSaslTransport.open(TSaslTransport.java:316)
> ~[storm-core-0.10.0.2.3.2.0-2950.jar:0.10.0.2.3.2.0-2950]
> at
> org.apache.thrift7.transport.TSaslClientTransport.open(TSaslClientTransport.java:37)
> ~[storm-core-0.10.0.2.3.2.0-2950.jar:0.10.0.2.3.2.0-2950]
> at
> backtype.storm.security.auth.kerberos.KerberosSaslTransportPlugin$1.run(KerberosSaslTransportPlugin.java:145)
> [storm-core-0.10.0.2.3.2.0-2950.jar:0.10.0.2.3.2.0-2950]
> at
> backtype.storm.security.auth.kerberos.KerberosSaslTransportPlugin$1.run(KerberosSaslTransportPlugin.java:141)
> [storm-core-0.10.0.2.3.2.0-2950.jar:0.10.0.2.3.2.0-2950]
> at java.security.AccessController.doPrivileged(Native Method) ~[?:1.7.0_67]
> at javax.security.auth.Subject.doAs(Subject.java:415) [?:1.7.0_67]
> at
> backtype.storm.security.auth.kerberos.KerberosSaslTransportPlugin.connect(KerberosSaslTransportPlugin.java:140)
> [storm-core-0.10.0.2.3.2.0-2950.jar:0.10.0.2.3.2.0-2950]
> at
> backtype.storm.security.auth.TBackoffConnect.doConnectWithRetry(TBackoffConnect.java:48)
> [storm-core-0.10.0.2.3.2.0-2950.jar:0.10.0.2.3.2.0-2950]
> at backtype.storm.security.auth.ThriftClient.reconnect(ThriftClient.java:103)
> [storm-core-0.10.0.2.3.2.0-2950.jar:0.10.0.2.3.2.0-2950]
> at backtype.storm.security.auth.ThriftClient.<init>(ThriftClient.java:72)
> [storm-core-0.10.0.2.3.2.0-2950.jar:0.10.0.2.3.2.0-2950]
> at backtype.storm.utils.NimbusClient.<init>(NimbusClient.java:104)
> [storm-core-0.10.0.2.3.2.0-2950.jar:0.10.0.2.3.2.0-2950]
> at backtype.storm.utils.NimbusClient.<init>(NimbusClient.java:95)
> [storm-core-0.10.0.2.3.2.0-2950.jar:0.10.0.2.3.2.0-2950]
> at
> backtype.storm.utils.NimbusClient.getConfiguredClientAs(NimbusClient.java:67)
> [storm-core-0.10.0.2.3.2.0-2950.jar:0.10.0.2.3.2.0-2950]
> at backtype.storm.StormSubmitter.submitTopologyAs(StormSubmitter.java:225)
> [storm-core-0.10.0.2.3.2.0-2950.jar:0.10.0.2.3.2.0-2950]
> at backtype.storm.StormSubmitter.submitTopology(StormSubmitter.java:271)
> [storm-core-0.10.0.2.3.2.0-2950.jar:0.10.0.2.3.2.0-2950]
> at backtype.storm.StormSubmitter.submitTopology(StormSubmitter.java:157)
> [storm-core-0.10.0.2.3.2.0-2950.jar:0.10.0.2.3.2.0-2950]
> at storm.starter.WordCountTopology.main(WordCountTopology.java:77)
> [wordCount.jar:?]
> 1675 [main] WARN b.s.u.NimbusClient - Ignoring exception while trying to get
> leader nimbus info from localhost. will retry with a different seed host.
> org.apache.thrift7.transport.TTransportException: SASL authentication not
> complete
> at org.apache.thrift7.transport.TSaslTransport.write(TSaslTransport.java:474)
> ~[storm-core-0.10.0.2.3.2.0-2950.jar:0.10.0.2.3.2.0-2950]
> at
> org.apache.thrift7.transport.TSaslClientTransport.write(TSaslClientTransport.java:37)
> ~[storm-core-0.10.0.2.3.2.0-2950.jar:0.10.0.2.3.2.0-2950]
> at
> org.apache.thrift7.protocol.TBinaryProtocol.writeI32(TBinaryProtocol.java:178)
> ~[storm-core-0.10.0.2.3.2.0-2950.jar:0.10.0.2.3.2.0-2950]
> at
> org.apache.thrift7.protocol.TBinaryProtocol.writeMessageBegin(TBinaryProtocol.java:106)
> ~[storm-core-0.10.0.2.3.2.0-2950.jar:0.10.0.2.3.2.0-2950]
> at org.apache.thrift7.TServiceClient.sendBase(TServiceClient.java:62)
> ~[storm-core-0.10.0.2.3.2.0-2950.jar:0.10.0.2.3.2.0-2950]
> at
> backtype.storm.generated.Nimbus$Client.send_getClusterInfo(Nimbus.java:553)
> ~[storm-core-0.10.0.2.3.2.0-2950.jar:0.10.0.2.3.2.0-2950]
> at backtype.storm.generated.Nimbus$Client.getClusterInfo(Nimbus.java:546)
> ~[storm-core-0.10.0.2.3.2.0-2950.jar:0.10.0.2.3.2.0-2950]
> at
> backtype.storm.utils.NimbusClient.getConfiguredClientAs(NimbusClient.java:68)
> [storm-core-0.10.0.2.3.2.0-2950.jar:0.10.0.2.3.2.0-2950]
> at backtype.storm.StormSubmitter.submitTopologyAs(StormSubmitter.java:225)
> [storm-core-0.10.0.2.3.2.0-2950.jar:0.10.0.2.3.2.0-2950]
> at backtype.storm.StormSubmitter.submitTopology(StormSubmitter.java:271)
> [storm-core-0.10.0.2.3.2.0-2950.jar:0.10.0.2.3.2.0-2950]
> at backtype.storm.StormSubmitter.submitTopology(StormSubmitter.java:157)
> [storm-core-0.10.0.2.3.2.0-2950.jar:0.10.0.2.3.2.0-2950]
> at storm.starter.WordCountTopology.main(WordCountTopology.java:77)
> [wordCount.jar:?]
> Exception in thread "main" java.lang.RuntimeException: Could not find leader
> nimbus from seed hosts [localhost]. Did you specify a valid list of nimbus
> hosts for config nimbus.seeds
> at
> backtype.storm.utils.NimbusClient.getConfiguredClientAs(NimbusClient.java:90)
> at backtype.storm.StormSubmitter.submitTopologyAs(StormSubmitter.java:225)
> at backtype.storm.StormSubmitter.submitTopology(StormSubmitter.java:271)
> at backtype.storm.StormSubmitter.submitTopology(StormSubmitter.java:157)
> at storm.starter.WordCountTopology.main(WordCountTopology.java:77)
> stdout: /var/lib/ambari-agent/data/output-729.txt
> 2016-06-15 06:24:45,500 - Using hadoop conf dir:
> /usr/hdp/current/hadoop-client/conf
> 2016-06-15 06:24:45,503 - checked_call['hostid'] {}
> 2016-06-15 06:24:45,516 - checked_call returned (0, '16ac396c')
> 2016-06-15 06:24:45,517 - File['/tmp/wordCount.jar'] {'owner': 'storm',
> 'content': StaticFile('wordCount.jar')}
> 2016-06-15 06:24:45,526 - Execute['storm jar /tmp/wordCount.jar
> storm.starter.WordCountTopology WordCountid16ac396c_date241516']
> {'logoutput': True, 'path': ['/usr/hdp/current/storm-client/bin'], 'user':
> 'storm'}
> 951 [main] INFO b.s.u.Utils - Using defaults.yaml from resources
> 1088 [main] INFO b.s.u.Utils - Using storm.yaml from resources
> 1184 [main] INFO b.s.u.Utils - Using defaults.yaml from resources
> 1240 [main] INFO b.s.u.Utils - Using storm.yaml from resources
> 1285 [main] INFO b.s.StormSubmitter - Generated ZooKeeper secret payload for
> MD5-digest: -7463484230184273671:-7128399429695564111
> 1287 [main] INFO b.s.s.a.AuthUtils - Got AutoCreds []
> 1292 [main] WARN b.s.u.NimbusClient - Using deprecated config nimbus.host
> for backward compatibility. Please update your storm.yaml so it only has
> config nimbus.seeds
> 1352 [main] INFO b.s.u.StormBoundedExponentialBackoffRetry - The
> baseSleepTimeMs [2000] the maxSleepTimeMs [60000] the maxRetries [5]
> 1615 [main] INFO o.a.s.z.Login - successfully logged in.
> 1650 [main] ERROR o.a.t.t.TSaslTransport - SASL negotiation failure
> javax.security.sasl.SaslException: GSS initiate failed
> at
> com.sun.security.sasl.gsskerb.GssKrb5Client.evaluateChallenge(GssKrb5Client.java:212)
> ~[?:1.7.0_67]
> at
> org.apache.thrift7.transport.TSaslClientTransport.handleSaslStartMessage(TSaslClientTransport.java:94)
> ~[storm-core-0.10.0.2.3.2.0-2950.jar:0.10.0.2.3.2.0-2950]
> at org.apache.thrift7.transport.TSaslTransport.open(TSaslTransport.java:271)
> [storm-core-0.10.0.2.3.2.0-2950.jar:0.10.0.2.3.2.0-2950]
> at
> org.apache.thrift7.transport.TSaslClientTransport.open(TSaslClientTransport.java:37)
> [storm-core-0.10.0.2.3.2.0-2950.jar:0.10.0.2.3.2.0-2950]
> at
> backtype.storm.security.auth.kerberos.KerberosSaslTransportPlugin$1.run(KerberosSaslTransportPlugin.java:145)
> [storm-core-0.10.0.2.3.2.0-2950.jar:0.10.0.2.3.2.0-2950]
> at
> backtype.storm.security.auth.kerberos.KerberosSaslTransportPlugin$1.run(KerberosSaslTransportPlugin.java:141)
> [storm-core-0.10.0.2.3.2.0-2950.jar:0.10.0.2.3.2.0-2950]
> at java.security.AccessController.doPrivileged(Native Method) ~[?:1.7.0_67]
> at javax.security.auth.Subject.doAs(Subject.java:415) [?:1.7.0_67]
> at
> backtype.storm.security.auth.kerberos.KerberosSaslTransportPlugin.connect(KerberosSaslTransportPlugin.java:140)
> [storm-core-0.10.0.2.3.2.0-2950.jar:0.10.0.2.3.2.0-2950]
> at
> backtype.storm.security.auth.TBackoffConnect.doConnectWithRetry(TBackoffConnect.java:48)
> [storm-core-0.10.0.2.3.2.0-2950.jar:0.10.0.2.3.2.0-2950]
> at backtype.storm.security.auth.ThriftClient.reconnect(ThriftClient.java:103)
> [storm-core-0.10.0.2.3.2.0-2950.jar:0.10.0.2.3.2.0-2950]
> at backtype.storm.security.auth.ThriftClient.<init>(ThriftClient.java:72)
> [storm-core-0.10.0.2.3.2.0-2950.jar:0.10.0.2.3.2.0-2950]
> at backtype.storm.utils.NimbusClient.<init>(NimbusClient.java:104)
> [storm-core-0.10.0.2.3.2.0-2950.jar:0.10.0.2.3.2.0-2950]
> at backtype.storm.utils.NimbusClient.<init>(NimbusClient.java:95)
> [storm-core-0.10.0.2.3.2.0-2950.jar:0.10.0.2.3.2.0-2950]
> at
> backtype.storm.utils.NimbusClient.getConfiguredClientAs(NimbusClient.java:67)
> [storm-core-0.10.0.2.3.2.0-2950.jar:0.10.0.2.3.2.0-2950]
> at backtype.storm.StormSubmitter.submitTopologyAs(StormSubmitter.java:225)
> [storm-core-0.10.0.2.3.2.0-2950.jar:0.10.0.2.3.2.0-2950]
> at backtype.storm.StormSubmitter.submitTopology(StormSubmitter.java:271)
> [storm-core-0.10.0.2.3.2.0-2950.jar:0.10.0.2.3.2.0-2950]
> at backtype.storm.StormSubmitter.submitTopology(StormSubmitter.java:157)
> [storm-core-0.10.0.2.3.2.0-2950.jar:0.10.0.2.3.2.0-2950]
> at storm.starter.WordCountTopology.main(WordCountTopology.java:77)
> [wordCount.jar:?]
> Caused by: org.ietf.jgss.GSSException: No valid credentials provided
> (Mechanism level: Server not found in Kerberos database (7) - UNKNOWN_SERVER)
> at sun.security.jgss.krb5.Krb5Context.initSecContext(Krb5Context.java:710)
> ~[?:1.7.0_67]
> at sun.security.jgss.GSSContextImpl.initSecContext(GSSContextImpl.java:248)
> ~[?:1.7.0_67]
> at sun.security.jgss.GSSContextImpl.initSecContext(GSSContextImpl.java:179)
> ~[?:1.7.0_67]
> at
> com.sun.security.sasl.gsskerb.GssKrb5Client.evaluateChallenge(GssKrb5Client.java:193)
> ~[?:1.7.0_67]
> ... 18 more
> Caused by: sun.security.krb5.KrbException: Server not found in Kerberos
> database (7) - UNKNOWN_SERVER
> at sun.security.krb5.KrbTgsRep.<init>(KrbTgsRep.java:73) ~[?:1.7.0_67]
> at sun.security.krb5.KrbTgsReq.getReply(KrbTgsReq.java:192) ~[?:1.7.0_67]
> at sun.security.krb5.KrbTgsReq.sendAndGetCreds(KrbTgsReq.java:203)
> ~[?:1.7.0_67]
> at
> sun.security.krb5.internal.CredentialsUtil.serviceCreds(CredentialsUtil.java:309)
> ~[?:1.7.0_67]
> at
> sun.security.krb5.internal.CredentialsUtil.acquireServiceCreds(CredentialsUtil.java:115)
> ~[?:1.7.0_67]
> at sun.security.krb5.Credentials.acquireServiceCreds(Credentials.java:454)
> ~[?:1.7.0_67]
> at sun.security.jgss.krb5.Krb5Context.initSecContext(Krb5Context.java:641)
> ~[?:1.7.0_67]
> at sun.security.jgss.GSSContextImpl.initSecContext(GSSContextImpl.java:248)
> ~[?:1.7.0_67]
> at sun.security.jgss.GSSContextImpl.initSecContext(GSSContextImpl.java:179)
> ~[?:1.7.0_67]
> at
> com.sun.security.sasl.gsskerb.GssKrb5Client.evaluateChallenge(GssKrb5Client.java:193)
> ~[?:1.7.0_67]
> ... 18 more
> Caused by: sun.security.krb5.Asn1Exception: Identifier doesn't match expected
> value (906)
> at sun.security.krb5.internal.KDCRep.init(KDCRep.java:143) ~[?:1.7.0_67]
> at sun.security.krb5.internal.TGSRep.init(TGSRep.java:66) ~[?:1.7.0_67]
> at sun.security.krb5.internal.TGSRep.<init>(TGSRep.java:61) ~[?:1.7.0_67]
> at sun.security.krb5.KrbTgsRep.<init>(KrbTgsRep.java:55) ~[?:1.7.0_67]
> at sun.security.krb5.KrbTgsReq.getReply(KrbTgsReq.java:192) ~[?:1.7.0_67]
> at sun.security.krb5.KrbTgsReq.sendAndGetCreds(KrbTgsReq.java:203)
> ~[?:1.7.0_67]
> at
> sun.security.krb5.internal.CredentialsUtil.serviceCreds(CredentialsUtil.java:309)
> ~[?:1.7.0_67]
> at
> sun.security.krb5.internal.CredentialsUtil.acquireServiceCreds(CredentialsUtil.java:115)
> ~[?:1.7.0_67]
> at sun.security.krb5.Credentials.acquireServiceCreds(Credentials.java:454)
> ~[?:1.7.0_67]
> at sun.security.jgss.krb5.Krb5Context.initSecContext(Krb5Context.java:641)
> ~[?:1.7.0_67]
> at sun.security.jgss.GSSContextImpl.initSecContext(GSSContextImpl.java:248)
> ~[?:1.7.0_67]
> at sun.security.jgss.GSSContextImpl.initSecContext(GSSContextImpl.java:179)
> ~[?:1.7.0_67]
> at
> com.sun.security.sasl.gsskerb.GssKrb5Client.evaluateChallenge(GssKrb5Client.java:193)
> ~[?:1.7.0_67]
> ... 18 more
> 1666 [main] ERROR b.s.s.a.k.KerberosSaslTransportPlugin - Client failed to
> open SaslClientTransport to interact with a server during session initiation:
> org.apache.thrift7.transport.TTransportException: GSS initiate failed
> org.apache.thrift7.transport.TTransportException: GSS initiate failed
> at
> org.apache.thrift7.transport.TSaslTransport.sendAndThrowMessage(TSaslTransport.java:232)
> ~[storm-core-0.10.0.2.3.2.0-2950.jar:0.10.0.2.3.2.0-2950]
> at org.apache.thrift7.transport.TSaslTransport.open(TSaslTransport.java:316)
> ~[storm-core-0.10.0.2.3.2.0-2950.jar:0.10.0.2.3.2.0-2950]
> at
> org.apache.thrift7.transport.TSaslClientTransport.open(TSaslClientTransport.java:37)
> ~[storm-core-0.10.0.2.3.2.0-2950.jar:0.10.0.2.3.2.0-2950]
> at
> backtype.storm.security.auth.kerberos.KerberosSaslTransportPlugin$1.run(KerberosSaslTransportPlugin.java:145)
> [storm-core-0.10.0.2.3.2.0-2950.jar:0.10.0.2.3.2.0-2950]
> at
> backtype.storm.security.auth.kerberos.KerberosSaslTransportPlugin$1.run(KerberosSaslTransportPlugin.java:141)
> [storm-core-0.10.0.2.3.2.0-2950.jar:0.10.0.2.3.2.0-2950]
> at java.security.AccessController.doPrivileged(Native Method) ~[?:1.7.0_67]
> at javax.security.auth.Subject.doAs(Subject.java:415) [?:1.7.0_67]
> at
> backtype.storm.security.auth.kerberos.KerberosSaslTransportPlugin.connect(KerberosSaslTransportPlugin.java:140)
> [storm-core-0.10.0.2.3.2.0-2950.jar:0.10.0.2.3.2.0-2950]
> at
> backtype.storm.security.auth.TBackoffConnect.doConnectWithRetry(TBackoffConnect.java:48)
> [storm-core-0.10.0.2.3.2.0-2950.jar:0.10.0.2.3.2.0-2950]
> at backtype.storm.security.auth.ThriftClient.reconnect(ThriftClient.java:103)
> [storm-core-0.10.0.2.3.2.0-2950.jar:0.10.0.2.3.2.0-2950]
> at backtype.storm.security.auth.ThriftClient.<init>(ThriftClient.java:72)
> [storm-core-0.10.0.2.3.2.0-2950.jar:0.10.0.2.3.2.0-2950]
> at backtype.storm.utils.NimbusClient.<init>(NimbusClient.java:104)
> [storm-core-0.10.0.2.3.2.0-2950.jar:0.10.0.2.3.2.0-2950]
> at backtype.storm.utils.NimbusClient.<init>(NimbusClient.java:95)
> [storm-core-0.10.0.2.3.2.0-2950.jar:0.10.0.2.3.2.0-2950]
> at
> backtype.storm.utils.NimbusClient.getConfiguredClientAs(NimbusClient.java:67)
> [storm-core-0.10.0.2.3.2.0-2950.jar:0.10.0.2.3.2.0-2950]
> at backtype.storm.StormSubmitter.submitTopologyAs(StormSubmitter.java:225)
> [storm-core-0.10.0.2.3.2.0-2950.jar:0.10.0.2.3.2.0-2950]
> at backtype.storm.StormSubmitter.submitTopology(StormSubmitter.java:271)
> [storm-core-0.10.0.2.3.2.0-2950.jar:0.10.0.2.3.2.0-2950]
> at backtype.storm.StormSubmitter.submitTopology(StormSubmitter.java:157)
> [storm-core-0.10.0.2.3.2.0-2950.jar:0.10.0.2.3.2.0-2950]
> at storm.starter.WordCountTopology.main(WordCountTopology.java:77)
> [wordCount.jar:?]
> 1675 [main] WARN b.s.u.NimbusClient - Ignoring exception while trying to get
> leader nimbus info from localhost. will retry with a different seed host.
> org.apache.thrift7.transport.TTransportException: SASL authentication not
> complete
> at org.apache.thrift7.transport.TSaslTransport.write(TSaslTransport.java:474)
> ~[storm-core-0.10.0.2.3.2.0-2950.jar:0.10.0.2.3.2.0-2950]
> at
> org.apache.thrift7.transport.TSaslClientTransport.write(TSaslClientTransport.java:37)
> ~[storm-core-0.10.0.2.3.2.0-2950.jar:0.10.0.2.3.2.0-2950]
> at
> org.apache.thrift7.protocol.TBinaryProtocol.writeI32(TBinaryProtocol.java:178)
> ~[storm-core-0.10.0.2.3.2.0-2950.jar:0.10.0.2.3.2.0-2950]
> at
> org.apache.thrift7.protocol.TBinaryProtocol.writeMessageBegin(TBinaryProtocol.java:106)
> ~[storm-core-0.10.0.2.3.2.0-2950.jar:0.10.0.2.3.2.0-2950]
> at org.apache.thrift7.TServiceClient.sendBase(TServiceClient.java:62)
> ~[storm-core-0.10.0.2.3.2.0-2950.jar:0.10.0.2.3.2.0-2950]
> at
> backtype.storm.generated.Nimbus$Client.send_getClusterInfo(Nimbus.java:553)
> ~[storm-core-0.10.0.2.3.2.0-2950.jar:0.10.0.2.3.2.0-2950]
> at backtype.storm.generated.Nimbus$Client.getClusterInfo(Nimbus.java:546)
> ~[storm-core-0.10.0.2.3.2.0-2950.jar:0.10.0.2.3.2.0-2950]
> at
> backtype.storm.utils.NimbusClient.getConfiguredClientAs(NimbusClient.java:68)
> [storm-core-0.10.0.2.3.2.0-2950.jar:0.10.0.2.3.2.0-2950]
> at backtype.storm.StormSubmitter.submitTopologyAs(StormSubmitter.java:225)
> [storm-core-0.10.0.2.3.2.0-2950.jar:0.10.0.2.3.2.0-2950]
> at backtype.storm.StormSubmitter.submitTopology(StormSubmitter.java:271)
> [storm-core-0.10.0.2.3.2.0-2950.jar:0.10.0.2.3.2.0-2950]
> at backtype.storm.StormSubmitter.submitTopology(StormSubmitter.java:157)
> [storm-core-0.10.0.2.3.2.0-2950.jar:0.10.0.2.3.2.0-2950]
> at storm.starter.WordCountTopology.main(WordCountTopology.java:77)
> [wordCount.jar:?]
> Exception in thread "main" java.lang.RuntimeException: Could not find leader
> nimbus from seed hosts [localhost]. Did you specify a valid list of nimbus
> hosts for config nimbus.seeds
> at
> backtype.storm.utils.NimbusClient.getConfiguredClientAs(NimbusClient.java:90)
> at backtype.storm.StormSubmitter.submitTopologyAs(StormSubmitter.java:225)
> at backtype.storm.StormSubmitter.submitTopology(StormSubmitter.java:271)
> at backtype.storm.StormSubmitter.submitTopology(StormSubmitter.java:157)
> at storm.starter.WordCountTopology.main(WordCountTopology.java:77)
> Command failed after 1 tries
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
