Re: Review Request 48735: Storm service check failed after Ambari upgrade
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/48735/#review137810 --- Ship it! Ship It! - Sriharsha Chintalapani On June 15, 2016, 4:59 p.m., Dmitro Lisnichenko wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/48735/ > --- > > (Updated June 15, 2016, 4:59 p.m.) > > > Review request for Ambari, Jonathan Hurley and Nate Cole. > > > Bugs: AMBARI-17258 > https://issues.apache.org/jira/browse/AMBARI-17258 > > > Repository: ambari > > > Description > --- > > ambari-server-2.4.0.0-665.x86_64 > ambari-server --hash > > *Steps* > # Deploy HDP-2.3.2.0 cluster with Ambari 2.1.2 (secure, non-HA cluster) > # Upgrade Ambari to 2.4.0.0-665 > # Stop and Start all services > # Run service check for Storm > > *Result* > Failure as below: > {code} > stderr: /var/lib/ambari-agent/data/errors-729.txt > > Traceback (most recent call last): > File > "/var/lib/ambari-agent/cache/common-services/STORM/0.9.1/package/scripts/service_check.py", > line 79, in > ServiceCheck().execute() > File > "/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py", > line 257, in execute > method(env) > File > "/var/lib/ambari-agent/cache/common-services/STORM/0.9.1/package/scripts/service_check.py", > line 70, in service_check > user=params.storm_user > File "/usr/lib/python2.6/site-packages/resource_management/core/base.py", > line 155, in __init__ > self.env.run() > File > "/usr/lib/python2.6/site-packages/resource_management/core/environment.py", > line 160, in run > self.run_action(resource, action) > File > "/usr/lib/python2.6/site-packages/resource_management/core/environment.py", > line 124, in run_action > provider_action() > File > "/usr/lib/python2.6/site-packages/resource_management/core/providers/system.py", > line 273, in action_run > tries=self.resource.tries, try_sleep=self.resource.try_sleep) > File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", > line 70, in inner > result = function(command, **kwargs) > File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", > line 92, in checked_call > tries=tries, try_sleep=try_sleep) > File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", > line 140, in _call_wrapper > result = _call(command, **kwargs_copy) > File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", > line 293, in _call > raise Fail(err_msg) > resource_management.core.exceptions.Fail: Execution of 'storm jar > /tmp/wordCount.jar storm.starter.WordCountTopology > WordCountid16ac396c_date241516' returned 1. 951 [main] INFO b.s.u.Utils - > Using defaults.yaml from resources > 1088 [main] INFO b.s.u.Utils - Using storm.yaml from resources > 1184 [main] INFO b.s.u.Utils - Using defaults.yaml from resources > 1240 [main] INFO b.s.u.Utils - Using storm.yaml from resources > 1285 [main] INFO b.s.StormSubmitter - Generated ZooKeeper secret payload for > MD5-digest: -7463484230184273671:-7128399429695564111 > 1287 [main] INFO b.s.s.a.AuthUtils - Got AutoCreds [] > 1292 [main] WARN b.s.u.NimbusClient - Using deprecated config nimbus.host > for backward compatibility. Please update your storm.yaml so it only has > config nimbus.seeds > 1352 [main] INFO b.s.u.StormBoundedExponentialBackoffRetry - The > baseSleepTimeMs [2000] the maxSleepTimeMs [6] the maxRetries [5] > 1615 [main] INFO o.a.s.z.Login - successfully logged in. > 1650 [main] ERROR o.a.t.t.TSaslTransport - SASL negotiation failure > javax.security.sasl.SaslException: GSS initiate failed > at > com.sun.security.sasl.gsskerb.GssKrb5Client.evaluateChallenge(GssKrb5Client.java:212) > ~[?:1.7.0_67] > at > org.apache.thrift7.transport.TSaslClientTransport.handleSaslStartMessage(TSaslClientTransport.java:94) > ~[storm-core-0.10.0.2.3.2.0-2950.jar:0.10.0.2.3.2.0-2950] > at org.apache.thrift7.transport.TSaslTransport.open(TSaslTransport.java:271) > [storm-core-0.10.0.2.3.2.0-2950.jar:0.10.0.2.3.2.0-2950] > at > org.apache.thrift7.transport.TSaslClientTransport.open(TSaslClientTransport.java:37) > [storm-core-0.10.0.2.3.2.0-2950.jar:0.10.0.2.3.2.0-2950] > at > backtype.storm.security.auth.kerberos.KerberosSaslTransportPlugin$1.run(KerberosSaslTransportPlugin.java:145) > [storm-core-0.10.0.2.3.2.0-2950.jar:0.10.0.2.3.2.0-2950] > at > backtype.storm.security.auth.kerberos.KerberosSaslTransportPlugin$1.run(KerberosSaslTransportPlugin.java:141) > [storm-core-0.10.0.2.3.2.0-2950.jar:0.10.0.2.3.2.0-2950] > at java.security.AccessController.doPrivileged(Native Method) ~[?:1.7.0_67] > at javax.security.auth.Subject.doAs(Subject.java:415) [?:1.7.0_67] > at >
Re: Review Request 48735: Storm service check failed after Ambari upgrade
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/48735/#review137808 --- Please include Sriharsha Chintalapani in the code review. - Alejandro Fernandez On June 15, 2016, 4:59 p.m., Dmitro Lisnichenko wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/48735/ > --- > > (Updated June 15, 2016, 4:59 p.m.) > > > Review request for Ambari, Jonathan Hurley and Nate Cole. > > > Bugs: AMBARI-17258 > https://issues.apache.org/jira/browse/AMBARI-17258 > > > Repository: ambari > > > Description > --- > > ambari-server-2.4.0.0-665.x86_64 > ambari-server --hash > > *Steps* > # Deploy HDP-2.3.2.0 cluster with Ambari 2.1.2 (secure, non-HA cluster) > # Upgrade Ambari to 2.4.0.0-665 > # Stop and Start all services > # Run service check for Storm > > *Result* > Failure as below: > {code} > stderr: /var/lib/ambari-agent/data/errors-729.txt > > Traceback (most recent call last): > File > "/var/lib/ambari-agent/cache/common-services/STORM/0.9.1/package/scripts/service_check.py", > line 79, in > ServiceCheck().execute() > File > "/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py", > line 257, in execute > method(env) > File > "/var/lib/ambari-agent/cache/common-services/STORM/0.9.1/package/scripts/service_check.py", > line 70, in service_check > user=params.storm_user > File "/usr/lib/python2.6/site-packages/resource_management/core/base.py", > line 155, in __init__ > self.env.run() > File > "/usr/lib/python2.6/site-packages/resource_management/core/environment.py", > line 160, in run > self.run_action(resource, action) > File > "/usr/lib/python2.6/site-packages/resource_management/core/environment.py", > line 124, in run_action > provider_action() > File > "/usr/lib/python2.6/site-packages/resource_management/core/providers/system.py", > line 273, in action_run > tries=self.resource.tries, try_sleep=self.resource.try_sleep) > File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", > line 70, in inner > result = function(command, **kwargs) > File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", > line 92, in checked_call > tries=tries, try_sleep=try_sleep) > File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", > line 140, in _call_wrapper > result = _call(command, **kwargs_copy) > File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", > line 293, in _call > raise Fail(err_msg) > resource_management.core.exceptions.Fail: Execution of 'storm jar > /tmp/wordCount.jar storm.starter.WordCountTopology > WordCountid16ac396c_date241516' returned 1. 951 [main] INFO b.s.u.Utils - > Using defaults.yaml from resources > 1088 [main] INFO b.s.u.Utils - Using storm.yaml from resources > 1184 [main] INFO b.s.u.Utils - Using defaults.yaml from resources > 1240 [main] INFO b.s.u.Utils - Using storm.yaml from resources > 1285 [main] INFO b.s.StormSubmitter - Generated ZooKeeper secret payload for > MD5-digest: -7463484230184273671:-7128399429695564111 > 1287 [main] INFO b.s.s.a.AuthUtils - Got AutoCreds [] > 1292 [main] WARN b.s.u.NimbusClient - Using deprecated config nimbus.host > for backward compatibility. Please update your storm.yaml so it only has > config nimbus.seeds > 1352 [main] INFO b.s.u.StormBoundedExponentialBackoffRetry - The > baseSleepTimeMs [2000] the maxSleepTimeMs [6] the maxRetries [5] > 1615 [main] INFO o.a.s.z.Login - successfully logged in. > 1650 [main] ERROR o.a.t.t.TSaslTransport - SASL negotiation failure > javax.security.sasl.SaslException: GSS initiate failed > at > com.sun.security.sasl.gsskerb.GssKrb5Client.evaluateChallenge(GssKrb5Client.java:212) > ~[?:1.7.0_67] > at > org.apache.thrift7.transport.TSaslClientTransport.handleSaslStartMessage(TSaslClientTransport.java:94) > ~[storm-core-0.10.0.2.3.2.0-2950.jar:0.10.0.2.3.2.0-2950] > at org.apache.thrift7.transport.TSaslTransport.open(TSaslTransport.java:271) > [storm-core-0.10.0.2.3.2.0-2950.jar:0.10.0.2.3.2.0-2950] > at > org.apache.thrift7.transport.TSaslClientTransport.open(TSaslClientTransport.java:37) > [storm-core-0.10.0.2.3.2.0-2950.jar:0.10.0.2.3.2.0-2950] > at > backtype.storm.security.auth.kerberos.KerberosSaslTransportPlugin$1.run(KerberosSaslTransportPlugin.java:145) > [storm-core-0.10.0.2.3.2.0-2950.jar:0.10.0.2.3.2.0-2950] > at > backtype.storm.security.auth.kerberos.KerberosSaslTransportPlugin$1.run(KerberosSaslTransportPlugin.java:141) > [storm-core-0.10.0.2.3.2.0-2950.jar:0.10.0.2.3.2.0-2950] > at java.security.AccessController.doPrivileged(Native Method) ~[?:1.7.0_67] > at javax.security.auth.Subject.doAs(Subject.java:415) [?:1.7.0_67] >
Review Request 48735: Storm service check failed after Ambari upgrade
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/48735/ --- Review request for Ambari, Jonathan Hurley and Nate Cole. Bugs: AMBARI-17258 https://issues.apache.org/jira/browse/AMBARI-17258 Repository: ambari Description --- ambari-server-2.4.0.0-665.x86_64 ambari-server --hash *Steps* # Deploy HDP-2.3.2.0 cluster with Ambari 2.1.2 (secure, non-HA cluster) # Upgrade Ambari to 2.4.0.0-665 # Stop and Start all services # Run service check for Storm *Result* Failure as below: {code} stderr: /var/lib/ambari-agent/data/errors-729.txt Traceback (most recent call last): File "/var/lib/ambari-agent/cache/common-services/STORM/0.9.1/package/scripts/service_check.py", line 79, in ServiceCheck().execute() File "/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py", line 257, in execute method(env) File "/var/lib/ambari-agent/cache/common-services/STORM/0.9.1/package/scripts/service_check.py", line 70, in service_check user=params.storm_user File "/usr/lib/python2.6/site-packages/resource_management/core/base.py", line 155, in __init__ self.env.run() File "/usr/lib/python2.6/site-packages/resource_management/core/environment.py", line 160, in run self.run_action(resource, action) File "/usr/lib/python2.6/site-packages/resource_management/core/environment.py", line 124, in run_action provider_action() File "/usr/lib/python2.6/site-packages/resource_management/core/providers/system.py", line 273, in action_run tries=self.resource.tries, try_sleep=self.resource.try_sleep) File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 70, in inner result = function(command, **kwargs) File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 92, in checked_call tries=tries, try_sleep=try_sleep) File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 140, in _call_wrapper result = _call(command, **kwargs_copy) File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 293, in _call raise Fail(err_msg) resource_management.core.exceptions.Fail: Execution of 'storm jar /tmp/wordCount.jar storm.starter.WordCountTopology WordCountid16ac396c_date241516' returned 1. 951 [main] INFO b.s.u.Utils - Using defaults.yaml from resources 1088 [main] INFO b.s.u.Utils - Using storm.yaml from resources 1184 [main] INFO b.s.u.Utils - Using defaults.yaml from resources 1240 [main] INFO b.s.u.Utils - Using storm.yaml from resources 1285 [main] INFO b.s.StormSubmitter - Generated ZooKeeper secret payload for MD5-digest: -7463484230184273671:-7128399429695564111 1287 [main] INFO b.s.s.a.AuthUtils - Got AutoCreds [] 1292 [main] WARN b.s.u.NimbusClient - Using deprecated config nimbus.host for backward compatibility. Please update your storm.yaml so it only has config nimbus.seeds 1352 [main] INFO b.s.u.StormBoundedExponentialBackoffRetry - The baseSleepTimeMs [2000] the maxSleepTimeMs [6] the maxRetries [5] 1615 [main] INFO o.a.s.z.Login - successfully logged in. 1650 [main] ERROR o.a.t.t.TSaslTransport - SASL negotiation failure javax.security.sasl.SaslException: GSS initiate failed at com.sun.security.sasl.gsskerb.GssKrb5Client.evaluateChallenge(GssKrb5Client.java:212) ~[?:1.7.0_67] at org.apache.thrift7.transport.TSaslClientTransport.handleSaslStartMessage(TSaslClientTransport.java:94) ~[storm-core-0.10.0.2.3.2.0-2950.jar:0.10.0.2.3.2.0-2950] at org.apache.thrift7.transport.TSaslTransport.open(TSaslTransport.java:271) [storm-core-0.10.0.2.3.2.0-2950.jar:0.10.0.2.3.2.0-2950] at org.apache.thrift7.transport.TSaslClientTransport.open(TSaslClientTransport.java:37) [storm-core-0.10.0.2.3.2.0-2950.jar:0.10.0.2.3.2.0-2950] at backtype.storm.security.auth.kerberos.KerberosSaslTransportPlugin$1.run(KerberosSaslTransportPlugin.java:145) [storm-core-0.10.0.2.3.2.0-2950.jar:0.10.0.2.3.2.0-2950] at backtype.storm.security.auth.kerberos.KerberosSaslTransportPlugin$1.run(KerberosSaslTransportPlugin.java:141) [storm-core-0.10.0.2.3.2.0-2950.jar:0.10.0.2.3.2.0-2950] at java.security.AccessController.doPrivileged(Native Method) ~[?:1.7.0_67] at javax.security.auth.Subject.doAs(Subject.java:415) [?:1.7.0_67] at backtype.storm.security.auth.kerberos.KerberosSaslTransportPlugin.connect(KerberosSaslTransportPlugin.java:140) [storm-core-0.10.0.2.3.2.0-2950.jar:0.10.0.2.3.2.0-2950] at backtype.storm.security.auth.TBackoffConnect.doConnectWithRetry(TBackoffConnect.java:48) [storm-core-0.10.0.2.3.2.0-2950.jar:0.10.0.2.3.2.0-2950] at backtype.storm.security.auth.ThriftClient.reconnect(ThriftClient.java:103) [storm-core-0.10.0.2.3.2.0-2950.jar:0.10.0.2.3.2.0-2950] at backtype.storm.security.auth.ThriftClient.(ThriftClient.java:72) [storm-core-0.10.0.2.3.2.0-2950.jar:0.10.0.2.3.2.0-2950] at