----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/27549/#review59686 -----------------------------------------------------------
Ship it! Ship It! - Mahadev Konar On Nov. 4, 2014, 12:20 a.m., Alejandro Fernandez wrote: > > ----------------------------------------------------------- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/27549/ > ----------------------------------------------------------- > > (Updated Nov. 4, 2014, 12:20 a.m.) > > > Review request for Ambari, Mahadev Konar, Sumit Mohanty, and Sid Wagle. > > > Bugs: AMBARI-8131 > https://issues.apache.org/jira/browse/AMBARI-8131 > > > Repository: ambari > > > Description > ------- > > Create a secure cluster with Storm, kinit as ambari-qa, then attempt to run > the following command to add a topology, which fails, > > ``` > [root@alejandrohdp22-1 tmp]# /usr/hdp/current/storm-client/bin/storm -c > java.security.auth.login.config=/etc/storm/conf/client_jaas.conf jar > /usr/hdp/current/storm-client/contrib/storm-starter/storm-starter-*-jar-with-dependencies.jar > storm.starter.ExclamationTopology topo1 > Running: java -client > -Dstorm.options=java.security.auth.login.config=/etc/storm/conf/client_jaas.conf > -Dstorm.home=/usr/hdp/2.2.0.0-1744/storm > -Dstorm.log.dir=/usr/hdp/2.2.0.0-1744/storm/logs > -Djava.library.path=/usr/local/lib:/opt/local/lib:/usr/lib:/usr/hdp/current/storm-client/lib > -Dstorm.conf.file= -cp > /usr/hdp/2.2.0.0-1744/storm/lib/asm-4.0.jar:/usr/hdp/2.2.0.0-1744/storm/lib/log4j-over-slf4j-1.6.6.jar:/usr/hdp/2.2.0.0-1744/storm/lib/jetty-security-7.6.13.v20130916.jar:/usr/hdp/2.2.0.0-1744/storm/lib/gson-2.2.4.jar:/usr/hdp/2.2.0.0-1744/storm/lib/gmetric4j-1.0.7.jar:/usr/hdp/2.2.0.0-1744/storm/lib/tools.namespace-0.2.4.jar:/usr/hdp/2.2.0.0-1744/storm/lib/zookeeper.jar:/usr/hdp/2.2.0.0-1744/storm/lib/clj-stacktrace-0.2.4.jar:/usr/hdp/2.2.0.0-1744/storm/lib/snakeyaml-1.11.jar:/usr/hdp/2.2.0.0-1744/storm/lib/crypto-random-1.2.0.jar:/usr/hdp/2.2.0.0-1744/storm/lib/ranger-plugins-cred-0.4.0.2.2.0.0-1744.jar:/usr/hdp/2.2.0.0-1744/storm/lib/commons-lang-2.5.jar:/usr/hdp/2.2.0.0-1744 /storm/lib/ring-jetty-adapter-1.3.0.jar:/usr/hdp/2.2.0.0-1744/storm/lib/javax.servlet-2.5.0.v201103041518.jar:/usr/hdp/2.2.0.0-1744/storm/lib/mysql-connector-java.jar:/usr/hdp/2.2.0.0-1744/storm/lib/tools.cli-0.2.4.jar:/usr/hdp/2.2.0.0-1744/storm/lib/chill-java-0.3.5.jar:/usr/hdp/2.2.0.0-1744/storm/lib/jetty-http-7.6.13.v20130916.jar:/usr/hdp/2.2.0.0-1744/storm/lib/eclipselink-2.5.2-M1.jar:/usr/hdp/2.2.0.0-1744/storm/lib/jetty-servlets-7.6.13.v20130916.jar:/usr/hdp/2.2.0.0-1744/storm/lib/clojure-1.5.1.jar:/usr/hdp/2.2.0.0-1744/storm/lib/jetty-continuation-7.6.13.v20130916.jar:/usr/hdp/2.2.0.0-1744/storm/lib/ranger-plugins-impl-0.4.0.2.2.0.0-1744.jar:/usr/hdp/2.2.0.0-1744/storm/lib/jetty-util-7.6.13.v20130916.jar:/usr/hdp/2.2.0.0-1744/storm/lib/tools.logging-0.2.3.jar:/usr/hdp/2.2.0.0-1744/storm/lib/commons-fileupload-1.2.1.jar:/usr/hdp/2.2.0.0-1744/storm/lib/commons-exec-1.1.jar:/usr/hdp/2.2.0.0-1744/storm/lib/kryo-2.21.jar:/usr/hdp/2.2.0.0-1744/storm/lib/compojure-1.1.3.jar:/usr/hd p/2.2.0.0-1744/storm/lib/javax.persistence-2.1.0.jar:/usr/hdp/2.2.0.0-1744/storm/lib/commons-configuration-1.10.jar:/usr/hdp/2.2.0.0-1744/storm/lib/commons-codec-1.6.jar:/usr/hdp/2.2.0.0-1744/storm/lib/servlet-api-2.5.jar:/usr/hdp/2.2.0.0-1744/storm/lib/oncrpc-1.0.7.jar:/usr/hdp/2.2.0.0-1744/storm/lib/hadoop-common-2.6.0.2.2.0.0-1744.jar:/usr/hdp/2.2.0.0-1744/storm/lib/logback-core-1.0.6.jar:/usr/hdp/2.2.0.0-1744/storm/lib/storm-core-0.9.3.2.2.0.0-1744.jar:/usr/hdp/2.2.0.0-1744/storm/lib/disruptor-3.2.1.jar:/usr/hdp/2.2.0.0-1744/storm/lib/clout-1.0.1.jar:/usr/hdp/2.2.0.0-1744/storm/lib/ns-tracker-0.2.2.jar:/usr/hdp/2.2.0.0-1744/storm/lib/json-simple-1.1.jar:/usr/hdp/2.2.0.0-1744/storm/lib/tools.macro-0.1.0.jar:/usr/hdp/2.2.0.0-1744/storm/lib/slf4j-api-1.6.5.jar:/usr/hdp/2.2.0.0-1744/storm/lib/ring-devel-1.3.0.jar:/usr/hdp/2.2.0.0-1744/storm/lib/crypto-equality-1.0.0.jar:/usr/hdp/2.2.0.0-1744/storm/lib/ring-anti-forgery-1.0.0.jar:/usr/hdp/2.2.0.0-1744/storm/lib/jetty-servlet-7.6.13.v 20130916.jar:/usr/hdp/2.2.0.0-1744/storm/lib/objenesis-1.2.jar:/usr/hdp/2.2.0.0-1744/storm/lib/core.incubator-0.1.0.jar:/usr/hdp/2.2.0.0-1744/storm/lib/minlog-1.2.jar:/usr/hdp/2.2.0.0-1744/storm/lib/clj-time-0.4.1.jar:/usr/hdp/2.2.0.0-1744/storm/lib/jetty-client-7.6.13.v20130916.jar:/usr/hdp/2.2.0.0-1744/storm/lib/jetty-server-7.6.13.v20130916.jar:/usr/hdp/2.2.0.0-1744/storm/lib/commons-collections-3.2.1.jar:/usr/hdp/2.2.0.0-1744/storm/lib/reflectasm-1.07-shaded.jar:/usr/hdp/2.2.0.0-1744/storm/lib/hiccup-0.3.6.jar:/usr/hdp/2.2.0.0-1744/storm/lib/guava-11.0.2.jar:/usr/hdp/2.2.0.0-1744/storm/lib/ring-core-1.1.5.jar:/usr/hdp/2.2.0.0-1744/storm/lib/jersey-bundle-1.17.1.jar:/usr/hdp/2.2.0.0-1744/storm/lib/math.numeric-tower-0.0.1.jar:/usr/hdp/2.2.0.0-1744/storm/lib/logback-classic-1.0.6.jar:/usr/hdp/2.2.0.0-1744/storm/lib/ring-servlet-1.3.0.jar:/usr/hdp/2.2.0.0-1744/storm/lib/hadoop-auth-2.4.0.jar:/usr/hdp/2.2.0.0-1744/storm/lib/java.classpath-0.2.2.jar:/usr/hdp/2.2.0.0-1744/storm/lib/co mmons-io-2.4.jar:/usr/hdp/2.2.0.0-1744/storm/lib/ranger-plugins-audit-0.4.0.2.2.0.0-1744.jar:/usr/hdp/2.2.0.0-1744/storm/lib/carbonite-1.4.0.jar:/usr/hdp/2.2.0.0-1744/storm/lib/jetty-io-7.6.13.v20130916.jar:/usr/hdp/2.2.0.0-1744/storm/lib/jgrapht-core-0.9.0.jar:/usr/hdp/2.2.0.0-1744/storm/lib/ranger-storm-plugin-0.4.0.2.2.0.0-1744.jar:/usr/hdp/2.2.0.0-1744/storm/lib/joda-time-2.0.jar:/usr/hdp/2.2.0.0-1744/storm/lib/commons-logging-1.2.jar:/usr/hdp/2.2.0.0-1744/storm/lib/ranger-plugins-common-0.4.0.2.2.0.0-1744.jar:/usr/hdp/current/storm-client/contrib/storm-starter/storm-starter-0.9.3.2.2.0.0-1744-jar-with-dependencies.jar:/usr/hdp/2.2.0.0-1744/storm/conf:/usr/hdp/2.2.0.0-1744/storm/bin -Dstorm.jar=/usr/hdp/current/storm-client/contrib/storm-starter/storm-starter-0.9.3.2.2.0.0-1744-jar-with-dependencies.jar storm.starter.ExclamationTopology topo1 > 773 [main] INFO backtype.storm.StormSubmitter - Generated ZooKeeper secret > payload for MD5-digest: -5766984969082465146:-6900977625499590902 > 788 [main] INFO backtype.storm.security.auth.AuthUtils - Got AutoCreds [] > Exception in thread "main" java.lang.RuntimeException: > java.security.NoSuchAlgorithmException: Error constructing implementation > (algorithm: JavaLoginConfig, provider: SUN, class: > sun.security.provider.ConfigSpiFile) > at > backtype.storm.security.auth.AuthUtils.GetConfiguration(AuthUtils.java:66) > at > backtype.storm.security.auth.ThriftClient.reconnect(ThriftClient.java:82) > at > backtype.storm.security.auth.ThriftClient.<init>(ThriftClient.java:66) > at backtype.storm.utils.NimbusClient.<init>(NimbusClient.java:52) > at > backtype.storm.utils.NimbusClient.getConfiguredClient(NimbusClient.java:36) > at backtype.storm.StormSubmitter.submitTopology(StormSubmitter.java:211) > at > backtype.storm.StormSubmitter.submitTopologyWithProgressBar(StormSubmitter.java:273) > at > backtype.storm.StormSubmitter.submitTopologyWithProgressBar(StormSubmitter.java:254) > at storm.starter.ExclamationTopology.main(ExclamationTopology.java:76) > Caused by: java.security.NoSuchAlgorithmException: Error constructing > implementation (algorithm: JavaLoginConfig, provider: SUN, class: > sun.security.provider.ConfigSpiFile) > at java.security.Provider$Service.newInstance(Provider.java:1259) > at sun.security.jca.GetInstance.getInstance(GetInstance.java:243) > at sun.security.jca.GetInstance.getInstance(GetInstance.java:190) > at > javax.security.auth.login.Configuration.getInstance(Configuration.java:352) > at > backtype.storm.security.auth.AuthUtils.GetConfiguration(AuthUtils.java:64) > ... 8 more > Caused by: java.io.IOException: Configuration Error: > Line 1: expected [{], found [null] > at com.sun.security.auth.login.ConfigFile.match(ConfigFile.java:572) > at > com.sun.security.auth.login.ConfigFile.parseLoginEntry(ConfigFile.java:405) > at > com.sun.security.auth.login.ConfigFile.readConfig(ConfigFile.java:383) > at com.sun.security.auth.login.ConfigFile.init(ConfigFile.java:283) > at com.sun.security.auth.login.ConfigFile.init(ConfigFile.java:166) > at com.sun.security.auth.login.ConfigFile.<init>(ConfigFile.java:124) > at sun.security.provider.ConfigSpiFile$1.run(ConfigSpiFile.java:72) > at sun.security.provider.ConfigSpiFile$1.run(ConfigSpiFile.java:61) > # > at java.security.AccessController.doPrivileged(Native Method) > at sun.security.provider.ConfigSpiFile.<init>(ConfigSpiFile.java:61) > at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) > at > sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57) > at > sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) > at java.lang.reflect.Constructor.newInstance(Constructor.java:526) > at java.security.Provider$Service.newInstance(Provider.java:1254) > ... 12 more > ``` > > > Diffs > ----- > > > ambari-server/src/main/resources/stacks/HDP/2.1/services/STORM/package/scripts/params.py > 32cb60d > > ambari-server/src/main/resources/stacks/HDP/2.1/services/STORM/package/scripts/storm.py > 5699e57 > > ambari-server/src/main/resources/stacks/HDP/2.1/services/STORM/package/templates/client_jaas.conf.j2 > cf78af4 > > ambari-server/src/main/resources/stacks/HDP/2.1/services/STORM/package/templates/storm.yaml.j2 > 82ff239 > > ambari-server/src/main/resources/stacks/HDP/2.1/services/STORM/package/templates/storm_jaas.conf.j2 > 7c650df > > Diff: https://reviews.apache.org/r/27549/diff/ > > > Testing > ------- > > Ran unit tests on ambari-server, > ---------------------------------------------------------------------- > Total run:684 > Total errors:0 > Total failures:0 > OK > > > Reproduced the issue on a live cluster, and was able to get the topology > added. > E.g., > su - ambari-qa > kinit -kt /etc/security/keytabs/smokeuser.headless.keytab > [email protected] > > scp -i ../gce-key > ambari-server/src/main/resources/stacks/HDP/2.1/services/STORM/package/scripts/params.py > [email protected]:/tmp > scp -i ../gce-key > ambari-server/src/main/resources/stacks/HDP/2.1/services/STORM/package/scripts/storm.py > [email protected]:/tmp > scp -i ../gce-key > ambari-server/src/main/resources/stacks/HDP/2.1/services/STORM/package/templates/client_jaas.conf.j2 > [email protected]:/tmp > scp -i ../gce-key > ambari-server/src/main/resources/stacks/HDP/2.1/services/STORM/package/templates/storm.yaml.j2 > [email protected]:/tmp > scp -i ../gce-key > ambari-server/src/main/resources/stacks/HDP/2.1/services/STORM/package/templates/storm_jaas.conf.j2 > [email protected]:/tmp > scp -i ../gce-key > ambari-server/src/main/resources/stacks/HDP/2.1/services/STORM/package/templates/worker-launcher.cfg.j2 > [email protected]:/tmp > > > yes | cp /tmp/params.py > /var/lib/ambari-agent/cache/stacks/HDP/2.1/services/STORM/package/scripts/params.py > yes | cp /tmp/storm.py > /var/lib/ambari-agent/cache/stacks/HDP/2.1/services/STORM/package/scripts/storm.py > yes | cp /tmp/client_jaas.conf.j2 > /var/lib/ambari-agent/cache/stacks/HDP/2.1/services/STORM/package/templates/client_jaas.conf.j2 > yes | cp /tmp/storm.yaml.j2 > /var/lib/ambari-agent/cache/stacks/HDP/2.1/services/STORM/package/templates/storm.yaml.j2 > yes | cp /tmp/storm_jaas.conf.j2 > /var/lib/ambari-agent/cache/stacks/HDP/2.1/services/STORM/package/templates/storm_jaas.conf.j2 > yes | cp /tmp/worker-launcher.cfg.j2 > /var/lib/ambari-agent/cache/stacks/HDP/2.1/services/STORM/package/templates/worker-launcher.cfg.j2 > > ambari-server restart (to reload templates) > > Ensure that client_jaas.conf is in /etc/storm/conf/ . Then run, > /usr/hdp/current/storm-client/bin/storm -c > java.security.auth.login.config=/etc/storm/conf/client_jaas.conf jar > /usr/hdp/current/storm-client/contrib/storm-starter/storm-starter-*-jar-with-dependencies.jar > storm.starter.ExclamationTopology topo1 > > > Thanks, > > Alejandro Fernandez > >
