-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/27549/
-----------------------------------------------------------
Review request for Ambari, Mahadev Konar, Sumit Mohanty, and Sid Wagle.
Bugs: AMBARI-8131
https://issues.apache.org/jira/browse/AMBARI-8131
Repository: ambari
Description
-------
Create a secure cluster with Storm, kinit as ambari-qa, then attempt to run the
following command to add a topology, which fails,
```
[root@alejandrohdp22-1 tmp]# /usr/hdp/current/storm-client/bin/storm -c
java.security.auth.login.config=/etc/storm/conf/client_jaas.conf jar
/usr/hdp/current/storm-client/contrib/storm-starter/storm-starter-*-jar-with-dependencies.jar
storm.starter.ExclamationTopology topo1
Running: java -client
-Dstorm.options=java.security.auth.login.config=/etc/storm/conf/client_jaas.conf
-Dstorm.home=/usr/hdp/2.2.0.0-1744/storm
-Dstorm.log.dir=/usr/hdp/2.2.0.0-1744/storm/logs
-Djava.library.path=/usr/local/lib:/opt/local/lib:/usr/lib:/usr/hdp/current/storm-client/lib
-Dstorm.conf.file= -cp
/usr/hdp/2.2.0.0-1744/storm/lib/asm-4.0.jar:/usr/hdp/2.2.0.0-1744/storm/lib/log4j-over-slf4j-1.6.6.jar:/usr/hdp/2.2.0.0-1744/storm/lib/jetty-security-7.6.13.v20130916.jar:/usr/hdp/2.2.0.0-1744/storm/lib/gson-2.2.4.jar:/usr/hdp/2.2.0.0-1744/storm/lib/gmetric4j-1.0.7.jar:/usr/hdp/2.2.0.0-1744/storm/lib/tools.namespace-0.2.4.jar:/usr/hdp/2.2.0.0-1744/storm/lib/zookeeper.jar:/usr/hdp/2.2.0.0-1744/storm/lib/clj-stacktrace-0.2.4.jar:/usr/hdp/2.2.0.0-1744/storm/lib/snakeyaml-1.11.jar:/usr/hdp/2.2.0.0-1744/storm/lib/crypto-random-1.2.0.jar:/usr/hdp/2.2.0.0-1744/storm/lib/ranger-plugins-cred-0.4.0.2.2.0.0-1744.jar:/usr/hdp/2.2.0.0-1744/storm/lib/commons-lang-2.5.jar:/usr/hdp/2.2.0.0-1744/s
torm/lib/ring-jetty-adapter-1.3.0.jar:/usr/hdp/2.2.0.0-1744/storm/lib/javax.servlet-2.5.0.v201103041518.jar:/usr/hdp/2.2.0.0-1744/storm/lib/mysql-connector-java.jar:/usr/hdp/2.2.0.0-1744/storm/lib/tools.cli-0.2.4.jar:/usr/hdp/2.2.0.0-1744/storm/lib/chill-java-0.3.5.jar:/usr/hdp/2.2.0.0-1744/storm/lib/jetty-http-7.6.13.v20130916.jar:/usr/hdp/2.2.0.0-1744/storm/lib/eclipselink-2.5.2-M1.jar:/usr/hdp/2.2.0.0-1744/storm/lib/jetty-servlets-7.6.13.v20130916.jar:/usr/hdp/2.2.0.0-1744/storm/lib/clojure-1.5.1.jar:/usr/hdp/2.2.0.0-1744/storm/lib/jetty-continuation-7.6.13.v20130916.jar:/usr/hdp/2.2.0.0-1744/storm/lib/ranger-plugins-impl-0.4.0.2.2.0.0-1744.jar:/usr/hdp/2.2.0.0-1744/storm/lib/jetty-util-7.6.13.v20130916.jar:/usr/hdp/2.2.0.0-1744/storm/lib/tools.logging-0.2.3.jar:/usr/hdp/2.2.0.0-1744/storm/lib/commons-fileupload-1.2.1.jar:/usr/hdp/2.2.0.0-1744/storm/lib/commons-exec-1.1.jar:/usr/hdp/2.2.0.0-1744/storm/lib/kryo-2.21.jar:/usr/hdp/2.2.0.0-1744/storm/lib/compojure-1.1.3.jar:/usr/hdp/
2.2.0.0-1744/storm/lib/javax.persistence-2.1.0.jar:/usr/hdp/2.2.0.0-1744/storm/lib/commons-configuration-1.10.jar:/usr/hdp/2.2.0.0-1744/storm/lib/commons-codec-1.6.jar:/usr/hdp/2.2.0.0-1744/storm/lib/servlet-api-2.5.jar:/usr/hdp/2.2.0.0-1744/storm/lib/oncrpc-1.0.7.jar:/usr/hdp/2.2.0.0-1744/storm/lib/hadoop-common-2.6.0.2.2.0.0-1744.jar:/usr/hdp/2.2.0.0-1744/storm/lib/logback-core-1.0.6.jar:/usr/hdp/2.2.0.0-1744/storm/lib/storm-core-0.9.3.2.2.0.0-1744.jar:/usr/hdp/2.2.0.0-1744/storm/lib/disruptor-3.2.1.jar:/usr/hdp/2.2.0.0-1744/storm/lib/clout-1.0.1.jar:/usr/hdp/2.2.0.0-1744/storm/lib/ns-tracker-0.2.2.jar:/usr/hdp/2.2.0.0-1744/storm/lib/json-simple-1.1.jar:/usr/hdp/2.2.0.0-1744/storm/lib/tools.macro-0.1.0.jar:/usr/hdp/2.2.0.0-1744/storm/lib/slf4j-api-1.6.5.jar:/usr/hdp/2.2.0.0-1744/storm/lib/ring-devel-1.3.0.jar:/usr/hdp/2.2.0.0-1744/storm/lib/crypto-equality-1.0.0.jar:/usr/hdp/2.2.0.0-1744/storm/lib/ring-anti-forgery-1.0.0.jar:/usr/hdp/2.2.0.0-1744/storm/lib/jetty-servlet-7.6.13.v20
130916.jar:/usr/hdp/2.2.0.0-1744/storm/lib/objenesis-1.2.jar:/usr/hdp/2.2.0.0-1744/storm/lib/core.incubator-0.1.0.jar:/usr/hdp/2.2.0.0-1744/storm/lib/minlog-1.2.jar:/usr/hdp/2.2.0.0-1744/storm/lib/clj-time-0.4.1.jar:/usr/hdp/2.2.0.0-1744/storm/lib/jetty-client-7.6.13.v20130916.jar:/usr/hdp/2.2.0.0-1744/storm/lib/jetty-server-7.6.13.v20130916.jar:/usr/hdp/2.2.0.0-1744/storm/lib/commons-collections-3.2.1.jar:/usr/hdp/2.2.0.0-1744/storm/lib/reflectasm-1.07-shaded.jar:/usr/hdp/2.2.0.0-1744/storm/lib/hiccup-0.3.6.jar:/usr/hdp/2.2.0.0-1744/storm/lib/guava-11.0.2.jar:/usr/hdp/2.2.0.0-1744/storm/lib/ring-core-1.1.5.jar:/usr/hdp/2.2.0.0-1744/storm/lib/jersey-bundle-1.17.1.jar:/usr/hdp/2.2.0.0-1744/storm/lib/math.numeric-tower-0.0.1.jar:/usr/hdp/2.2.0.0-1744/storm/lib/logback-classic-1.0.6.jar:/usr/hdp/2.2.0.0-1744/storm/lib/ring-servlet-1.3.0.jar:/usr/hdp/2.2.0.0-1744/storm/lib/hadoop-auth-2.4.0.jar:/usr/hdp/2.2.0.0-1744/storm/lib/java.classpath-0.2.2.jar:/usr/hdp/2.2.0.0-1744/storm/lib/comm
ons-io-2.4.jar:/usr/hdp/2.2.0.0-1744/storm/lib/ranger-plugins-audit-0.4.0.2.2.0.0-1744.jar:/usr/hdp/2.2.0.0-1744/storm/lib/carbonite-1.4.0.jar:/usr/hdp/2.2.0.0-1744/storm/lib/jetty-io-7.6.13.v20130916.jar:/usr/hdp/2.2.0.0-1744/storm/lib/jgrapht-core-0.9.0.jar:/usr/hdp/2.2.0.0-1744/storm/lib/ranger-storm-plugin-0.4.0.2.2.0.0-1744.jar:/usr/hdp/2.2.0.0-1744/storm/lib/joda-time-2.0.jar:/usr/hdp/2.2.0.0-1744/storm/lib/commons-logging-1.2.jar:/usr/hdp/2.2.0.0-1744/storm/lib/ranger-plugins-common-0.4.0.2.2.0.0-1744.jar:/usr/hdp/current/storm-client/contrib/storm-starter/storm-starter-0.9.3.2.2.0.0-1744-jar-with-dependencies.jar:/usr/hdp/2.2.0.0-1744/storm/conf:/usr/hdp/2.2.0.0-1744/storm/bin
-Dstorm.jar=/usr/hdp/current/storm-client/contrib/storm-starter/storm-starter-0.9.3.2.2.0.0-1744-jar-with-dependencies.jar
storm.starter.ExclamationTopology topo1
773 [main] INFO backtype.storm.StormSubmitter - Generated ZooKeeper secret
payload for MD5-digest: -5766984969082465146:-6900977625499590902
788 [main] INFO backtype.storm.security.auth.AuthUtils - Got AutoCreds []
Exception in thread "main" java.lang.RuntimeException:
java.security.NoSuchAlgorithmException: Error constructing implementation
(algorithm: JavaLoginConfig, provider: SUN, class:
sun.security.provider.ConfigSpiFile)
at
backtype.storm.security.auth.AuthUtils.GetConfiguration(AuthUtils.java:66)
at
backtype.storm.security.auth.ThriftClient.reconnect(ThriftClient.java:82)
at
backtype.storm.security.auth.ThriftClient.<init>(ThriftClient.java:66)
at backtype.storm.utils.NimbusClient.<init>(NimbusClient.java:52)
at
backtype.storm.utils.NimbusClient.getConfiguredClient(NimbusClient.java:36)
at backtype.storm.StormSubmitter.submitTopology(StormSubmitter.java:211)
at
backtype.storm.StormSubmitter.submitTopologyWithProgressBar(StormSubmitter.java:273)
at
backtype.storm.StormSubmitter.submitTopologyWithProgressBar(StormSubmitter.java:254)
at storm.starter.ExclamationTopology.main(ExclamationTopology.java:76)
Caused by: java.security.NoSuchAlgorithmException: Error constructing
implementation (algorithm: JavaLoginConfig, provider: SUN, class:
sun.security.provider.ConfigSpiFile)
at java.security.Provider$Service.newInstance(Provider.java:1259)
at sun.security.jca.GetInstance.getInstance(GetInstance.java:243)
at sun.security.jca.GetInstance.getInstance(GetInstance.java:190)
at
javax.security.auth.login.Configuration.getInstance(Configuration.java:352)
at
backtype.storm.security.auth.AuthUtils.GetConfiguration(AuthUtils.java:64)
... 8 more
Caused by: java.io.IOException: Configuration Error:
Line 1: expected [{], found [null]
at com.sun.security.auth.login.ConfigFile.match(ConfigFile.java:572)
at
com.sun.security.auth.login.ConfigFile.parseLoginEntry(ConfigFile.java:405)
at
com.sun.security.auth.login.ConfigFile.readConfig(ConfigFile.java:383)
at com.sun.security.auth.login.ConfigFile.init(ConfigFile.java:283)
at com.sun.security.auth.login.ConfigFile.init(ConfigFile.java:166)
at com.sun.security.auth.login.ConfigFile.<init>(ConfigFile.java:124)
at sun.security.provider.ConfigSpiFile$1.run(ConfigSpiFile.java:72)
at sun.security.provider.ConfigSpiFile$1.run(ConfigSpiFile.java:61)
#
at java.security.AccessController.doPrivileged(Native Method)
at sun.security.provider.ConfigSpiFile.<init>(ConfigSpiFile.java:61)
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at
sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
at
sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
at java.security.Provider$Service.newInstance(Provider.java:1254)
... 12 more
```
Diffs
-----
ambari-server/src/main/resources/stacks/HDP/2.1/services/STORM/package/scripts/params.py
32cb60d
ambari-server/src/main/resources/stacks/HDP/2.1/services/STORM/package/scripts/storm.py
5699e57
ambari-server/src/main/resources/stacks/HDP/2.1/services/STORM/package/templates/client_jaas.conf.j2
cf78af4
ambari-server/src/main/resources/stacks/HDP/2.1/services/STORM/package/templates/storm.yaml.j2
82ff239
ambari-server/src/main/resources/stacks/HDP/2.1/services/STORM/package/templates/storm_jaas.conf.j2
7c650df
Diff: https://reviews.apache.org/r/27549/diff/
Testing
-------
Ran unit tests on ambari-server,
----------------------------------------------------------------------
Total run:684
Total errors:0
Total failures:0
OK
Reproduced the issue on a live cluster, and was able to get the topology added.
E.g.,
su - ambari-qa
kinit -kt /etc/security/keytabs/smokeuser.headless.keytab [email protected]
scp -i ../gce-key
ambari-server/src/main/resources/stacks/HDP/2.1/services/STORM/package/scripts/params.py
[email protected]:/tmp
scp -i ../gce-key
ambari-server/src/main/resources/stacks/HDP/2.1/services/STORM/package/scripts/storm.py
[email protected]:/tmp
scp -i ../gce-key
ambari-server/src/main/resources/stacks/HDP/2.1/services/STORM/package/templates/client_jaas.conf.j2
[email protected]:/tmp
scp -i ../gce-key
ambari-server/src/main/resources/stacks/HDP/2.1/services/STORM/package/templates/storm.yaml.j2
[email protected]:/tmp
scp -i ../gce-key
ambari-server/src/main/resources/stacks/HDP/2.1/services/STORM/package/templates/storm_jaas.conf.j2
[email protected]:/tmp
scp -i ../gce-key
ambari-server/src/main/resources/stacks/HDP/2.1/services/STORM/package/templates/worker-launcher.cfg.j2
[email protected]:/tmp
yes | cp /tmp/params.py
/var/lib/ambari-agent/cache/stacks/HDP/2.1/services/STORM/package/scripts/params.py
yes | cp /tmp/storm.py
/var/lib/ambari-agent/cache/stacks/HDP/2.1/services/STORM/package/scripts/storm.py
yes | cp /tmp/client_jaas.conf.j2
/var/lib/ambari-agent/cache/stacks/HDP/2.1/services/STORM/package/templates/client_jaas.conf.j2
yes | cp /tmp/storm.yaml.j2
/var/lib/ambari-agent/cache/stacks/HDP/2.1/services/STORM/package/templates/storm.yaml.j2
yes | cp /tmp/storm_jaas.conf.j2
/var/lib/ambari-agent/cache/stacks/HDP/2.1/services/STORM/package/templates/storm_jaas.conf.j2
yes | cp /tmp/worker-launcher.cfg.j2
/var/lib/ambari-agent/cache/stacks/HDP/2.1/services/STORM/package/templates/worker-launcher.cfg.j2
ambari-server restart (to reload templates)
Ensure that client_jaas.conf is in /etc/storm/conf/ . Then run,
/usr/hdp/current/storm-client/bin/storm -c
java.security.auth.login.config=/etc/storm/conf/client_jaas.conf jar
/usr/hdp/current/storm-client/contrib/storm-starter/storm-starter-*-jar-with-dependencies.jar
storm.starter.ExclamationTopology topo1
Thanks,
Alejandro Fernandez
