[
https://issues.apache.org/jira/browse/AMBARI-17100?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alejandro Fernandez updated AMBARI-17100:
-----------------------------------------
Description:
STR:
* Install Ambari 2.4
* Install HDP 2.4
* Kerberize the cluster
* Perform EU to HDP 2.5
Storm UI Server and DRPC Server:
{code}
2016-06-07 06:16:19.395 b.s.s.a.a.SimpleACLAuthorizer [INFO] [req 5] Access
from: null principal:ambari-server-...@example.com op:getClusterInfo
==> /grid/0/log/storm/ui.out <==
Running: /usr/jdk64/jdk1.8.0_77/bin/java -server -Ddaemon.name=ui
-Dstorm.options= -Dstorm.home=/grid/0/hdp/2.5.0.0-664/storm
-Dstorm.log.dir=/grid/0/log/storm
-Djava.library.path=/usr/local/lib:/opt/local/lib:/usr/lib:/usr/hdp/current/storm-client/lib
-Dstorm.conf.file= -cp
/grid/0/hdp/2.5.0.0-664/storm/lib/log4j-core-2.1.jar:/grid/0/hdp/2.5.0.0-664/storm/lib/storm-rename-hack-1.0.1.2.5.0.0-664.jar:/grid/0/hdp/2.5.0.0-664/storm/lib/slf4j-api-1.7.7.jar:/grid/0/hdp/2.5.0.0-664/storm/lib/ambari-metrics-storm-sink.jar:/grid/0/hdp/2.5.0.0-664/storm/lib/log4j-over-slf4j-1.6.6.jar:/grid/0/hdp/2.5.0.0-664/storm/lib/servlet-api-2.5.jar:/grid/0/hdp/2.5.0.0-664/storm/lib/clojure-1.7.0.jar:/grid/0/hdp/2.5.0.0-664/storm/lib/asm-5.0.3.jar:/grid/0/hdp/2.5.0.0-664/storm/lib/kryo-3.0.3.jar:/grid/0/hdp/2.5.0.0-664/storm/lib/log4j-slf4j-impl-2.1.jar:/grid/0/hdp/2.5.0.0-664/storm/lib/zookeeper.jar:/grid/0/hdp/2.5.0.0-664/storm/lib/disruptor-3.3.2.jar:/grid/0/hdp/2.5.0.0-664/storm/lib/reflectasm-1.10.1.jar:/grid/0/hdp/2.5.0.0-664/storm/lib/storm-core-1.0.1.2.5.0.0-664.jar:/grid/0/hdp/2.5.0.0-664/storm/lib/objenesis-2.1.jar:/grid/0/hdp/2.5.0.0-664/storm/lib/minlog-1.3.0.jar:/grid/0/hdp/2.5.0.0-664/storm/lib/log4j-api-2.1.jar:/grid/0/hdp/2.5.0.0-664/storm/extlib-daemon/ranger-storm-plugin-shim-0.6.0.2.5.0.0-664.jar:/grid/0/hdp/2.5.0.0-664/storm/extlib-daemon/ojdbc6.jar:/grid/0/hdp/2.5.0.0-664/storm/extlib-daemon/ranger-plugin-classloader-0.6.0.2.5.0.0-664.jar:/grid/0/hdp/2.5.0.0-664/storm:/usr/hdp/current/storm-client/conf
-Xmx768m
-Djava.security.auth.login.config=/usr/hdp/current/storm-client/conf/storm_jaas.conf
-Dlogfile.name=ui.log
-DLog4jContextSelector=org.apache.logging.log4j.core.async.AsyncLoggerContextSelector
-Dlog4j.configurationFile=/grid/0/hdp/2.5.0.0-664/storm/log4j2/cluster.xml
org.apache.storm.ui.core
Exception in thread "main" java.lang.ExceptionInInitializerError
at java.lang.Class.forName0(Native Method)
at java.lang.Class.forName(Class.java:348)
at clojure.lang.RT.classForName(RT.java:2154)
at clojure.lang.RT.classForName(RT.java:2163)
at clojure.lang.RT.loadClassForName(RT.java:2182)
at clojure.lang.RT.load(RT.java:436)
at clojure.lang.RT.load(RT.java:412)
at clojure.core$load$fn__5448.invoke(core.clj:5866)
at clojure.core$load.doInvoke(core.clj:5865)
at clojure.lang.RestFn.invoke(RestFn.java:408)
at clojure.lang.Var.invoke(Var.java:379)
at org.apache.storm.ui.core.<clinit>(Unknown Source)
Caused by: java.lang.RuntimeException: java.lang.ClassNotFoundException:
backtype.storm.security.auth.KerberosPrincipalToLocal
at
org.apache.storm.security.auth.AuthUtils.GetPrincipalToLocalPlugin(AuthUtils.java:125)
at
org.apache.storm.security.auth.authorizer.SimpleACLAuthorizer.prepare(SimpleACLAuthorizer.java:101)
at
org.apache.storm.daemon.common$mk_authorization_handler.invoke(common.clj:414)
at org.apache.storm.ui.core__init.load(Unknown Source)
at org.apache.storm.ui.core__init.<clinit>(Unknown Source)
... 12 more
Caused by: java.lang.ClassNotFoundException:
backtype.storm.security.auth.KerberosPrincipalToLocal
at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:331)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
at java.lang.Class.forName0(Native Method)
at java.lang.Class.forName(Class.java:264)
at
org.apache.storm.security.auth.AuthUtils.GetPrincipalToLocalPlugin(AuthUtils.java:121)
... 16 more
{code}
During RU, it made config changes to storm-site,
{code}
storm-site/client.jartransformer.class changed to
"org.apache.storm.hack.StormShadeTransformer"
String "backtype.storm.security.auth.SimpleTransportPlugin" was not found in
storm-site/_storm.thrift.nonsecure.transport
String "backtype.storm.security.auth.KerberosSaslTransportPlugin" was not found
in storm-site/_storm.thrift.secure.transport
String "backtype.storm.messaging.netty.Context" was not found in
storm-site/storm.messaging.transport
String "backtype.storm.nimbus.DefaultTopologyValidator" was not found in
storm-site/nimbus.topology.validator
String "backtype.storm.spout.SleepSpoutWaitStrategy" was not found in
storm-site/topology.spout.wait.strategy
String "backtype.storm.serialization.DefaultKryoFactory" was not found in
storm-site/topology.kryo.factory
String "backtype.storm.serialization.types.ListDelegateSerializer" was not
found in storm-site/topology.tuple.serializer
Replaced storm-site/nimbus.authorizer containing
"backtype.storm.security.auth.authorizer.SimpleACLAuthorizer" with
"org.apache.storm.security.auth.authorizer.SimpleACLAuthorizer"
Replaced storm-site/drpc.authorizer containing
"backtype.storm.security.auth.authorizer.DRPCSimpleACLAuthorizer" with
"org.apache.storm.security.auth.authorizer.DRPCSimpleACLAuthorizer"
String "backtype.storm.security.auth.KerberosPrincipalToLocal" was not found in
storm-site/ui.filter
{code}
The config packs have
{code}
<replace key="ui.filter"
find="backtype.storm.security.auth.KerberosPrincipalToLocal"
replace-with="org.apache.storm.security.auth.KerberosPrincipalToLocal" />
{code}
However, storm.yaml still has {code}storm.principal.tolocal :
'backtype.storm.security.auth.KerberosPrincipalToLocal'{code}
I replaced that property as well and it started working.
Storm 1.0.1 already has its kerberos.json file using "storm.principal.tolocal":
"org.apache.storm.security.auth.KerberosPrincipalToLocal", and stack HDP 2.5
uses that version of Storm, so EU/RU should set the property if it exists,
meaning the cluster is kerberized.
was:
STR:
* Install Ambari 2.4
* Install HDP 2.4
* Kerberize the cluster
* Perform EU to HDP 2.5
https://172.22.73.132:8443
Storm UI Server and DRPC Server:
{code}
2016-06-07 06:16:19.395 b.s.s.a.a.SimpleACLAuthorizer [INFO] [req 5] Access
from: null principal:ambari-server-...@example.com op:getClusterInfo
==> /grid/0/log/storm/ui.out <==
Running: /usr/jdk64/jdk1.8.0_77/bin/java -server -Ddaemon.name=ui
-Dstorm.options= -Dstorm.home=/grid/0/hdp/2.5.0.0-664/storm
-Dstorm.log.dir=/grid/0/log/storm
-Djava.library.path=/usr/local/lib:/opt/local/lib:/usr/lib:/usr/hdp/current/storm-client/lib
-Dstorm.conf.file= -cp
/grid/0/hdp/2.5.0.0-664/storm/lib/log4j-core-2.1.jar:/grid/0/hdp/2.5.0.0-664/storm/lib/storm-rename-hack-1.0.1.2.5.0.0-664.jar:/grid/0/hdp/2.5.0.0-664/storm/lib/slf4j-api-1.7.7.jar:/grid/0/hdp/2.5.0.0-664/storm/lib/ambari-metrics-storm-sink.jar:/grid/0/hdp/2.5.0.0-664/storm/lib/log4j-over-slf4j-1.6.6.jar:/grid/0/hdp/2.5.0.0-664/storm/lib/servlet-api-2.5.jar:/grid/0/hdp/2.5.0.0-664/storm/lib/clojure-1.7.0.jar:/grid/0/hdp/2.5.0.0-664/storm/lib/asm-5.0.3.jar:/grid/0/hdp/2.5.0.0-664/storm/lib/kryo-3.0.3.jar:/grid/0/hdp/2.5.0.0-664/storm/lib/log4j-slf4j-impl-2.1.jar:/grid/0/hdp/2.5.0.0-664/storm/lib/zookeeper.jar:/grid/0/hdp/2.5.0.0-664/storm/lib/disruptor-3.3.2.jar:/grid/0/hdp/2.5.0.0-664/storm/lib/reflectasm-1.10.1.jar:/grid/0/hdp/2.5.0.0-664/storm/lib/storm-core-1.0.1.2.5.0.0-664.jar:/grid/0/hdp/2.5.0.0-664/storm/lib/objenesis-2.1.jar:/grid/0/hdp/2.5.0.0-664/storm/lib/minlog-1.3.0.jar:/grid/0/hdp/2.5.0.0-664/storm/lib/log4j-api-2.1.jar:/grid/0/hdp/2.5.0.0-664/storm/extlib-daemon/ranger-storm-plugin-shim-0.6.0.2.5.0.0-664.jar:/grid/0/hdp/2.5.0.0-664/storm/extlib-daemon/ojdbc6.jar:/grid/0/hdp/2.5.0.0-664/storm/extlib-daemon/ranger-plugin-classloader-0.6.0.2.5.0.0-664.jar:/grid/0/hdp/2.5.0.0-664/storm:/usr/hdp/current/storm-client/conf
-Xmx768m
-Djava.security.auth.login.config=/usr/hdp/current/storm-client/conf/storm_jaas.conf
-Dlogfile.name=ui.log
-DLog4jContextSelector=org.apache.logging.log4j.core.async.AsyncLoggerContextSelector
-Dlog4j.configurationFile=/grid/0/hdp/2.5.0.0-664/storm/log4j2/cluster.xml
org.apache.storm.ui.core
Exception in thread "main" java.lang.ExceptionInInitializerError
at java.lang.Class.forName0(Native Method)
at java.lang.Class.forName(Class.java:348)
at clojure.lang.RT.classForName(RT.java:2154)
at clojure.lang.RT.classForName(RT.java:2163)
at clojure.lang.RT.loadClassForName(RT.java:2182)
at clojure.lang.RT.load(RT.java:436)
at clojure.lang.RT.load(RT.java:412)
at clojure.core$load$fn__5448.invoke(core.clj:5866)
at clojure.core$load.doInvoke(core.clj:5865)
at clojure.lang.RestFn.invoke(RestFn.java:408)
at clojure.lang.Var.invoke(Var.java:379)
at org.apache.storm.ui.core.<clinit>(Unknown Source)
Caused by: java.lang.RuntimeException: java.lang.ClassNotFoundException:
backtype.storm.security.auth.KerberosPrincipalToLocal
at
org.apache.storm.security.auth.AuthUtils.GetPrincipalToLocalPlugin(AuthUtils.java:125)
at
org.apache.storm.security.auth.authorizer.SimpleACLAuthorizer.prepare(SimpleACLAuthorizer.java:101)
at
org.apache.storm.daemon.common$mk_authorization_handler.invoke(common.clj:414)
at org.apache.storm.ui.core__init.load(Unknown Source)
at org.apache.storm.ui.core__init.<clinit>(Unknown Source)
... 12 more
Caused by: java.lang.ClassNotFoundException:
backtype.storm.security.auth.KerberosPrincipalToLocal
at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:331)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
at java.lang.Class.forName0(Native Method)
at java.lang.Class.forName(Class.java:264)
at
org.apache.storm.security.auth.AuthUtils.GetPrincipalToLocalPlugin(AuthUtils.java:121)
... 16 more
{code}
During RU, it made config changes to storm-site,
{code}
storm-site/client.jartransformer.class changed to
"org.apache.storm.hack.StormShadeTransformer"
String "backtype.storm.security.auth.SimpleTransportPlugin" was not found in
storm-site/_storm.thrift.nonsecure.transport
String "backtype.storm.security.auth.KerberosSaslTransportPlugin" was not found
in storm-site/_storm.thrift.secure.transport
String "backtype.storm.messaging.netty.Context" was not found in
storm-site/storm.messaging.transport
String "backtype.storm.nimbus.DefaultTopologyValidator" was not found in
storm-site/nimbus.topology.validator
String "backtype.storm.spout.SleepSpoutWaitStrategy" was not found in
storm-site/topology.spout.wait.strategy
String "backtype.storm.serialization.DefaultKryoFactory" was not found in
storm-site/topology.kryo.factory
String "backtype.storm.serialization.types.ListDelegateSerializer" was not
found in storm-site/topology.tuple.serializer
Replaced storm-site/nimbus.authorizer containing
"backtype.storm.security.auth.authorizer.SimpleACLAuthorizer" with
"org.apache.storm.security.auth.authorizer.SimpleACLAuthorizer"
Replaced storm-site/drpc.authorizer containing
"backtype.storm.security.auth.authorizer.DRPCSimpleACLAuthorizer" with
"org.apache.storm.security.auth.authorizer.DRPCSimpleACLAuthorizer"
String "backtype.storm.security.auth.KerberosPrincipalToLocal" was not found in
storm-site/ui.filter
{code}
The config packs have
{code}
<replace key="ui.filter"
find="backtype.storm.security.auth.KerberosPrincipalToLocal"
replace-with="org.apache.storm.security.auth.KerberosPrincipalToLocal" />
{code}
However, storm.yaml still has {code}storm.principal.tolocal :
'backtype.storm.security.auth.KerberosPrincipalToLocal'{code}
I replaced that property as well and it started working.
Storm 1.0.1 already has its kerberos.json file using "storm.principal.tolocal":
"org.apache.storm.security.auth.KerberosPrincipalToLocal", and stack HDP 2.5
uses that version of Storm, so EU/RU should set the property if it exists,
meaning the cluster is kerberized.
> EU - HDP 2.4 to 2.5 fails restarting DRPC server on a kerberized cluster,
> need to use org.apache.storm.security.auth.KerberosPrincipalToLocal
> ---------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: AMBARI-17100
> URL: https://issues.apache.org/jira/browse/AMBARI-17100
> Project: Ambari
> Issue Type: Bug
> Components: stacks
> Affects Versions: 2.4.0
> Reporter: Alejandro Fernandez
> Assignee: Alejandro Fernandez
> Fix For: 2.4.0
>
> Attachments: AMBARI-17100.branch-2.4.patch, AMBARI-17100.trunk.patch
>
>
> STR:
> * Install Ambari 2.4
> * Install HDP 2.4
> * Kerberize the cluster
> * Perform EU to HDP 2.5
> Storm UI Server and DRPC Server:
> {code}
> 2016-06-07 06:16:19.395 b.s.s.a.a.SimpleACLAuthorizer [INFO] [req 5] Access
> from: null principal:ambari-server-...@example.com op:getClusterInfo
> ==> /grid/0/log/storm/ui.out <==
> Running: /usr/jdk64/jdk1.8.0_77/bin/java -server -Ddaemon.name=ui
> -Dstorm.options= -Dstorm.home=/grid/0/hdp/2.5.0.0-664/storm
> -Dstorm.log.dir=/grid/0/log/storm
> -Djava.library.path=/usr/local/lib:/opt/local/lib:/usr/lib:/usr/hdp/current/storm-client/lib
> -Dstorm.conf.file= -cp
> /grid/0/hdp/2.5.0.0-664/storm/lib/log4j-core-2.1.jar:/grid/0/hdp/2.5.0.0-664/storm/lib/storm-rename-hack-1.0.1.2.5.0.0-664.jar:/grid/0/hdp/2.5.0.0-664/storm/lib/slf4j-api-1.7.7.jar:/grid/0/hdp/2.5.0.0-664/storm/lib/ambari-metrics-storm-sink.jar:/grid/0/hdp/2.5.0.0-664/storm/lib/log4j-over-slf4j-1.6.6.jar:/grid/0/hdp/2.5.0.0-664/storm/lib/servlet-api-2.5.jar:/grid/0/hdp/2.5.0.0-664/storm/lib/clojure-1.7.0.jar:/grid/0/hdp/2.5.0.0-664/storm/lib/asm-5.0.3.jar:/grid/0/hdp/2.5.0.0-664/storm/lib/kryo-3.0.3.jar:/grid/0/hdp/2.5.0.0-664/storm/lib/log4j-slf4j-impl-2.1.jar:/grid/0/hdp/2.5.0.0-664/storm/lib/zookeeper.jar:/grid/0/hdp/2.5.0.0-664/storm/lib/disruptor-3.3.2.jar:/grid/0/hdp/2.5.0.0-664/storm/lib/reflectasm-1.10.1.jar:/grid/0/hdp/2.5.0.0-664/storm/lib/storm-core-1.0.1.2.5.0.0-664.jar:/grid/0/hdp/2.5.0.0-664/storm/lib/objenesis-2.1.jar:/grid/0/hdp/2.5.0.0-664/storm/lib/minlog-1.3.0.jar:/grid/0/hdp/2.5.0.0-664/storm/lib/log4j-api-2.1.jar:/grid/0/hdp/2.5.0.0-664/storm/extlib-daemon/ranger-storm-plugin-shim-0.6.0.2.5.0.0-664.jar:/grid/0/hdp/2.5.0.0-664/storm/extlib-daemon/ojdbc6.jar:/grid/0/hdp/2.5.0.0-664/storm/extlib-daemon/ranger-plugin-classloader-0.6.0.2.5.0.0-664.jar:/grid/0/hdp/2.5.0.0-664/storm:/usr/hdp/current/storm-client/conf
> -Xmx768m
> -Djava.security.auth.login.config=/usr/hdp/current/storm-client/conf/storm_jaas.conf
> -Dlogfile.name=ui.log
> -DLog4jContextSelector=org.apache.logging.log4j.core.async.AsyncLoggerContextSelector
> -Dlog4j.configurationFile=/grid/0/hdp/2.5.0.0-664/storm/log4j2/cluster.xml
> org.apache.storm.ui.core
> Exception in thread "main" java.lang.ExceptionInInitializerError
> at java.lang.Class.forName0(Native Method)
> at java.lang.Class.forName(Class.java:348)
> at clojure.lang.RT.classForName(RT.java:2154)
> at clojure.lang.RT.classForName(RT.java:2163)
> at clojure.lang.RT.loadClassForName(RT.java:2182)
> at clojure.lang.RT.load(RT.java:436)
> at clojure.lang.RT.load(RT.java:412)
> at clojure.core$load$fn__5448.invoke(core.clj:5866)
> at clojure.core$load.doInvoke(core.clj:5865)
> at clojure.lang.RestFn.invoke(RestFn.java:408)
> at clojure.lang.Var.invoke(Var.java:379)
> at org.apache.storm.ui.core.<clinit>(Unknown Source)
> Caused by: java.lang.RuntimeException: java.lang.ClassNotFoundException:
> backtype.storm.security.auth.KerberosPrincipalToLocal
> at
> org.apache.storm.security.auth.AuthUtils.GetPrincipalToLocalPlugin(AuthUtils.java:125)
> at
> org.apache.storm.security.auth.authorizer.SimpleACLAuthorizer.prepare(SimpleACLAuthorizer.java:101)
> at
> org.apache.storm.daemon.common$mk_authorization_handler.invoke(common.clj:414)
> at org.apache.storm.ui.core__init.load(Unknown Source)
> at org.apache.storm.ui.core__init.<clinit>(Unknown Source)
> ... 12 more
> Caused by: java.lang.ClassNotFoundException:
> backtype.storm.security.auth.KerberosPrincipalToLocal
> at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:331)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
> at java.lang.Class.forName0(Native Method)
> at java.lang.Class.forName(Class.java:264)
> at
> org.apache.storm.security.auth.AuthUtils.GetPrincipalToLocalPlugin(AuthUtils.java:121)
> ... 16 more
> {code}
> During RU, it made config changes to storm-site,
> {code}
> storm-site/client.jartransformer.class changed to
> "org.apache.storm.hack.StormShadeTransformer"
> String "backtype.storm.security.auth.SimpleTransportPlugin" was not found in
> storm-site/_storm.thrift.nonsecure.transport
> String "backtype.storm.security.auth.KerberosSaslTransportPlugin" was not
> found in storm-site/_storm.thrift.secure.transport
> String "backtype.storm.messaging.netty.Context" was not found in
> storm-site/storm.messaging.transport
> String "backtype.storm.nimbus.DefaultTopologyValidator" was not found in
> storm-site/nimbus.topology.validator
> String "backtype.storm.spout.SleepSpoutWaitStrategy" was not found in
> storm-site/topology.spout.wait.strategy
> String "backtype.storm.serialization.DefaultKryoFactory" was not found in
> storm-site/topology.kryo.factory
> String "backtype.storm.serialization.types.ListDelegateSerializer" was not
> found in storm-site/topology.tuple.serializer
> Replaced storm-site/nimbus.authorizer containing
> "backtype.storm.security.auth.authorizer.SimpleACLAuthorizer" with
> "org.apache.storm.security.auth.authorizer.SimpleACLAuthorizer"
> Replaced storm-site/drpc.authorizer containing
> "backtype.storm.security.auth.authorizer.DRPCSimpleACLAuthorizer" with
> "org.apache.storm.security.auth.authorizer.DRPCSimpleACLAuthorizer"
> String "backtype.storm.security.auth.KerberosPrincipalToLocal" was not found
> in storm-site/ui.filter
> {code}
> The config packs have
> {code}
> <replace key="ui.filter"
> find="backtype.storm.security.auth.KerberosPrincipalToLocal"
>
> replace-with="org.apache.storm.security.auth.KerberosPrincipalToLocal" />
> {code}
> However, storm.yaml still has {code}storm.principal.tolocal :
> 'backtype.storm.security.auth.KerberosPrincipalToLocal'{code}
> I replaced that property as well and it started working.
> Storm 1.0.1 already has its kerberos.json file using
> "storm.principal.tolocal":
> "org.apache.storm.security.auth.KerberosPrincipalToLocal", and stack HDP 2.5
> uses that version of Storm, so EU/RU should set the property if it exists,
> meaning the cluster is kerberized.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
