Alejandro Fernandez created AMBARI-21129:
--------------------------------------------
Summary: Nimbus fails to start when Ambari is upgraded to 2.5.1,
EU to HDP 2.6.1, and cluster is then Kerberized
Key: AMBARI-21129
URL: https://issues.apache.org/jira/browse/AMBARI-21129
Project: Ambari
Issue Type: Bug
Components: stacks
Affects Versions: 2.5.1
Reporter: Alejandro Fernandez
Assignee: Alejandro Fernandez
Fix For: 2.5.2
STR:
# Ambari 2.5.0 with Storm and HDP 2.6 and Storm
# Upgrade Ambari to 2.5.1
# EU to HDP 2.6.1
# Kerberize the cluster
The exception found in nimbus.log:
{code}
2017-05-25 11:31:10.235 o.a.s.n.NimbusInfo [INFO] Nimbus figures out its name
to natu146-fpas-dlm30tofnsec-u14-re-re-4.openstacklocal
2017-05-25 11:31:10.247 o.a.s.d.nimbus [ERROR] Error on initialization of
server service-handler
java.lang.RuntimeException: java.lang.ClassNotFoundException:
org.apache.storm.hdfs.security.AutoHDFS
at
org.apache.storm.security.auth.AuthUtils.getNimbusAutoCredPlugins(AuthUtils.java:188)
~[storm-core-1.1.0.2.6.1.0-109.jar:1.1.0.2.6.1.0-109]
at org.apache.storm.daemon.nimbus$nimbus_data.invoke(nimbus.clj:223)
~[storm-core-1.1.0.2.6.1.0-109.jar:1.1.0.2.6.1.0-109]
at
org.apache.storm.daemon.nimbus$fn__9066$exec_fn__2832__auto____9067.invoke(nimbus.clj:2416)
~[storm-core-1.1.0.2.6.1.0-109.jar:1.1.0.2.6.1.0-109]
at clojure.lang.AFn.applyToHelper(AFn.java:156) ~[clojure-1.7.0.jar:?]
at clojure.lang.AFn.applyTo(AFn.java:144) ~[clojure-1.7.0.jar:?]
at clojure.core$apply.invoke(core.clj:630) ~[clojure-1.7.0.jar:?]
at
org.apache.storm.daemon.nimbus$fn__9066$service_handler__9099.doInvoke(nimbus.clj:2413)
~[storm-core-1.1.0.2.6.1.0-109.jar:1.1.0.2.6.1.0-109]
at clojure.lang.RestFn.invoke(RestFn.java:421) ~[clojure-1.7.0.jar:?]
at
org.apache.storm.daemon.nimbus$launch_server_BANG_.invoke(nimbus.clj:2501)
~[storm-core-1.1.0.2.6.1.0-109.jar:1.1.0.2.6.1.0-109]
at org.apache.storm.daemon.nimbus$_launch.invoke(nimbus.clj:2534)
~[storm-core-1.1.0.2.6.1.0-109.jar:1.1.0.2.6.1.0-109]
at org.apache.storm.daemon.nimbus$_main.invoke(nimbus.clj:2557)
~[storm-core-1.1.0.2.6.1.0-109.jar:1.1.0.2.6.1.0-109]
at clojure.lang.AFn.applyToHelper(AFn.java:152) ~[clojure-1.7.0.jar:?]
at clojure.lang.AFn.applyTo(AFn.java:144) ~[clojure-1.7.0.jar:?]
at org.apache.storm.daemon.nimbus.main(Unknown Source)
~[storm-core-1.1.0.2.6.1.0-109.jar:1.1.0.2.6.1.0-109]
Caused by: java.lang.ClassNotFoundException:
org.apache.storm.hdfs.security.AutoHDFS
at java.net.URLClassLoader$1.run(URLClassLoader.java:366) ~[?:1.7.0_67]
at java.net.URLClassLoader$1.run(URLClassLoader.java:355) ~[?:1.7.0_67]
at java.security.AccessController.doPrivileged(Native Method)
~[?:1.7.0_67]
at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
~[?:1.7.0_67]
at java.lang.ClassLoader.loadClass(ClassLoader.java:425) ~[?:1.7.0_67]
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
~[?:1.7.0_67]
at java.lang.ClassLoader.loadClass(ClassLoader.java:358) ~[?:1.7.0_67]
at java.lang.Class.forName0(Native Method) ~[?:1.7.0_67]
at java.lang.Class.forName(Class.java:190) ~[?:1.7.0_67]
at
org.apache.storm.security.auth.AuthUtils.getNimbusAutoCredPlugins(AuthUtils.java:181)
~[storm-core-1.1.0.2.6.1.0-109.jar:1.1.0.2.6.1.0-109]
... 13 more
2017-05-25 11:31:10.295 o.a.s.util [ERROR] Halting process: ("Error on
initialization")
java.lang.RuntimeException: ("Error on initialization")
at org.apache.storm.util$exit_process_BANG_.doInvoke(util.clj:341)
~[storm-core-1.1.0.2.6.1.0-109.jar:1.1.0.2.6.1.0-109]
at clojure.lang.RestFn.invoke(RestFn.java:423) ~[clojure-1.7.0.jar:?]
at
org.apache.storm.daemon.nimbus$fn__9066$service_handler__9099.doInvoke(nimbus.clj:2413)
~[storm-core-1.1.0.2.6.1.0-109.jar:1.1.0.2.6.1.0-109]
at clojure.lang.RestFn.invoke(RestFn.java:421) ~[clojure-1.7.0.jar:?]
at
org.apache.storm.daemon.nimbus$launch_server_BANG_.invoke(nimbus.clj:2501)
~[storm-core-1.1.0.2.6.1.0-109.jar:1.1.0.2.6.1.0-109]
at org.apache.storm.daemon.nimbus$_launch.invoke(nimbus.clj:2534)
~[storm-core-1.1.0.2.6.1.0-109.jar:1.1.0.2.6.1.0-109]
at org.apache.storm.daemon.nimbus$_main.invoke(nimbus.clj:2557)
~[storm-core-1.1.0.2.6.1.0-109.jar:1.1.0.2.6.1.0-109]
at clojure.lang.AFn.applyToHelper(AFn.java:152) ~[clojure-1.7.0.jar:?]
at clojure.lang.AFn.applyTo(AFn.java:144) ~[clojure-1.7.0.jar:?]
at org.apache.storm.daemon.nimbus.main(Unknown Source)
~[storm-core-1.1.0.2.6.1.0-109.jar:1.1.0.2.6.1.0-109]
{code}
This is because upgrading Ambari from an older version to 2.5.1 right now will
update storm-env only if Storm is present and the cluster is Kerberized.
In this case, Ambari was upgraded to 2.5.1, EU was performed to HDP 2.6.1, and
then the cluster was kerberized; kerberization has no trigger to change configs
and will not invoke Stack Advisor.
The fix is to relax the constraint so that Ambari Upgrade adds the exports to
storm-env as long as Storm is present. If Storm is added after the fact, then
its definition inside of HDP 2.6 (plus Stack Advisor) will handle that use case.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
