Re: Review Request 51241: PREVIEW - RU: Storm components were stopped during RU and can not be started

2016-08-19 Thread Alejandro Fernandez 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/51241/#review146230
---


Ship it!




Ship It!

- Alejandro Fernandez


On Aug. 19, 2016, 4:54 p.m., Dmitro Lisnichenko wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/51241/
> ---
> 
> (Updated Aug. 19, 2016, 4:54 p.m.)
> 
> 
> Review request for Ambari, Jonathan Hurley, Jayush Luniya, and Nate Cole.
> 
> 
> Bugs: AMBARI-18213
> https://issues.apache.org/jira/browse/AMBARI-18213
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> STR:
> # Install cluster 2.4.2.0-258 on Ambari 2.2.2.0
> # Enable HA
> # Enable security
> # Upgrade ambari to 240
> # Perform RU to 2.5.0.0-1208
> 
> Deeper study shows that kerberos descriptor json in database ("artifact" 
> table) still contains values and properties that are actual for 2.4 stack.
> So the issue workflow should look like:
> - Old stack version is installed
> - Kerberos descriptor gets saved to database
> - Security is enabled
> - Stack upgrade is performed
> - Keytab regeneration is performed, and it populates service config with 
> obsolete property values
> 
> The issue happens on "Stack upgrade is performed" step. We never update 
> kerberos descriptor json in database to correspond to a new stack.
> 
> From nimbus.out
> {code}Exception in thread "main" java.lang.ExceptionInInitializerError
> at java.lang.Class.forName0(Native Method)
> at java.lang.Class.forName(Class.java:270)
> at clojure.lang.RT.classForName(RT.java:2154)
> at clojure.lang.RT.classForName(RT.java:2163)
> at clojure.lang.RT.loadClassForName(RT.java:2182)
> at clojure.lang.RT.load(RT.java:436)
> at clojure.lang.RT.load(RT.java:412)
> at clojure.core$load$fn__5448.invoke(core.clj:5866)
> at clojure.core$load.doInvoke(core.clj:5865)
> at clojure.lang.RestFn.invoke(RestFn.java:408)
> at clojure.core$load_one.invoke(core.clj:5671)
> at clojure.core$load_lib$fn__5397.invoke(core.clj:5711)
> at clojure.core$load_lib.doInvoke(core.clj:5710)
> at clojure.lang.RestFn.applyTo(RestFn.java:142)
> at clojure.core$apply.invoke(core.clj:632)
> at clojure.core$load_libs.doInvoke(core.clj:5749)
> at clojure.lang.RestFn.applyTo(RestFn.java:137)
> at clojure.core$apply.invoke(core.clj:632)
> at clojure.core$require.doInvoke(core.clj:5832)
> at clojure.lang.RestFn.invoke(RestFn.java:408)
> at 
> org.apache.storm.daemon.nimbus$loading__5340__auto8560.invoke(nimbus.clj:16)
> at org.apache.storm.daemon.nimbus__init.load(Unknown Source)
> at org.apache.storm.daemon.nimbus__init.(Unknown Source)
> at java.lang.Class.forName0(Native Method)
> at java.lang.Class.forName(Class.java:270)
> at clojure.lang.RT.classForName(RT.java:2154)
> at clojure.lang.RT.classForName(RT.java:2163)
> at clojure.lang.RT.loadClassForName(RT.java:2182)
> at clojure.lang.RT.load(RT.java:436)
> at clojure.lang.RT.load(RT.java:412)
> at clojure.core$load$fn__5448.invoke(core.clj:5866)
> at clojure.core$load.doInvoke(core.clj:5865)
> at clojure.lang.RestFn.invoke(RestFn.java:408)
> at clojure.lang.Var.invoke(Var.java:379)
> at org.apache.storm.daemon.nimbus.(Unknown Source)
> Caused by: java.lang.ClassNotFoundException: 
> backtype.storm.security.auth.authorizer.SimpleACLAuthorizer
> at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
> at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
> at java.security.AccessController.doPrivileged(Native Method)
> at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
> at java.lang.Class.forName0(Native Method)
> at java.lang.Class.forName(Class.java:190)
> at 
> org.apache.storm.daemon.common$mk_authorization_handler.invoke(common.clj:412)
> at org.apache.storm.ui.core__init.load(Unknown Source)
> at org.apache.storm.ui.core__init.(Unknown Source)
> ... 35 more
> {code}
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/serveraction/upgrades/StormUpgradeKerberosDescriptorConfig.java
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/nonrolling-upgrade-2.5.xml
>  65a7880 
>   ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/upgrade-2.5.xml 
> bc8e9f9 
>   
> ambari-server/src/main/resources/stacks/HDP/2.4/upgrades/nonrolling-upgrade-2.5.xml
>  98e5038 
>   ambari-server/src/main/resources/stacks/HDP/2.4/upgrades/upgrade-2.5.xml 
> e886301 
> 
> Diff: https://reviews.apache.org/r/51241/diff/
> 
> 
> Testing
> ---
> 
> As of now, testing on live cluster
> 
> 
> Thanks,
> 
> Dmitro Lisnichenko
> 
>

Re: Review Request 51241: PREVIEW - RU: Storm components were stopped during RU and can not be started

2016-08-19 Thread Nate Cole 


> On Aug. 19, 2016, 11:16 a.m., Nate Cole wrote:
> > ambari-server/src/main/java/org/apache/ambari/server/serveraction/upgrades/StormUpgradeKerberosDescriptorConfig.java,
> >  line 81
> > 
> >
> > Probably use a common function for this (have to make it public:  
> > KerberosHelperImpl.getKerberosDescriptorUpdates(Cluster cluster)
> > 
> > Have to inject the KerberosHelper.
> 
> Dmitro Lisnichenko wrote:
> We would in any case need ArtifactDAO here (at least to write new value). 
> Extracted a separate method
> 
> Jonathan Hurley wrote:
> I agree with Nate here; you should use the helper method to get the exact 
> right cluster. Otherwise, you're updating all of them. I know we don't really 
> support multi-cluster, but we shouldn't just do all of them.

I'm not saying you can't use the ArtifactDAO to merge - but use the 
KerberosHelperImpl method to get the ONLY entity for the current cluster.  Your 
code iterates all of them, updating the property for all entities, whether it's 
for the cluster of interest or not.


- Nate


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/51241/#review146182
---


On Aug. 19, 2016, 12:54 p.m., Dmitro Lisnichenko wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/51241/
> ---
> 
> (Updated Aug. 19, 2016, 12:54 p.m.)
> 
> 
> Review request for Ambari, Jonathan Hurley, Jayush Luniya, and Nate Cole.
> 
> 
> Bugs: AMBARI-18213
> https://issues.apache.org/jira/browse/AMBARI-18213
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> STR:
> # Install cluster 2.4.2.0-258 on Ambari 2.2.2.0
> # Enable HA
> # Enable security
> # Upgrade ambari to 240
> # Perform RU to 2.5.0.0-1208
> 
> Deeper study shows that kerberos descriptor json in database ("artifact" 
> table) still contains values and properties that are actual for 2.4 stack.
> So the issue workflow should look like:
> - Old stack version is installed
> - Kerberos descriptor gets saved to database
> - Security is enabled
> - Stack upgrade is performed
> - Keytab regeneration is performed, and it populates service config with 
> obsolete property values
> 
> The issue happens on "Stack upgrade is performed" step. We never update 
> kerberos descriptor json in database to correspond to a new stack.
> 
> From nimbus.out
> {code}Exception in thread "main" java.lang.ExceptionInInitializerError
> at java.lang.Class.forName0(Native Method)
> at java.lang.Class.forName(Class.java:270)
> at clojure.lang.RT.classForName(RT.java:2154)
> at clojure.lang.RT.classForName(RT.java:2163)
> at clojure.lang.RT.loadClassForName(RT.java:2182)
> at clojure.lang.RT.load(RT.java:436)
> at clojure.lang.RT.load(RT.java:412)
> at clojure.core$load$fn__5448.invoke(core.clj:5866)
> at clojure.core$load.doInvoke(core.clj:5865)
> at clojure.lang.RestFn.invoke(RestFn.java:408)
> at clojure.core$load_one.invoke(core.clj:5671)
> at clojure.core$load_lib$fn__5397.invoke(core.clj:5711)
> at clojure.core$load_lib.doInvoke(core.clj:5710)
> at clojure.lang.RestFn.applyTo(RestFn.java:142)
> at clojure.core$apply.invoke(core.clj:632)
> at clojure.core$load_libs.doInvoke(core.clj:5749)
> at clojure.lang.RestFn.applyTo(RestFn.java:137)
> at clojure.core$apply.invoke(core.clj:632)
> at clojure.core$require.doInvoke(core.clj:5832)
> at clojure.lang.RestFn.invoke(RestFn.java:408)
> at 
> org.apache.storm.daemon.nimbus$loading__5340__auto8560.invoke(nimbus.clj:16)
> at org.apache.storm.daemon.nimbus__init.load(Unknown Source)
> at org.apache.storm.daemon.nimbus__init.(Unknown Source)
> at java.lang.Class.forName0(Native Method)
> at java.lang.Class.forName(Class.java:270)
> at clojure.lang.RT.classForName(RT.java:2154)
> at clojure.lang.RT.classForName(RT.java:2163)
> at clojure.lang.RT.loadClassForName(RT.java:2182)
> at clojure.lang.RT.load(RT.java:436)
> at clojure.lang.RT.load(RT.java:412)
> at clojure.core$load$fn__5448.invoke(core.clj:5866)
> at clojure.core$load.doInvoke(core.clj:5865)
> at clojure.lang.RestFn.invoke(RestFn.java:408)
> at clojure.lang.Var.invoke(Var.java:379)
> at org.apache.storm.daemon.nimbus.(Unknown Source)
> Caused by: java.lang.ClassNotFoundException: 
> backtype.storm.security.auth.authorizer.SimpleACLAuthorizer
> at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
> at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
> at java.security.AccessController.doPrivileged(Native Method)
> at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
> at java.lang.ClassLoader.lo

Re: Review Request 51241: PREVIEW - RU: Storm components were stopped during RU and can not be started

2016-08-19 Thread Jonathan Hurley 


> On Aug. 19, 2016, 11:16 a.m., Nate Cole wrote:
> > ambari-server/src/main/java/org/apache/ambari/server/serveraction/upgrades/StormUpgradeKerberosDescriptorConfig.java,
> >  line 81
> > 
> >
> > Probably use a common function for this (have to make it public:  
> > KerberosHelperImpl.getKerberosDescriptorUpdates(Cluster cluster)
> > 
> > Have to inject the KerberosHelper.
> 
> Dmitro Lisnichenko wrote:
> We would in any case need ArtifactDAO here (at least to write new value). 
> Extracted a separate method

I agree with Nate here; you should use the helper method to get the exact right 
cluster. Otherwise, you're updating all of them. I know we don't really support 
multi-cluster, but we shouldn't just do all of them.


- Jonathan


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/51241/#review146182
---


On Aug. 19, 2016, 12:54 p.m., Dmitro Lisnichenko wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/51241/
> ---
> 
> (Updated Aug. 19, 2016, 12:54 p.m.)
> 
> 
> Review request for Ambari, Jonathan Hurley, Jayush Luniya, and Nate Cole.
> 
> 
> Bugs: AMBARI-18213
> https://issues.apache.org/jira/browse/AMBARI-18213
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> STR:
> # Install cluster 2.4.2.0-258 on Ambari 2.2.2.0
> # Enable HA
> # Enable security
> # Upgrade ambari to 240
> # Perform RU to 2.5.0.0-1208
> 
> Deeper study shows that kerberos descriptor json in database ("artifact" 
> table) still contains values and properties that are actual for 2.4 stack.
> So the issue workflow should look like:
> - Old stack version is installed
> - Kerberos descriptor gets saved to database
> - Security is enabled
> - Stack upgrade is performed
> - Keytab regeneration is performed, and it populates service config with 
> obsolete property values
> 
> The issue happens on "Stack upgrade is performed" step. We never update 
> kerberos descriptor json in database to correspond to a new stack.
> 
> From nimbus.out
> {code}Exception in thread "main" java.lang.ExceptionInInitializerError
> at java.lang.Class.forName0(Native Method)
> at java.lang.Class.forName(Class.java:270)
> at clojure.lang.RT.classForName(RT.java:2154)
> at clojure.lang.RT.classForName(RT.java:2163)
> at clojure.lang.RT.loadClassForName(RT.java:2182)
> at clojure.lang.RT.load(RT.java:436)
> at clojure.lang.RT.load(RT.java:412)
> at clojure.core$load$fn__5448.invoke(core.clj:5866)
> at clojure.core$load.doInvoke(core.clj:5865)
> at clojure.lang.RestFn.invoke(RestFn.java:408)
> at clojure.core$load_one.invoke(core.clj:5671)
> at clojure.core$load_lib$fn__5397.invoke(core.clj:5711)
> at clojure.core$load_lib.doInvoke(core.clj:5710)
> at clojure.lang.RestFn.applyTo(RestFn.java:142)
> at clojure.core$apply.invoke(core.clj:632)
> at clojure.core$load_libs.doInvoke(core.clj:5749)
> at clojure.lang.RestFn.applyTo(RestFn.java:137)
> at clojure.core$apply.invoke(core.clj:632)
> at clojure.core$require.doInvoke(core.clj:5832)
> at clojure.lang.RestFn.invoke(RestFn.java:408)
> at 
> org.apache.storm.daemon.nimbus$loading__5340__auto8560.invoke(nimbus.clj:16)
> at org.apache.storm.daemon.nimbus__init.load(Unknown Source)
> at org.apache.storm.daemon.nimbus__init.(Unknown Source)
> at java.lang.Class.forName0(Native Method)
> at java.lang.Class.forName(Class.java:270)
> at clojure.lang.RT.classForName(RT.java:2154)
> at clojure.lang.RT.classForName(RT.java:2163)
> at clojure.lang.RT.loadClassForName(RT.java:2182)
> at clojure.lang.RT.load(RT.java:436)
> at clojure.lang.RT.load(RT.java:412)
> at clojure.core$load$fn__5448.invoke(core.clj:5866)
> at clojure.core$load.doInvoke(core.clj:5865)
> at clojure.lang.RestFn.invoke(RestFn.java:408)
> at clojure.lang.Var.invoke(Var.java:379)
> at org.apache.storm.daemon.nimbus.(Unknown Source)
> Caused by: java.lang.ClassNotFoundException: 
> backtype.storm.security.auth.authorizer.SimpleACLAuthorizer
> at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
> at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
> at java.security.AccessController.doPrivileged(Native Method)
> at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
> at java.lang.Class.forName0(Native Method)
> at java.lang.Class.forName(Class.java:190)
> at 
> org.apache.storm.daemon.common$mk_authorization_handler.invoke(common.clj:412)
> at org.apache.storm.ui.core__init.load(Unknown Source)
> at org.apache.storm.ui.core__init.(

Re: Review Request 51241: PREVIEW - RU: Storm components were stopped during RU and can not be started

2016-08-19 Thread Jayush Luniya 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/51241/#review146202
---


Ship it!




Ship It!

- Jayush Luniya


On Aug. 19, 2016, 4:54 p.m., Dmitro Lisnichenko wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/51241/
> ---
> 
> (Updated Aug. 19, 2016, 4:54 p.m.)
> 
> 
> Review request for Ambari, Jonathan Hurley, Jayush Luniya, and Nate Cole.
> 
> 
> Bugs: AMBARI-18213
> https://issues.apache.org/jira/browse/AMBARI-18213
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> STR:
> # Install cluster 2.4.2.0-258 on Ambari 2.2.2.0
> # Enable HA
> # Enable security
> # Upgrade ambari to 240
> # Perform RU to 2.5.0.0-1208
> 
> Deeper study shows that kerberos descriptor json in database ("artifact" 
> table) still contains values and properties that are actual for 2.4 stack.
> So the issue workflow should look like:
> - Old stack version is installed
> - Kerberos descriptor gets saved to database
> - Security is enabled
> - Stack upgrade is performed
> - Keytab regeneration is performed, and it populates service config with 
> obsolete property values
> 
> The issue happens on "Stack upgrade is performed" step. We never update 
> kerberos descriptor json in database to correspond to a new stack.
> 
> From nimbus.out
> {code}Exception in thread "main" java.lang.ExceptionInInitializerError
> at java.lang.Class.forName0(Native Method)
> at java.lang.Class.forName(Class.java:270)
> at clojure.lang.RT.classForName(RT.java:2154)
> at clojure.lang.RT.classForName(RT.java:2163)
> at clojure.lang.RT.loadClassForName(RT.java:2182)
> at clojure.lang.RT.load(RT.java:436)
> at clojure.lang.RT.load(RT.java:412)
> at clojure.core$load$fn__5448.invoke(core.clj:5866)
> at clojure.core$load.doInvoke(core.clj:5865)
> at clojure.lang.RestFn.invoke(RestFn.java:408)
> at clojure.core$load_one.invoke(core.clj:5671)
> at clojure.core$load_lib$fn__5397.invoke(core.clj:5711)
> at clojure.core$load_lib.doInvoke(core.clj:5710)
> at clojure.lang.RestFn.applyTo(RestFn.java:142)
> at clojure.core$apply.invoke(core.clj:632)
> at clojure.core$load_libs.doInvoke(core.clj:5749)
> at clojure.lang.RestFn.applyTo(RestFn.java:137)
> at clojure.core$apply.invoke(core.clj:632)
> at clojure.core$require.doInvoke(core.clj:5832)
> at clojure.lang.RestFn.invoke(RestFn.java:408)
> at 
> org.apache.storm.daemon.nimbus$loading__5340__auto8560.invoke(nimbus.clj:16)
> at org.apache.storm.daemon.nimbus__init.load(Unknown Source)
> at org.apache.storm.daemon.nimbus__init.(Unknown Source)
> at java.lang.Class.forName0(Native Method)
> at java.lang.Class.forName(Class.java:270)
> at clojure.lang.RT.classForName(RT.java:2154)
> at clojure.lang.RT.classForName(RT.java:2163)
> at clojure.lang.RT.loadClassForName(RT.java:2182)
> at clojure.lang.RT.load(RT.java:436)
> at clojure.lang.RT.load(RT.java:412)
> at clojure.core$load$fn__5448.invoke(core.clj:5866)
> at clojure.core$load.doInvoke(core.clj:5865)
> at clojure.lang.RestFn.invoke(RestFn.java:408)
> at clojure.lang.Var.invoke(Var.java:379)
> at org.apache.storm.daemon.nimbus.(Unknown Source)
> Caused by: java.lang.ClassNotFoundException: 
> backtype.storm.security.auth.authorizer.SimpleACLAuthorizer
> at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
> at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
> at java.security.AccessController.doPrivileged(Native Method)
> at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
> at java.lang.Class.forName0(Native Method)
> at java.lang.Class.forName(Class.java:190)
> at 
> org.apache.storm.daemon.common$mk_authorization_handler.invoke(common.clj:412)
> at org.apache.storm.ui.core__init.load(Unknown Source)
> at org.apache.storm.ui.core__init.(Unknown Source)
> ... 35 more
> {code}
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/serveraction/upgrades/StormUpgradeKerberosDescriptorConfig.java
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/nonrolling-upgrade-2.5.xml
>  65a7880 
>   ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/upgrade-2.5.xml 
> bc8e9f9 
>   
> ambari-server/src/main/resources/stacks/HDP/2.4/upgrades/nonrolling-upgrade-2.5.xml
>  98e5038 
>   ambari-server/src/main/resources/stacks/HDP/2.4/upgrades/upgrade-2.5.xml 
> e886301 
> 
> Diff: https://reviews.apache.org/r/51241/diff/
> 
> 
> Testing
> ---
> 
> As of now, testing on live cluster
> 
> 
> Thanks,
> 
> Dmitro Lisnichenko
> 
>

Re: Review Request 51241: PREVIEW - RU: Storm components were stopped during RU and can not be started

2016-08-19 Thread Jayush Luniya 


> On Aug. 19, 2016, 3:41 p.m., Jonathan Hurley wrote:
> > I think it would be nice to make this a re-usable class in the future for 
> > updating Kerberos descriptors. But since we don't have the immediate need 
> > nor the time, doing a one-off is fine.

+1 on make it reusable in the future.


- Jayush


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/51241/#review146186
---


On Aug. 19, 2016, 4:54 p.m., Dmitro Lisnichenko wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/51241/
> ---
> 
> (Updated Aug. 19, 2016, 4:54 p.m.)
> 
> 
> Review request for Ambari, Jonathan Hurley, Jayush Luniya, and Nate Cole.
> 
> 
> Bugs: AMBARI-18213
> https://issues.apache.org/jira/browse/AMBARI-18213
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> STR:
> # Install cluster 2.4.2.0-258 on Ambari 2.2.2.0
> # Enable HA
> # Enable security
> # Upgrade ambari to 240
> # Perform RU to 2.5.0.0-1208
> 
> Deeper study shows that kerberos descriptor json in database ("artifact" 
> table) still contains values and properties that are actual for 2.4 stack.
> So the issue workflow should look like:
> - Old stack version is installed
> - Kerberos descriptor gets saved to database
> - Security is enabled
> - Stack upgrade is performed
> - Keytab regeneration is performed, and it populates service config with 
> obsolete property values
> 
> The issue happens on "Stack upgrade is performed" step. We never update 
> kerberos descriptor json in database to correspond to a new stack.
> 
> From nimbus.out
> {code}Exception in thread "main" java.lang.ExceptionInInitializerError
> at java.lang.Class.forName0(Native Method)
> at java.lang.Class.forName(Class.java:270)
> at clojure.lang.RT.classForName(RT.java:2154)
> at clojure.lang.RT.classForName(RT.java:2163)
> at clojure.lang.RT.loadClassForName(RT.java:2182)
> at clojure.lang.RT.load(RT.java:436)
> at clojure.lang.RT.load(RT.java:412)
> at clojure.core$load$fn__5448.invoke(core.clj:5866)
> at clojure.core$load.doInvoke(core.clj:5865)
> at clojure.lang.RestFn.invoke(RestFn.java:408)
> at clojure.core$load_one.invoke(core.clj:5671)
> at clojure.core$load_lib$fn__5397.invoke(core.clj:5711)
> at clojure.core$load_lib.doInvoke(core.clj:5710)
> at clojure.lang.RestFn.applyTo(RestFn.java:142)
> at clojure.core$apply.invoke(core.clj:632)
> at clojure.core$load_libs.doInvoke(core.clj:5749)
> at clojure.lang.RestFn.applyTo(RestFn.java:137)
> at clojure.core$apply.invoke(core.clj:632)
> at clojure.core$require.doInvoke(core.clj:5832)
> at clojure.lang.RestFn.invoke(RestFn.java:408)
> at 
> org.apache.storm.daemon.nimbus$loading__5340__auto8560.invoke(nimbus.clj:16)
> at org.apache.storm.daemon.nimbus__init.load(Unknown Source)
> at org.apache.storm.daemon.nimbus__init.(Unknown Source)
> at java.lang.Class.forName0(Native Method)
> at java.lang.Class.forName(Class.java:270)
> at clojure.lang.RT.classForName(RT.java:2154)
> at clojure.lang.RT.classForName(RT.java:2163)
> at clojure.lang.RT.loadClassForName(RT.java:2182)
> at clojure.lang.RT.load(RT.java:436)
> at clojure.lang.RT.load(RT.java:412)
> at clojure.core$load$fn__5448.invoke(core.clj:5866)
> at clojure.core$load.doInvoke(core.clj:5865)
> at clojure.lang.RestFn.invoke(RestFn.java:408)
> at clojure.lang.Var.invoke(Var.java:379)
> at org.apache.storm.daemon.nimbus.(Unknown Source)
> Caused by: java.lang.ClassNotFoundException: 
> backtype.storm.security.auth.authorizer.SimpleACLAuthorizer
> at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
> at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
> at java.security.AccessController.doPrivileged(Native Method)
> at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
> at java.lang.Class.forName0(Native Method)
> at java.lang.Class.forName(Class.java:190)
> at 
> org.apache.storm.daemon.common$mk_authorization_handler.invoke(common.clj:412)
> at org.apache.storm.ui.core__init.load(Unknown Source)
> at org.apache.storm.ui.core__init.(Unknown Source)
> ... 35 more
> {code}
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/serveraction/upgrades/StormUpgradeKerberosDescriptorConfig.java
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/nonrolling-upgrade-2.5.xml
>  65a7880 
>   ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/upgrade-2.5.xml 
> bc8e9f9 
>   
> ambari-server/src/main/resources/stacks/HDP/2.4/upgrades/nonrolling-upgrade-2.5.xml
>  98e5038 
>   ambari-server/src/main/resou

Re: Review Request 51241: PREVIEW - RU: Storm components were stopped during RU and can not be started

2016-08-19 Thread Dmitro Lisnichenko 


> On Aug. 19, 2016, 6:16 p.m., Nate Cole wrote:
> > ambari-server/src/main/java/org/apache/ambari/server/serveraction/upgrades/StormUpgradeKerberosDescriptorConfig.java,
> >  line 81
> > 
> >
> > Probably use a common function for this (have to make it public:  
> > KerberosHelperImpl.getKerberosDescriptorUpdates(Cluster cluster)
> > 
> > Have to inject the KerberosHelper.

We would in any case need ArtifactDAO here (at least to write new value). 
Extracted a separate method


- Dmitro


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/51241/#review146182
---


On Aug. 19, 2016, 7:54 p.m., Dmitro Lisnichenko wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/51241/
> ---
> 
> (Updated Aug. 19, 2016, 7:54 p.m.)
> 
> 
> Review request for Ambari, Jonathan Hurley, Jayush Luniya, and Nate Cole.
> 
> 
> Bugs: AMBARI-18213
> https://issues.apache.org/jira/browse/AMBARI-18213
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> STR:
> # Install cluster 2.4.2.0-258 on Ambari 2.2.2.0
> # Enable HA
> # Enable security
> # Upgrade ambari to 240
> # Perform RU to 2.5.0.0-1208
> 
> Deeper study shows that kerberos descriptor json in database ("artifact" 
> table) still contains values and properties that are actual for 2.4 stack.
> So the issue workflow should look like:
> - Old stack version is installed
> - Kerberos descriptor gets saved to database
> - Security is enabled
> - Stack upgrade is performed
> - Keytab regeneration is performed, and it populates service config with 
> obsolete property values
> 
> The issue happens on "Stack upgrade is performed" step. We never update 
> kerberos descriptor json in database to correspond to a new stack.
> 
> From nimbus.out
> {code}Exception in thread "main" java.lang.ExceptionInInitializerError
> at java.lang.Class.forName0(Native Method)
> at java.lang.Class.forName(Class.java:270)
> at clojure.lang.RT.classForName(RT.java:2154)
> at clojure.lang.RT.classForName(RT.java:2163)
> at clojure.lang.RT.loadClassForName(RT.java:2182)
> at clojure.lang.RT.load(RT.java:436)
> at clojure.lang.RT.load(RT.java:412)
> at clojure.core$load$fn__5448.invoke(core.clj:5866)
> at clojure.core$load.doInvoke(core.clj:5865)
> at clojure.lang.RestFn.invoke(RestFn.java:408)
> at clojure.core$load_one.invoke(core.clj:5671)
> at clojure.core$load_lib$fn__5397.invoke(core.clj:5711)
> at clojure.core$load_lib.doInvoke(core.clj:5710)
> at clojure.lang.RestFn.applyTo(RestFn.java:142)
> at clojure.core$apply.invoke(core.clj:632)
> at clojure.core$load_libs.doInvoke(core.clj:5749)
> at clojure.lang.RestFn.applyTo(RestFn.java:137)
> at clojure.core$apply.invoke(core.clj:632)
> at clojure.core$require.doInvoke(core.clj:5832)
> at clojure.lang.RestFn.invoke(RestFn.java:408)
> at 
> org.apache.storm.daemon.nimbus$loading__5340__auto8560.invoke(nimbus.clj:16)
> at org.apache.storm.daemon.nimbus__init.load(Unknown Source)
> at org.apache.storm.daemon.nimbus__init.(Unknown Source)
> at java.lang.Class.forName0(Native Method)
> at java.lang.Class.forName(Class.java:270)
> at clojure.lang.RT.classForName(RT.java:2154)
> at clojure.lang.RT.classForName(RT.java:2163)
> at clojure.lang.RT.loadClassForName(RT.java:2182)
> at clojure.lang.RT.load(RT.java:436)
> at clojure.lang.RT.load(RT.java:412)
> at clojure.core$load$fn__5448.invoke(core.clj:5866)
> at clojure.core$load.doInvoke(core.clj:5865)
> at clojure.lang.RestFn.invoke(RestFn.java:408)
> at clojure.lang.Var.invoke(Var.java:379)
> at org.apache.storm.daemon.nimbus.(Unknown Source)
> Caused by: java.lang.ClassNotFoundException: 
> backtype.storm.security.auth.authorizer.SimpleACLAuthorizer
> at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
> at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
> at java.security.AccessController.doPrivileged(Native Method)
> at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
> at java.lang.Class.forName0(Native Method)
> at java.lang.Class.forName(Class.java:190)
> at 
> org.apache.storm.daemon.common$mk_authorization_handler.invoke(common.clj:412)
> at org.apache.storm.ui.core__init.load(Unknown Source)
> at org.apache.storm.ui.core__init.(Unknown Source)
> ... 35 more
> {code}
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/serveraction/upgrades/StormUpgradeKerberosDescriptorConfig.java
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/stacks/HDP/2.3

Re: Review Request 51241: PREVIEW - RU: Storm components were stopped during RU and can not be started

2016-08-19 Thread Dmitro Lisnichenko 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/51241/
---

(Updated Aug. 19, 2016, 7:54 p.m.)


Review request for Ambari, Jonathan Hurley, Jayush Luniya, and Nate Cole.


Changes
---

Refectored implementation according to comments


Bugs: AMBARI-18213
https://issues.apache.org/jira/browse/AMBARI-18213


Repository: ambari


Description
---

STR:
# Install cluster 2.4.2.0-258 on Ambari 2.2.2.0
# Enable HA
# Enable security
# Upgrade ambari to 240
# Perform RU to 2.5.0.0-1208

Deeper study shows that kerberos descriptor json in database ("artifact" table) 
still contains values and properties that are actual for 2.4 stack.
So the issue workflow should look like:
- Old stack version is installed
- Kerberos descriptor gets saved to database
- Security is enabled
- Stack upgrade is performed
- Keytab regeneration is performed, and it populates service config with 
obsolete property values

The issue happens on "Stack upgrade is performed" step. We never update 
kerberos descriptor json in database to correspond to a new stack.

>From nimbus.out
{code}Exception in thread "main" java.lang.ExceptionInInitializerError
at java.lang.Class.forName0(Native Method)
at java.lang.Class.forName(Class.java:270)
at clojure.lang.RT.classForName(RT.java:2154)
at clojure.lang.RT.classForName(RT.java:2163)
at clojure.lang.RT.loadClassForName(RT.java:2182)
at clojure.lang.RT.load(RT.java:436)
at clojure.lang.RT.load(RT.java:412)
at clojure.core$load$fn__5448.invoke(core.clj:5866)
at clojure.core$load.doInvoke(core.clj:5865)
at clojure.lang.RestFn.invoke(RestFn.java:408)
at clojure.core$load_one.invoke(core.clj:5671)
at clojure.core$load_lib$fn__5397.invoke(core.clj:5711)
at clojure.core$load_lib.doInvoke(core.clj:5710)
at clojure.lang.RestFn.applyTo(RestFn.java:142)
at clojure.core$apply.invoke(core.clj:632)
at clojure.core$load_libs.doInvoke(core.clj:5749)
at clojure.lang.RestFn.applyTo(RestFn.java:137)
at clojure.core$apply.invoke(core.clj:632)
at clojure.core$require.doInvoke(core.clj:5832)
at clojure.lang.RestFn.invoke(RestFn.java:408)
at 
org.apache.storm.daemon.nimbus$loading__5340__auto8560.invoke(nimbus.clj:16)
at org.apache.storm.daemon.nimbus__init.load(Unknown Source)
at org.apache.storm.daemon.nimbus__init.(Unknown Source)
at java.lang.Class.forName0(Native Method)
at java.lang.Class.forName(Class.java:270)
at clojure.lang.RT.classForName(RT.java:2154)
at clojure.lang.RT.classForName(RT.java:2163)
at clojure.lang.RT.loadClassForName(RT.java:2182)
at clojure.lang.RT.load(RT.java:436)
at clojure.lang.RT.load(RT.java:412)
at clojure.core$load$fn__5448.invoke(core.clj:5866)
at clojure.core$load.doInvoke(core.clj:5865)
at clojure.lang.RestFn.invoke(RestFn.java:408)
at clojure.lang.Var.invoke(Var.java:379)
at org.apache.storm.daemon.nimbus.(Unknown Source)
Caused by: java.lang.ClassNotFoundException: 
backtype.storm.security.auth.authorizer.SimpleACLAuthorizer
at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
at java.lang.Class.forName0(Native Method)
at java.lang.Class.forName(Class.java:190)
at 
org.apache.storm.daemon.common$mk_authorization_handler.invoke(common.clj:412)
at org.apache.storm.ui.core__init.load(Unknown Source)
at org.apache.storm.ui.core__init.(Unknown Source)
... 35 more
{code}


Diffs (updated)
-

  
ambari-server/src/main/java/org/apache/ambari/server/serveraction/upgrades/StormUpgradeKerberosDescriptorConfig.java
 PRE-CREATION 
  
ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/nonrolling-upgrade-2.5.xml
 65a7880 
  ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/upgrade-2.5.xml 
bc8e9f9 
  
ambari-server/src/main/resources/stacks/HDP/2.4/upgrades/nonrolling-upgrade-2.5.xml
 98e5038 
  ambari-server/src/main/resources/stacks/HDP/2.4/upgrades/upgrade-2.5.xml 
e886301 

Diff: https://reviews.apache.org/r/51241/diff/


Testing
---

As of now, testing on live cluster


Thanks,

Dmitro Lisnichenko

Re: Review Request 51241: PREVIEW - RU: Storm components were stopped during RU and can not be started

2016-08-19 Thread Jonathan Hurley 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/51241/#review146186
---


Fix it, then Ship it!




I think it would be nice to make this a re-usable class in the future for 
updating Kerberos descriptors. But since we don't have the immediate need nor 
the time, doing a one-off is fine.


ambari-server/src/main/java/org/apache/ambari/server/serveraction/upgrades/StormUpgradeKerberosDescriptorConfig.java
 (lines 81 - 89)


For easy of readability, you could just do a quick return if the condition 
isn't satisfied. Like:

```
if(null == artifactEntities)
  return
```

That way, we're not indenting 4x before getting to the logic.



ambari-server/src/main/java/org/apache/ambari/server/serveraction/upgrades/StormUpgradeKerberosDescriptorConfig.java
 (line 140)


StringUtils.isBlank()/isNotBlank()


- Jonathan Hurley


On Aug. 19, 2016, 10:43 a.m., Dmitro Lisnichenko wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/51241/
> ---
> 
> (Updated Aug. 19, 2016, 10:43 a.m.)
> 
> 
> Review request for Ambari, Jonathan Hurley, Jayush Luniya, and Nate Cole.
> 
> 
> Bugs: AMBARI-18213
> https://issues.apache.org/jira/browse/AMBARI-18213
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> STR:
> # Install cluster 2.4.2.0-258 on Ambari 2.2.2.0
> # Enable HA
> # Enable security
> # Upgrade ambari to 240
> # Perform RU to 2.5.0.0-1208
> 
> Deeper study shows that kerberos descriptor json in database ("artifact" 
> table) still contains values and properties that are actual for 2.4 stack.
> So the issue workflow should look like:
> - Old stack version is installed
> - Kerberos descriptor gets saved to database
> - Security is enabled
> - Stack upgrade is performed
> - Keytab regeneration is performed, and it populates service config with 
> obsolete property values
> 
> The issue happens on "Stack upgrade is performed" step. We never update 
> kerberos descriptor json in database to correspond to a new stack.
> 
> From nimbus.out
> {code}Exception in thread "main" java.lang.ExceptionInInitializerError
> at java.lang.Class.forName0(Native Method)
> at java.lang.Class.forName(Class.java:270)
> at clojure.lang.RT.classForName(RT.java:2154)
> at clojure.lang.RT.classForName(RT.java:2163)
> at clojure.lang.RT.loadClassForName(RT.java:2182)
> at clojure.lang.RT.load(RT.java:436)
> at clojure.lang.RT.load(RT.java:412)
> at clojure.core$load$fn__5448.invoke(core.clj:5866)
> at clojure.core$load.doInvoke(core.clj:5865)
> at clojure.lang.RestFn.invoke(RestFn.java:408)
> at clojure.core$load_one.invoke(core.clj:5671)
> at clojure.core$load_lib$fn__5397.invoke(core.clj:5711)
> at clojure.core$load_lib.doInvoke(core.clj:5710)
> at clojure.lang.RestFn.applyTo(RestFn.java:142)
> at clojure.core$apply.invoke(core.clj:632)
> at clojure.core$load_libs.doInvoke(core.clj:5749)
> at clojure.lang.RestFn.applyTo(RestFn.java:137)
> at clojure.core$apply.invoke(core.clj:632)
> at clojure.core$require.doInvoke(core.clj:5832)
> at clojure.lang.RestFn.invoke(RestFn.java:408)
> at 
> org.apache.storm.daemon.nimbus$loading__5340__auto8560.invoke(nimbus.clj:16)
> at org.apache.storm.daemon.nimbus__init.load(Unknown Source)
> at org.apache.storm.daemon.nimbus__init.(Unknown Source)
> at java.lang.Class.forName0(Native Method)
> at java.lang.Class.forName(Class.java:270)
> at clojure.lang.RT.classForName(RT.java:2154)
> at clojure.lang.RT.classForName(RT.java:2163)
> at clojure.lang.RT.loadClassForName(RT.java:2182)
> at clojure.lang.RT.load(RT.java:436)
> at clojure.lang.RT.load(RT.java:412)
> at clojure.core$load$fn__5448.invoke(core.clj:5866)
> at clojure.core$load.doInvoke(core.clj:5865)
> at clojure.lang.RestFn.invoke(RestFn.java:408)
> at clojure.lang.Var.invoke(Var.java:379)
> at org.apache.storm.daemon.nimbus.(Unknown Source)
> Caused by: java.lang.ClassNotFoundException: 
> backtype.storm.security.auth.authorizer.SimpleACLAuthorizer
> at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
> at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
> at java.security.AccessController.doPrivileged(Native Method)
> at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
> at java.lang.Class.forName0(Native Method)
> at java.lang.Class.forName(Class.java:190)
> at 
> org.apache.storm.daemon.common$mk_authorization_handler.invoke(common.clj:412)
> at org.apache.storm.ui.core__init.load(

Re: Review Request 51241: PREVIEW - RU: Storm components were stopped during RU and can not be started

2016-08-19 Thread Nate Cole 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/51241/#review146182
---




ambari-server/src/main/java/org/apache/ambari/server/serveraction/upgrades/StormUpgradeKerberosDescriptorConfig.java
 (line 81)


Probably use a common function for this (have to make it public:  
KerberosHelperImpl.getKerberosDescriptorUpdates(Cluster cluster)

Have to inject the KerberosHelper.


- Nate Cole


On Aug. 19, 2016, 10:43 a.m., Dmitro Lisnichenko wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/51241/
> ---
> 
> (Updated Aug. 19, 2016, 10:43 a.m.)
> 
> 
> Review request for Ambari, Jonathan Hurley, Jayush Luniya, and Nate Cole.
> 
> 
> Bugs: AMBARI-18213
> https://issues.apache.org/jira/browse/AMBARI-18213
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> STR:
> # Install cluster 2.4.2.0-258 on Ambari 2.2.2.0
> # Enable HA
> # Enable security
> # Upgrade ambari to 240
> # Perform RU to 2.5.0.0-1208
> 
> Deeper study shows that kerberos descriptor json in database ("artifact" 
> table) still contains values and properties that are actual for 2.4 stack.
> So the issue workflow should look like:
> - Old stack version is installed
> - Kerberos descriptor gets saved to database
> - Security is enabled
> - Stack upgrade is performed
> - Keytab regeneration is performed, and it populates service config with 
> obsolete property values
> 
> The issue happens on "Stack upgrade is performed" step. We never update 
> kerberos descriptor json in database to correspond to a new stack.
> 
> From nimbus.out
> {code}Exception in thread "main" java.lang.ExceptionInInitializerError
> at java.lang.Class.forName0(Native Method)
> at java.lang.Class.forName(Class.java:270)
> at clojure.lang.RT.classForName(RT.java:2154)
> at clojure.lang.RT.classForName(RT.java:2163)
> at clojure.lang.RT.loadClassForName(RT.java:2182)
> at clojure.lang.RT.load(RT.java:436)
> at clojure.lang.RT.load(RT.java:412)
> at clojure.core$load$fn__5448.invoke(core.clj:5866)
> at clojure.core$load.doInvoke(core.clj:5865)
> at clojure.lang.RestFn.invoke(RestFn.java:408)
> at clojure.core$load_one.invoke(core.clj:5671)
> at clojure.core$load_lib$fn__5397.invoke(core.clj:5711)
> at clojure.core$load_lib.doInvoke(core.clj:5710)
> at clojure.lang.RestFn.applyTo(RestFn.java:142)
> at clojure.core$apply.invoke(core.clj:632)
> at clojure.core$load_libs.doInvoke(core.clj:5749)
> at clojure.lang.RestFn.applyTo(RestFn.java:137)
> at clojure.core$apply.invoke(core.clj:632)
> at clojure.core$require.doInvoke(core.clj:5832)
> at clojure.lang.RestFn.invoke(RestFn.java:408)
> at 
> org.apache.storm.daemon.nimbus$loading__5340__auto8560.invoke(nimbus.clj:16)
> at org.apache.storm.daemon.nimbus__init.load(Unknown Source)
> at org.apache.storm.daemon.nimbus__init.(Unknown Source)
> at java.lang.Class.forName0(Native Method)
> at java.lang.Class.forName(Class.java:270)
> at clojure.lang.RT.classForName(RT.java:2154)
> at clojure.lang.RT.classForName(RT.java:2163)
> at clojure.lang.RT.loadClassForName(RT.java:2182)
> at clojure.lang.RT.load(RT.java:436)
> at clojure.lang.RT.load(RT.java:412)
> at clojure.core$load$fn__5448.invoke(core.clj:5866)
> at clojure.core$load.doInvoke(core.clj:5865)
> at clojure.lang.RestFn.invoke(RestFn.java:408)
> at clojure.lang.Var.invoke(Var.java:379)
> at org.apache.storm.daemon.nimbus.(Unknown Source)
> Caused by: java.lang.ClassNotFoundException: 
> backtype.storm.security.auth.authorizer.SimpleACLAuthorizer
> at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
> at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
> at java.security.AccessController.doPrivileged(Native Method)
> at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
> at java.lang.Class.forName0(Native Method)
> at java.lang.Class.forName(Class.java:190)
> at 
> org.apache.storm.daemon.common$mk_authorization_handler.invoke(common.clj:412)
> at org.apache.storm.ui.core__init.load(Unknown Source)
> at org.apache.storm.ui.core__init.(Unknown Source)
> ... 35 more
> {code}
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/serveraction/upgrades/StormUpgradeKerberosDescriptorConfig.java
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/nonrolling-upgrade-2.5.xml
>  65a7880 
>   ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/upgrade-2.5.xml 
> bc8e9f9 
>   
> ambari-server/src/main/resources/stacks/HDP/2.4/upgrade

Review Request 51241: PREVIEW - RU: Storm components were stopped during RU and can not be started

2016-08-19 Thread Dmitro Lisnichenko 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/51241/
---

Review request for Ambari, Jonathan Hurley, Jayush Luniya, and Nate Cole.


Bugs: AMBARI-18213
https://issues.apache.org/jira/browse/AMBARI-18213


Repository: ambari


Description
---

STR:
# Install cluster 2.4.2.0-258 on Ambari 2.2.2.0
# Enable HA
# Enable security
# Upgrade ambari to 240
# Perform RU to 2.5.0.0-1208

Deeper study shows that kerberos descriptor json in database ("artifact" table) 
still contains values and properties that are actual for 2.4 stack.
So the issue workflow should look like:
- Old stack version is installed
- Kerberos descriptor gets saved to database
- Security is enabled
- Stack upgrade is performed
- Keytab regeneration is performed, and it populates service config with 
obsolete property values

The issue happens on "Stack upgrade is performed" step. We never update 
kerberos descriptor json in database to correspond to a new stack.

>From nimbus.out
{code}Exception in thread "main" java.lang.ExceptionInInitializerError
at java.lang.Class.forName0(Native Method)
at java.lang.Class.forName(Class.java:270)
at clojure.lang.RT.classForName(RT.java:2154)
at clojure.lang.RT.classForName(RT.java:2163)
at clojure.lang.RT.loadClassForName(RT.java:2182)
at clojure.lang.RT.load(RT.java:436)
at clojure.lang.RT.load(RT.java:412)
at clojure.core$load$fn__5448.invoke(core.clj:5866)
at clojure.core$load.doInvoke(core.clj:5865)
at clojure.lang.RestFn.invoke(RestFn.java:408)
at clojure.core$load_one.invoke(core.clj:5671)
at clojure.core$load_lib$fn__5397.invoke(core.clj:5711)
at clojure.core$load_lib.doInvoke(core.clj:5710)
at clojure.lang.RestFn.applyTo(RestFn.java:142)
at clojure.core$apply.invoke(core.clj:632)
at clojure.core$load_libs.doInvoke(core.clj:5749)
at clojure.lang.RestFn.applyTo(RestFn.java:137)
at clojure.core$apply.invoke(core.clj:632)
at clojure.core$require.doInvoke(core.clj:5832)
at clojure.lang.RestFn.invoke(RestFn.java:408)
at 
org.apache.storm.daemon.nimbus$loading__5340__auto8560.invoke(nimbus.clj:16)
at org.apache.storm.daemon.nimbus__init.load(Unknown Source)
at org.apache.storm.daemon.nimbus__init.(Unknown Source)
at java.lang.Class.forName0(Native Method)
at java.lang.Class.forName(Class.java:270)
at clojure.lang.RT.classForName(RT.java:2154)
at clojure.lang.RT.classForName(RT.java:2163)
at clojure.lang.RT.loadClassForName(RT.java:2182)
at clojure.lang.RT.load(RT.java:436)
at clojure.lang.RT.load(RT.java:412)
at clojure.core$load$fn__5448.invoke(core.clj:5866)
at clojure.core$load.doInvoke(core.clj:5865)
at clojure.lang.RestFn.invoke(RestFn.java:408)
at clojure.lang.Var.invoke(Var.java:379)
at org.apache.storm.daemon.nimbus.(Unknown Source)
Caused by: java.lang.ClassNotFoundException: 
backtype.storm.security.auth.authorizer.SimpleACLAuthorizer
at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
at java.lang.Class.forName0(Native Method)
at java.lang.Class.forName(Class.java:190)
at 
org.apache.storm.daemon.common$mk_authorization_handler.invoke(common.clj:412)
at org.apache.storm.ui.core__init.load(Unknown Source)
at org.apache.storm.ui.core__init.(Unknown Source)
... 35 more
{code}


Diffs
-

  
ambari-server/src/main/java/org/apache/ambari/server/serveraction/upgrades/StormUpgradeKerberosDescriptorConfig.java
 PRE-CREATION 
  
ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/nonrolling-upgrade-2.5.xml
 65a7880 
  ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/upgrade-2.5.xml 
bc8e9f9 
  
ambari-server/src/main/resources/stacks/HDP/2.4/upgrades/nonrolling-upgrade-2.5.xml
 98e5038 
  ambari-server/src/main/resources/stacks/HDP/2.4/upgrades/upgrade-2.5.xml 
e886301 

Diff: https://reviews.apache.org/r/51241/diff/


Testing
---

As of now, testing on live cluster


Thanks,

Dmitro Lisnichenko