Re: Review Request 47018: [preview] "ambari-server upgrade" shouldn't automatically add stack configs

2016-05-25 Thread Dmitro Lisnichenko 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/47018/
---

(Updated May 25, 2016, 7:06 p.m.)


Review request for Ambari, Alejandro Fernandez, Jonathan Hurley, Nate Cole, and 
Sumit Mohanty.


Changes
---

Fixed Nate's comments


Bugs: AMBARI-16272
https://issues.apache.org/jira/browse/AMBARI-16272


Repository: ambari


Description
---

Today, "ambari-server upgrade" will automatically add stack configs.
However, it also causes problems when default properties or properties with 
default value such as "localhost" end up being added.

This led to many bugs. E.g., cluster with NameNode HA shouldn't automatically 
add dfs.namenode.secondary.http-address

This logic today will even add new config types. E.g., add ranger-env even 
though Ranger is not installed. If the customer then upgrades the stack from 
HDP 2.2 to 2.3, and then adds Ranger, they can get the wrong configs.
If we change this behavior, it's good to do so in a major release such as 2.4

We add required xml tags/attributes to properties:

  prop_name
  prop_val
  
  



Diffs (updated)
-

  
ambari-server/src/main/java/org/apache/ambari/server/controller/internal/UpgradeResourceProvider.java
 fb3ae69 
  ambari-server/src/main/java/org/apache/ambari/server/stack/StackManager.java 
8a352bd 
  ambari-server/src/main/java/org/apache/ambari/server/state/ConfigHelper.java 
854dd0b 
  ambari-server/src/main/java/org/apache/ambari/server/state/PropertyInfo.java 
34b3ba1 
  
ambari-server/src/main/java/org/apache/ambari/server/state/PropertyUpgradeBehavior.java
 PRE-CREATION 
  
ambari-server/src/main/java/org/apache/ambari/server/upgrade/AbstractUpgradeCatalog.java
 2e857ed 
  
ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/configuration/hdfs-log4j.xml
 6038de0 
  ambari-server/src/main/resources/configuration-schema.xsd PRE-CREATION 
  
ambari-server/src/main/resources/stacks/HDP/2.1/services/HDFS/configuration/hdfs-site.xml
 d112137 
  
ambari-server/src/main/resources/stacks/HDP/2.3/services/HDFS/configuration/hdfs-site.xml
 3a94b1c 
  
ambari-server/src/test/java/org/apache/ambari/server/controller/internal/UpgradeResourceProviderHDP22Test.java
 a4a3108 
  
ambari-server/src/test/java/org/apache/ambari/server/controller/internal/UpgradeResourceProviderTest.java
 8f53f6a 
  
ambari-server/src/test/java/org/apache/ambari/server/state/PropertyInfoTest.java
 b11c5d8 
  
ambari-server/src/test/java/org/apache/ambari/server/state/ServicePropertiesTest.java
 PRE-CREATION 
  script.sh PRE-CREATION 

Diff: https://reviews.apache.org/r/47018/diff/


Testing
---

patch preview for now. Feedback is welcome


Thanks,

Dmitro Lisnichenko

Re: Review Request 47018: [preview] "ambari-server upgrade" shouldn't automatically add stack configs

2016-05-24 Thread Nate Cole 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/47018/#review134634
---




ambari-server/src/main/resources/property-schema.xsd (lines 1 - 21)


Maybe this file should be called configuration-schema.xsd since the root 
element is "configuration"?


- Nate Cole


On May 24, 2016, 12:11 p.m., Dmitro Lisnichenko wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/47018/
> ---
> 
> (Updated May 24, 2016, 12:11 p.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Jonathan Hurley, Nate Cole, 
> and Sumit Mohanty.
> 
> 
> Bugs: AMBARI-16272
> https://issues.apache.org/jira/browse/AMBARI-16272
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Today, "ambari-server upgrade" will automatically add stack configs.
> However, it also causes problems when default properties or properties with 
> default value such as "localhost" end up being added.
> 
> This led to many bugs. E.g., cluster with NameNode HA shouldn't automatically 
> add dfs.namenode.secondary.http-address
> 
> This logic today will even add new config types. E.g., add ranger-env even 
> though Ranger is not installed. If the customer then upgrades the stack from 
> HDP 2.2 to 2.3, and then adds Ranger, they can get the wrong configs.
> If we change this behavior, it's good to do so in a major release such as 2.4
> 
> We add required xml tags/attributes to properties:
> 
>   prop_name
>   prop_val
>   
>   
> 
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/UpgradeResourceProvider.java
>  fb3ae69 
>   
> ambari-server/src/main/java/org/apache/ambari/server/stack/StackManager.java 
> 8a352bd 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/ConfigHelper.java 
> 854dd0b 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/PropertyInfo.java 
> 34b3ba1 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/PropertyUpgradeBehavior.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/upgrade/AbstractUpgradeCatalog.java
>  2e857ed 
>   
> ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/configuration/hdfs-log4j.xml
>  6038de0 
>   ambari-server/src/main/resources/property-schema.xsd PRE-CREATION 
>   
> ambari-server/src/main/resources/stacks/HDP/2.1/services/HDFS/configuration/hdfs-site.xml
>  d112137 
>   
> ambari-server/src/main/resources/stacks/HDP/2.3/services/HDFS/configuration/hdfs-site.xml
>  3a94b1c 
>   
> ambari-server/src/test/java/org/apache/ambari/server/controller/internal/UpgradeResourceProviderHDP22Test.java
>  a4a3108 
>   
> ambari-server/src/test/java/org/apache/ambari/server/controller/internal/UpgradeResourceProviderTest.java
>  8f53f6a 
>   
> ambari-server/src/test/java/org/apache/ambari/server/state/PropertyInfoTest.java
>  b11c5d8 
>   
> ambari-server/src/test/java/org/apache/ambari/server/state/ServicePropertiesTest.java
>  PRE-CREATION 
>   script.sh PRE-CREATION 
> 
> Diff: https://reviews.apache.org/r/47018/diff/
> 
> 
> Testing
> ---
> 
> patch preview for now. Feedback is welcome
> 
> 
> Thanks,
> 
> Dmitro Lisnichenko
> 
>

Re: Review Request 47018: [preview] "ambari-server upgrade" shouldn't automatically add stack configs

2016-05-24 Thread Nate Cole 


> On May 19, 2016, 10:20 a.m., Nate Cole wrote:
> > ambari-server/src/main/java/org/apache/ambari/server/stack/StackManager.java,
> >  line 298
> > 
> >
> > Make a constant or read it from XML (if you define XSD location within 
> > XML)
> 
> Dmitro Lisnichenko wrote:
> Defining relative path to xsd location withing xml works, but is not 
> applicable for all cases. There are few troubles (path would be different for 
> files within stacks/ and common-services/ dirs, and also Idea when running 
> unit tests fails to resolve relative path like 
> "../../../../../../property-schema.xsd" and falls back to 
> /property-schema.xsd . So I decided to go further with explicit validator
> 
> Nate Cole wrote:
> You wouldn't define a relative path, you would use just the name of the 
> XSD and use code to load that file as a stream.  See 
> VersionDefinitionXml.load(InputStream) to see how we enforce XSD on a version 
> definition file out of the 'resources' directory.  Works for production 
> (a-s/src/main/resources) and test (a-s/src/test/resources) code.

The code on the stack side for this would be changed for the unmarshaller in 
ConfigurationDirectory.parsePath() to use this strategy.


- Nate


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/47018/#review133941
---


On May 24, 2016, 12:11 p.m., Dmitro Lisnichenko wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/47018/
> ---
> 
> (Updated May 24, 2016, 12:11 p.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Jonathan Hurley, Nate Cole, 
> and Sumit Mohanty.
> 
> 
> Bugs: AMBARI-16272
> https://issues.apache.org/jira/browse/AMBARI-16272
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Today, "ambari-server upgrade" will automatically add stack configs.
> However, it also causes problems when default properties or properties with 
> default value such as "localhost" end up being added.
> 
> This led to many bugs. E.g., cluster with NameNode HA shouldn't automatically 
> add dfs.namenode.secondary.http-address
> 
> This logic today will even add new config types. E.g., add ranger-env even 
> though Ranger is not installed. If the customer then upgrades the stack from 
> HDP 2.2 to 2.3, and then adds Ranger, they can get the wrong configs.
> If we change this behavior, it's good to do so in a major release such as 2.4
> 
> We add required xml tags/attributes to properties:
> 
>   prop_name
>   prop_val
>   
>   
> 
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/UpgradeResourceProvider.java
>  fb3ae69 
>   
> ambari-server/src/main/java/org/apache/ambari/server/stack/StackManager.java 
> 8a352bd 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/ConfigHelper.java 
> 854dd0b 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/PropertyInfo.java 
> 34b3ba1 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/PropertyUpgradeBehavior.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/upgrade/AbstractUpgradeCatalog.java
>  2e857ed 
>   
> ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/configuration/hdfs-log4j.xml
>  6038de0 
>   ambari-server/src/main/resources/property-schema.xsd PRE-CREATION 
>   
> ambari-server/src/main/resources/stacks/HDP/2.1/services/HDFS/configuration/hdfs-site.xml
>  d112137 
>   
> ambari-server/src/main/resources/stacks/HDP/2.3/services/HDFS/configuration/hdfs-site.xml
>  3a94b1c 
>   
> ambari-server/src/test/java/org/apache/ambari/server/controller/internal/UpgradeResourceProviderHDP22Test.java
>  a4a3108 
>   
> ambari-server/src/test/java/org/apache/ambari/server/controller/internal/UpgradeResourceProviderTest.java
>  8f53f6a 
>   
> ambari-server/src/test/java/org/apache/ambari/server/state/PropertyInfoTest.java
>  b11c5d8 
>   
> ambari-server/src/test/java/org/apache/ambari/server/state/ServicePropertiesTest.java
>  PRE-CREATION 
>   script.sh PRE-CREATION 
> 
> Diff: https://reviews.apache.org/r/47018/diff/
> 
> 
> Testing
> ---
> 
> patch preview for now. Feedback is welcome
> 
> 
> Thanks,
> 
> Dmitro Lisnichenko
> 
>

Re: Review Request 47018: [preview] "ambari-server upgrade" shouldn't automatically add stack configs

2016-05-24 Thread Nate Cole 


> On May 19, 2016, 10:20 a.m., Nate Cole wrote:
> > ambari-server/src/main/java/org/apache/ambari/server/stack/StackManager.java,
> >  line 298
> > 
> >
> > Make a constant or read it from XML (if you define XSD location within 
> > XML)
> 
> Dmitro Lisnichenko wrote:
> Defining relative path to xsd location withing xml works, but is not 
> applicable for all cases. There are few troubles (path would be different for 
> files within stacks/ and common-services/ dirs, and also Idea when running 
> unit tests fails to resolve relative path like 
> "../../../../../../property-schema.xsd" and falls back to 
> /property-schema.xsd . So I decided to go further with explicit validator

You wouldn't define a relative path, you would use just the name of the XSD and 
use code to load that file as a stream.  See 
VersionDefinitionXml.load(InputStream) to see how we enforce XSD on a version 
definition file out of the 'resources' directory.  Works for production 
(a-s/src/main/resources) and test (a-s/src/test/resources) code.


- Nate


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/47018/#review133941
---


On May 24, 2016, 12:11 p.m., Dmitro Lisnichenko wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/47018/
> ---
> 
> (Updated May 24, 2016, 12:11 p.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Jonathan Hurley, Nate Cole, 
> and Sumit Mohanty.
> 
> 
> Bugs: AMBARI-16272
> https://issues.apache.org/jira/browse/AMBARI-16272
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Today, "ambari-server upgrade" will automatically add stack configs.
> However, it also causes problems when default properties or properties with 
> default value such as "localhost" end up being added.
> 
> This led to many bugs. E.g., cluster with NameNode HA shouldn't automatically 
> add dfs.namenode.secondary.http-address
> 
> This logic today will even add new config types. E.g., add ranger-env even 
> though Ranger is not installed. If the customer then upgrades the stack from 
> HDP 2.2 to 2.3, and then adds Ranger, they can get the wrong configs.
> If we change this behavior, it's good to do so in a major release such as 2.4
> 
> We add required xml tags/attributes to properties:
> 
>   prop_name
>   prop_val
>   
>   
> 
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/UpgradeResourceProvider.java
>  fb3ae69 
>   
> ambari-server/src/main/java/org/apache/ambari/server/stack/StackManager.java 
> 8a352bd 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/ConfigHelper.java 
> 854dd0b 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/PropertyInfo.java 
> 34b3ba1 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/PropertyUpgradeBehavior.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/upgrade/AbstractUpgradeCatalog.java
>  2e857ed 
>   
> ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/configuration/hdfs-log4j.xml
>  6038de0 
>   ambari-server/src/main/resources/property-schema.xsd PRE-CREATION 
>   
> ambari-server/src/main/resources/stacks/HDP/2.1/services/HDFS/configuration/hdfs-site.xml
>  d112137 
>   
> ambari-server/src/main/resources/stacks/HDP/2.3/services/HDFS/configuration/hdfs-site.xml
>  3a94b1c 
>   
> ambari-server/src/test/java/org/apache/ambari/server/controller/internal/UpgradeResourceProviderHDP22Test.java
>  a4a3108 
>   
> ambari-server/src/test/java/org/apache/ambari/server/controller/internal/UpgradeResourceProviderTest.java
>  8f53f6a 
>   
> ambari-server/src/test/java/org/apache/ambari/server/state/PropertyInfoTest.java
>  b11c5d8 
>   
> ambari-server/src/test/java/org/apache/ambari/server/state/ServicePropertiesTest.java
>  PRE-CREATION 
>   script.sh PRE-CREATION 
> 
> Diff: https://reviews.apache.org/r/47018/diff/
> 
> 
> Testing
> ---
> 
> patch preview for now. Feedback is welcome
> 
> 
> Thanks,
> 
> Dmitro Lisnichenko
> 
>

Re: Review Request 47018: [preview] "ambari-server upgrade" shouldn't automatically add stack configs

2016-05-24 Thread Dmitro Lisnichenko 


> On May 19, 2016, 5:20 p.m., Nate Cole wrote:
> > ambari-server/src/main/java/org/apache/ambari/server/stack/StackManager.java,
> >  line 298
> > 
> >
> > Make a constant or read it from XML (if you define XSD location within 
> > XML)

Defining relative path to xsd location withing xml works, but is not applicable 
for all cases. There are few troubles (path would be different for files within 
stacks/ and common-services/ dirs, and also Idea when running unit tests fails 
to resolve relative path like "../../../../../../property-schema.xsd" and falls 
back to /property-schema.xsd . So I decided to go further with explicit 
validator


- Dmitro


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/47018/#review133941
---


On May 24, 2016, 7:11 p.m., Dmitro Lisnichenko wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/47018/
> ---
> 
> (Updated May 24, 2016, 7:11 p.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Jonathan Hurley, Nate Cole, 
> and Sumit Mohanty.
> 
> 
> Bugs: AMBARI-16272
> https://issues.apache.org/jira/browse/AMBARI-16272
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Today, "ambari-server upgrade" will automatically add stack configs.
> However, it also causes problems when default properties or properties with 
> default value such as "localhost" end up being added.
> 
> This led to many bugs. E.g., cluster with NameNode HA shouldn't automatically 
> add dfs.namenode.secondary.http-address
> 
> This logic today will even add new config types. E.g., add ranger-env even 
> though Ranger is not installed. If the customer then upgrades the stack from 
> HDP 2.2 to 2.3, and then adds Ranger, they can get the wrong configs.
> If we change this behavior, it's good to do so in a major release such as 2.4
> 
> We add required xml tags/attributes to properties:
> 
>   prop_name
>   prop_val
>   
>   
> 
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/UpgradeResourceProvider.java
>  fb3ae69 
>   
> ambari-server/src/main/java/org/apache/ambari/server/stack/StackManager.java 
> 8a352bd 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/ConfigHelper.java 
> 854dd0b 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/PropertyInfo.java 
> 34b3ba1 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/PropertyUpgradeBehavior.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/upgrade/AbstractUpgradeCatalog.java
>  2e857ed 
>   
> ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/configuration/hdfs-log4j.xml
>  6038de0 
>   ambari-server/src/main/resources/property-schema.xsd PRE-CREATION 
>   
> ambari-server/src/main/resources/stacks/HDP/2.1/services/HDFS/configuration/hdfs-site.xml
>  d112137 
>   
> ambari-server/src/main/resources/stacks/HDP/2.3/services/HDFS/configuration/hdfs-site.xml
>  3a94b1c 
>   
> ambari-server/src/test/java/org/apache/ambari/server/controller/internal/UpgradeResourceProviderHDP22Test.java
>  a4a3108 
>   
> ambari-server/src/test/java/org/apache/ambari/server/controller/internal/UpgradeResourceProviderTest.java
>  8f53f6a 
>   
> ambari-server/src/test/java/org/apache/ambari/server/state/PropertyInfoTest.java
>  b11c5d8 
>   
> ambari-server/src/test/java/org/apache/ambari/server/state/ServicePropertiesTest.java
>  PRE-CREATION 
>   script.sh PRE-CREATION 
> 
> Diff: https://reviews.apache.org/r/47018/diff/
> 
> 
> Testing
> ---
> 
> patch preview for now. Feedback is welcome
> 
> 
> Thanks,
> 
> Dmitro Lisnichenko
> 
>

Re: Review Request 47018: [preview] "ambari-server upgrade" shouldn't automatically add stack configs

2016-05-24 Thread Dmitro Lisnichenko 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/47018/
---

(Updated May 24, 2016, 7:11 p.m.)


Review request for Ambari, Alejandro Fernandez, Jonathan Hurley, Nate Cole, and 
Sumit Mohanty.


Bugs: AMBARI-16272
https://issues.apache.org/jira/browse/AMBARI-16272


Repository: ambari


Description
---

Today, "ambari-server upgrade" will automatically add stack configs.
However, it also causes problems when default properties or properties with 
default value such as "localhost" end up being added.

This led to many bugs. E.g., cluster with NameNode HA shouldn't automatically 
add dfs.namenode.secondary.http-address

This logic today will even add new config types. E.g., add ranger-env even 
though Ranger is not installed. If the customer then upgrades the stack from 
HDP 2.2 to 2.3, and then adds Ranger, they can get the wrong configs.
If we change this behavior, it's good to do so in a major release such as 2.4

We add required xml tags/attributes to properties:

  prop_name
  prop_val
  
  



Diffs (updated)
-

  
ambari-server/src/main/java/org/apache/ambari/server/controller/internal/UpgradeResourceProvider.java
 fb3ae69 
  ambari-server/src/main/java/org/apache/ambari/server/stack/StackManager.java 
8a352bd 
  ambari-server/src/main/java/org/apache/ambari/server/state/ConfigHelper.java 
854dd0b 
  ambari-server/src/main/java/org/apache/ambari/server/state/PropertyInfo.java 
34b3ba1 
  
ambari-server/src/main/java/org/apache/ambari/server/state/PropertyUpgradeBehavior.java
 PRE-CREATION 
  
ambari-server/src/main/java/org/apache/ambari/server/upgrade/AbstractUpgradeCatalog.java
 2e857ed 
  
ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/configuration/hdfs-log4j.xml
 6038de0 
  ambari-server/src/main/resources/property-schema.xsd PRE-CREATION 
  
ambari-server/src/main/resources/stacks/HDP/2.1/services/HDFS/configuration/hdfs-site.xml
 d112137 
  
ambari-server/src/main/resources/stacks/HDP/2.3/services/HDFS/configuration/hdfs-site.xml
 3a94b1c 
  
ambari-server/src/test/java/org/apache/ambari/server/controller/internal/UpgradeResourceProviderHDP22Test.java
 a4a3108 
  
ambari-server/src/test/java/org/apache/ambari/server/controller/internal/UpgradeResourceProviderTest.java
 8f53f6a 
  
ambari-server/src/test/java/org/apache/ambari/server/state/PropertyInfoTest.java
 b11c5d8 
  
ambari-server/src/test/java/org/apache/ambari/server/state/ServicePropertiesTest.java
 PRE-CREATION 
  script.sh PRE-CREATION 

Diff: https://reviews.apache.org/r/47018/diff/


Testing
---

patch preview for now. Feedback is welcome


Thanks,

Dmitro Lisnichenko

Re: Review Request 47018: [preview] "ambari-server upgrade" shouldn't automatically add stack configs

2016-05-23 Thread Nate Cole 


> On May 19, 2016, 10:20 a.m., Nate Cole wrote:
> > ambari-server/src/main/resources/property-schema.xsd, lines 68-70
> > 
> >
> > All 3 required?
> 
> Dmitro Lisnichenko wrote:
> Yes, these are mandatory attributes for both on-ambari-upgrade and 
> on-stack-upgrade tags

Ok, just confirming.


- Nate


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/47018/#review133941
---


On May 18, 2016, 1:11 p.m., Dmitro Lisnichenko wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/47018/
> ---
> 
> (Updated May 18, 2016, 1:11 p.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Jonathan Hurley, Nate Cole, 
> and Sumit Mohanty.
> 
> 
> Bugs: AMBARI-16272
> https://issues.apache.org/jira/browse/AMBARI-16272
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Today, "ambari-server upgrade" will automatically add stack configs.
> However, it also causes problems when default properties or properties with 
> default value such as "localhost" end up being added.
> 
> This led to many bugs. E.g., cluster with NameNode HA shouldn't automatically 
> add dfs.namenode.secondary.http-address
> 
> This logic today will even add new config types. E.g., add ranger-env even 
> though Ranger is not installed. If the customer then upgrades the stack from 
> HDP 2.2 to 2.3, and then adds Ranger, they can get the wrong configs.
> If we change this behavior, it's good to do so in a major release such as 2.4
> 
> We add required xml tags/attributes to properties:
> 
>   prop_name
>   prop_val
>   
>   
> 
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/UpgradeResourceProvider.java
>  fb3ae69 
>   
> ambari-server/src/main/java/org/apache/ambari/server/stack/StackManager.java 
> 8a352bd 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/ConfigHelper.java 
> 854dd0b 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/PropertyInfo.java 
> 34b3ba1 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/PropertyUpgradeBehavior.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/upgrade/AbstractUpgradeCatalog.java
>  2e857ed 
>   ambari-server/src/main/resources/property-schema.xsd PRE-CREATION 
>   
> ambari-server/src/test/java/org/apache/ambari/server/controller/internal/UpgradeResourceProviderHDP22Test.java
>  a4a3108 
>   
> ambari-server/src/test/java/org/apache/ambari/server/controller/internal/UpgradeResourceProviderTest.java
>  8f53f6a 
>   
> ambari-server/src/test/java/org/apache/ambari/server/state/PropertyInfoTest.java
>  b11c5d8 
>   
> ambari-server/src/test/java/org/apache/ambari/server/state/ServicePropertiesTest.java
>  PRE-CREATION 
>   ambari-server/src/test/resources/propertiesFile.xsd PRE-CREATION 
>   script.sh PRE-CREATION 
> 
> Diff: https://reviews.apache.org/r/47018/diff/
> 
> 
> Testing
> ---
> 
> patch preview for now. Feedback is welcome
> 
> 
> Thanks,
> 
> Dmitro Lisnichenko
> 
>

Re: Review Request 47018: [preview] "ambari-server upgrade" shouldn't automatically add stack configs

2016-05-23 Thread Dmitro Lisnichenko 


> On May 19, 2016, 5:20 p.m., Nate Cole wrote:
> > ambari-server/src/main/resources/property-schema.xsd, lines 41-42
> > 
> >
> > Requiring both?  Both should be optional I would think so that we can 
> > add exceptions, not the-rule.

As we discussed in jira, we are going to have both tags mandatory in order to 
enforce devs to put proper upgrade policies when adding a new property.


> On May 19, 2016, 5:20 p.m., Nate Cole wrote:
> > ambari-server/src/main/resources/property-schema.xsd, lines 68-70
> > 
> >
> > All 3 required?

Yes, these are mandatory attributes for both on-ambari-upgrade and 
on-stack-upgrade tags


> On May 19, 2016, 5:20 p.m., Nate Cole wrote:
> > script.sh, lines 1-5
> > 
> >
> > What is this?

that was a draft script that batch modifies all xml files with properties to 
set up upgrade policies. Will not be present at a final patch


- Dmitro


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/47018/#review133941
---


On May 18, 2016, 8:11 p.m., Dmitro Lisnichenko wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/47018/
> ---
> 
> (Updated May 18, 2016, 8:11 p.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Jonathan Hurley, Nate Cole, 
> and Sumit Mohanty.
> 
> 
> Bugs: AMBARI-16272
> https://issues.apache.org/jira/browse/AMBARI-16272
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Today, "ambari-server upgrade" will automatically add stack configs.
> However, it also causes problems when default properties or properties with 
> default value such as "localhost" end up being added.
> 
> This led to many bugs. E.g., cluster with NameNode HA shouldn't automatically 
> add dfs.namenode.secondary.http-address
> 
> This logic today will even add new config types. E.g., add ranger-env even 
> though Ranger is not installed. If the customer then upgrades the stack from 
> HDP 2.2 to 2.3, and then adds Ranger, they can get the wrong configs.
> If we change this behavior, it's good to do so in a major release such as 2.4
> 
> We add required xml tags/attributes to properties:
> 
>   prop_name
>   prop_val
>   
>   
> 
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/UpgradeResourceProvider.java
>  fb3ae69 
>   
> ambari-server/src/main/java/org/apache/ambari/server/stack/StackManager.java 
> 8a352bd 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/ConfigHelper.java 
> 854dd0b 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/PropertyInfo.java 
> 34b3ba1 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/PropertyUpgradeBehavior.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/upgrade/AbstractUpgradeCatalog.java
>  2e857ed 
>   ambari-server/src/main/resources/property-schema.xsd PRE-CREATION 
>   
> ambari-server/src/test/java/org/apache/ambari/server/controller/internal/UpgradeResourceProviderHDP22Test.java
>  a4a3108 
>   
> ambari-server/src/test/java/org/apache/ambari/server/controller/internal/UpgradeResourceProviderTest.java
>  8f53f6a 
>   
> ambari-server/src/test/java/org/apache/ambari/server/state/PropertyInfoTest.java
>  b11c5d8 
>   
> ambari-server/src/test/java/org/apache/ambari/server/state/ServicePropertiesTest.java
>  PRE-CREATION 
>   ambari-server/src/test/resources/propertiesFile.xsd PRE-CREATION 
>   script.sh PRE-CREATION 
> 
> Diff: https://reviews.apache.org/r/47018/diff/
> 
> 
> Testing
> ---
> 
> patch preview for now. Feedback is welcome
> 
> 
> Thanks,
> 
> Dmitro Lisnichenko
> 
>

Re: Review Request 47018: [preview] "ambari-server upgrade" shouldn't automatically add stack configs

2016-05-19 Thread Nate Cole 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/47018/#review133941
---




ambari-server/src/main/java/org/apache/ambari/server/stack/StackManager.java 
(line 46)


bad import?



ambari-server/src/main/java/org/apache/ambari/server/stack/StackManager.java 
(line 298)


Make a constant or read it from XML (if you define XSD location within XML)



ambari-server/src/main/java/org/apache/ambari/server/stack/StackManager.java 
(lines 308 - 312)


formatting



ambari-server/src/main/java/org/apache/ambari/server/stack/StackManager.java 
(lines 315 - 317)


Seems awfully specific.  We know when we're parsing config XML, can this be 
done there?  Also, should we be defining the XML to use the XSD instead of 
back-checking it?



ambari-server/src/main/java/org/apache/ambari/server/state/PropertyUpgradeBehavior.java
 (line 25)


Javadoc



ambari-server/src/main/resources/property-schema.xsd (lines 41 - 42)


Requiring both?  Both should be optional I would think so that we can add 
exceptions, not the-rule.



ambari-server/src/main/resources/property-schema.xsd (lines 68 - 70)


All 3 required?



ambari-server/src/test/java/org/apache/ambari/server/controller/internal/UpgradeResourceProviderTest.java
 (line 77)


no *



script.sh (lines 1 - 5)


What is this?


- Nate Cole


On May 18, 2016, 1:11 p.m., Dmitro Lisnichenko wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/47018/
> ---
> 
> (Updated May 18, 2016, 1:11 p.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Jonathan Hurley, Nate Cole, 
> and Sumit Mohanty.
> 
> 
> Bugs: AMBARI-16272
> https://issues.apache.org/jira/browse/AMBARI-16272
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Today, "ambari-server upgrade" will automatically add stack configs.
> However, it also causes problems when default properties or properties with 
> default value such as "localhost" end up being added.
> 
> This led to many bugs. E.g., cluster with NameNode HA shouldn't automatically 
> add dfs.namenode.secondary.http-address
> 
> This logic today will even add new config types. E.g., add ranger-env even 
> though Ranger is not installed. If the customer then upgrades the stack from 
> HDP 2.2 to 2.3, and then adds Ranger, they can get the wrong configs.
> If we change this behavior, it's good to do so in a major release such as 2.4
> 
> We add required xml tags/attributes to properties:
> 
>   prop_name
>   prop_val
>   
>   
> 
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/UpgradeResourceProvider.java
>  fb3ae69 
>   
> ambari-server/src/main/java/org/apache/ambari/server/stack/StackManager.java 
> 8a352bd 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/ConfigHelper.java 
> 854dd0b 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/PropertyInfo.java 
> 34b3ba1 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/PropertyUpgradeBehavior.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/upgrade/AbstractUpgradeCatalog.java
>  2e857ed 
>   ambari-server/src/main/resources/property-schema.xsd PRE-CREATION 
>   
> ambari-server/src/test/java/org/apache/ambari/server/controller/internal/UpgradeResourceProviderHDP22Test.java
>  a4a3108 
>   
> ambari-server/src/test/java/org/apache/ambari/server/controller/internal/UpgradeResourceProviderTest.java
>  8f53f6a 
>   
> ambari-server/src/test/java/org/apache/ambari/server/state/PropertyInfoTest.java
>  b11c5d8 
>   
> ambari-server/src/test/java/org/apache/ambari/server/state/ServicePropertiesTest.java
>  PRE-CREATION 
>   ambari-server/src/test/resources/propertiesFile.xsd PRE-CREATION 
>   script.sh PRE-CREATION 
> 
> Diff: https://reviews.apache.org/r/47018/diff/
> 
> 
> Testing
> ---
> 
> patch preview for now. Feedback is welcome
> 
> 
> Thanks,
> 
> Dmitro Lisnichenko
> 
>

Re: Review Request 47018: [preview] "ambari-server upgrade" shouldn't automatically add stack configs

2016-05-18 Thread Dmitro Lisnichenko 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/47018/
---

(Updated May 18, 2016, 8:11 p.m.)


Review request for Ambari, Alejandro Fernandez, Jonathan Hurley, Nate Cole, and 
Sumit Mohanty.


Changes
---

As of now, a bit stuck trying to write correct path finding at 
ServicePropertiesTest .


Bugs: AMBARI-16272
https://issues.apache.org/jira/browse/AMBARI-16272


Repository: ambari


Description (updated)
---

Today, "ambari-server upgrade" will automatically add stack configs.
However, it also causes problems when default properties or properties with 
default value such as "localhost" end up being added.

This led to many bugs. E.g., cluster with NameNode HA shouldn't automatically 
add dfs.namenode.secondary.http-address

This logic today will even add new config types. E.g., add ranger-env even 
though Ranger is not installed. If the customer then upgrades the stack from 
HDP 2.2 to 2.3, and then adds Ranger, they can get the wrong configs.
If we change this behavior, it's good to do so in a major release such as 2.4

We add required xml tags/attributes to properties:

  prop_name
  prop_val
  
  



Diffs (updated)
-

  
ambari-server/src/main/java/org/apache/ambari/server/controller/internal/UpgradeResourceProvider.java
 fb3ae69 
  ambari-server/src/main/java/org/apache/ambari/server/stack/StackManager.java 
8a352bd 
  ambari-server/src/main/java/org/apache/ambari/server/state/ConfigHelper.java 
854dd0b 
  ambari-server/src/main/java/org/apache/ambari/server/state/PropertyInfo.java 
34b3ba1 
  
ambari-server/src/main/java/org/apache/ambari/server/state/PropertyUpgradeBehavior.java
 PRE-CREATION 
  
ambari-server/src/main/java/org/apache/ambari/server/upgrade/AbstractUpgradeCatalog.java
 2e857ed 
  ambari-server/src/main/resources/property-schema.xsd PRE-CREATION 
  
ambari-server/src/test/java/org/apache/ambari/server/controller/internal/UpgradeResourceProviderHDP22Test.java
 a4a3108 
  
ambari-server/src/test/java/org/apache/ambari/server/controller/internal/UpgradeResourceProviderTest.java
 8f53f6a 
  
ambari-server/src/test/java/org/apache/ambari/server/state/PropertyInfoTest.java
 b11c5d8 
  
ambari-server/src/test/java/org/apache/ambari/server/state/ServicePropertiesTest.java
 PRE-CREATION 
  ambari-server/src/test/resources/propertiesFile.xsd PRE-CREATION 
  script.sh PRE-CREATION 

Diff: https://reviews.apache.org/r/47018/diff/


Testing
---

patch preview for now. Feedback is welcome


Thanks,

Dmitro Lisnichenko