Re: Review Request 47018: "ambari-server upgrade" shouldn't automatically add stack configs
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/47018/#review135643 --- Ship it! Ship It! - Nate Cole On May 26, 2016, 12:15 p.m., Dmitro Lisnichenko wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/47018/ > --- > > (Updated May 26, 2016, 12:15 p.m.) > > > Review request for Ambari, Alejandro Fernandez, Jonathan Hurley, Nate Cole, > and Sumit Mohanty. > > > Bugs: AMBARI-16272 > https://issues.apache.org/jira/browse/AMBARI-16272 > > > Repository: ambari > > > Description > --- > > Today, "ambari-server upgrade" will automatically add stack configs. > However, it also causes problems when default properties or properties with > default value such as "localhost" end up being added. > > This led to many bugs. E.g., cluster with NameNode HA shouldn't automatically > add dfs.namenode.secondary.http-address > > This logic today will even add new config types. E.g., add ranger-env even > though Ranger is not installed. If the customer then upgrades the stack from > HDP 2.2 to 2.3, and then adds Ranger, they can get the wrong configs. > If we change this behavior, it's good to do so in a major release such as 2.4 > > We add required xml tags/attributes to properties: > > prop_name > prop_val > > > > > we are going to enforce developers to explicitly specify what to do during > ambari/stack upgrade when adding any new config property > If any of stack configuration files does not pass XSD schema validation (that > contains this enforcement), then ambari-server unit tests will fail. > > > Diffs > - > > > ambari-server/src/main/java/org/apache/ambari/server/controller/internal/UpgradeResourceProvider.java > fb3ae69 > > ambari-server/src/main/java/org/apache/ambari/server/stack/StackManager.java > 8a352bd > > ambari-server/src/main/java/org/apache/ambari/server/state/ConfigHelper.java > a36df7b > > ambari-server/src/main/java/org/apache/ambari/server/state/PropertyInfo.java > 34b3ba1 > > ambari-server/src/main/java/org/apache/ambari/server/state/PropertyUpgradeBehavior.java > PRE-CREATION > > ambari-server/src/main/java/org/apache/ambari/server/upgrade/AbstractUpgradeCatalog.java > 2e857ed > > ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/configuration/hdfs-site.xml > 260fe65 > ambari-server/src/main/resources/configuration-schema.xsd PRE-CREATION > > ambari-server/src/main/resources/scripts/configurations-set-default-update-policy.sh > PRE-CREATION > > ambari-server/src/test/java/org/apache/ambari/server/controller/internal/UpgradeResourceProviderHDP22Test.java > a4a3108 > > ambari-server/src/test/java/org/apache/ambari/server/controller/internal/UpgradeResourceProviderTest.java > 8f53f6a > > ambari-server/src/test/java/org/apache/ambari/server/state/PropertyInfoTest.java > b11c5d8 > > ambari-server/src/test/java/org/apache/ambari/server/state/ServicePropertiesTest.java > PRE-CREATION > script.sh PRE-CREATION > > Diff: https://reviews.apache.org/r/47018/diff/ > > > Testing > --- > > Ran unit tests. Tried ambari-upgrade and stack upgrade, seems to work well. > > > Thanks, > > Dmitro Lisnichenko > >
Re: Review Request 47018: "ambari-server upgrade" shouldn't automatically add stack configs
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/47018/ --- (Updated May 26, 2016, 7:15 p.m.) Review request for Ambari, Alejandro Fernandez, Jonathan Hurley, Nate Cole, and Sumit Mohanty. Changes --- Uploaded patch that is going to be final. Did not include into review auto-generated changes to all 600 existing config xmls to avoid review becoming unviewable, will include them into commit. Summary (updated) - "ambari-server upgrade" shouldn't automatically add stack configs Bugs: AMBARI-16272 https://issues.apache.org/jira/browse/AMBARI-16272 Repository: ambari Description (updated) --- Today, "ambari-server upgrade" will automatically add stack configs. However, it also causes problems when default properties or properties with default value such as "localhost" end up being added. This led to many bugs. E.g., cluster with NameNode HA shouldn't automatically add dfs.namenode.secondary.http-address This logic today will even add new config types. E.g., add ranger-env even though Ranger is not installed. If the customer then upgrades the stack from HDP 2.2 to 2.3, and then adds Ranger, they can get the wrong configs. If we change this behavior, it's good to do so in a major release such as 2.4 We add required xml tags/attributes to properties: prop_name prop_val we are going to enforce developers to explicitly specify what to do during ambari/stack upgrade when adding any new config property If any of stack configuration files does not pass XSD schema validation (that contains this enforcement), then ambari-server unit tests will fail. Diffs (updated) - ambari-server/src/main/java/org/apache/ambari/server/controller/internal/UpgradeResourceProvider.java fb3ae69 ambari-server/src/main/java/org/apache/ambari/server/stack/StackManager.java 8a352bd ambari-server/src/main/java/org/apache/ambari/server/state/ConfigHelper.java a36df7b ambari-server/src/main/java/org/apache/ambari/server/state/PropertyInfo.java 34b3ba1 ambari-server/src/main/java/org/apache/ambari/server/state/PropertyUpgradeBehavior.java PRE-CREATION ambari-server/src/main/java/org/apache/ambari/server/upgrade/AbstractUpgradeCatalog.java 2e857ed ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/configuration/hdfs-site.xml 260fe65 ambari-server/src/main/resources/configuration-schema.xsd PRE-CREATION ambari-server/src/main/resources/scripts/configurations-set-default-update-policy.sh PRE-CREATION ambari-server/src/test/java/org/apache/ambari/server/controller/internal/UpgradeResourceProviderHDP22Test.java a4a3108 ambari-server/src/test/java/org/apache/ambari/server/controller/internal/UpgradeResourceProviderTest.java 8f53f6a ambari-server/src/test/java/org/apache/ambari/server/state/PropertyInfoTest.java b11c5d8 ambari-server/src/test/java/org/apache/ambari/server/state/ServicePropertiesTest.java PRE-CREATION script.sh PRE-CREATION Diff: https://reviews.apache.org/r/47018/diff/ Testing (updated) --- Ran unit tests. Tried ambari-upgrade and stack upgrade, seems to work well. Thanks, Dmitro Lisnichenko
