----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/50350/#review143457 -----------------------------------------------------------
ambari-common/src/main/python/resource_management/libraries/functions/setup_atlas_hook.py (line 97) <https://reviews.apache.org/r/50350/#comment209256> Needed to pass in service name to determine if it's a non-client service, which has slightly different configs. E.g., Sqoop. ambari-server/src/main/resources/common-services/ATLAS/0.1.0.2.3/package/scripts/params.py (line 140) <https://reviews.apache.org/r/50350/#comment209257> This value now comes from a config instead of being constructed on-the-fly. ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/metainfo.xml <https://reviews.apache.org/r/50350/#comment209258> This file was moved to HDP 2.5 stack. ambari-server/src/main/resources/common-services/STORM/1.0.1/configuration/storm-site.xml (line 118) <https://reviews.apache.org/r/50350/#comment209259> Deleted prop ambari-server/src/main/resources/stacks/HDP/2.5/services/SQOOP/configuration/sqoop-site.xml (line 28) <https://reviews.apache.org/r/50350/#comment209261> Delete this prop. ambari-server/src/main/resources/stacks/HDP/2.5/services/SQOOP/kerberos.json (line 7) <https://reviews.apache.org/r/50350/#comment209260> When Sqoop is kerberized, add these props. - Alejandro Fernandez On July 26, 2016, 1:22 a.m., Alejandro Fernandez wrote: > > ----------------------------------------------------------- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/50350/ > ----------------------------------------------------------- > > (Updated July 26, 2016, 1:22 a.m.) > > > Review request for Ambari, Di Li, Jonathan Hurley, Jayush Luniya, Madhan > Neethiraj, Nate Cole, Sumit Mohanty, Swapan Shridhar, Suma Shivaprasad, Sid > Wagle, and Tim Thorpe. > > > Bugs: AMBARI-17782 > https://issues.apache.org/jira/browse/AMBARI-17782 > > > Repository: ambari > > > Description > ------- > > To support Atlas in HDP 2.5, make several config changes: > > /etc/hive/conf/atlas-application.properties: > > * atlas.rest.address (add this property) > > /etc/hive/conf/hive-site.xml: > > * atlas.cluster.name (remove this property) > * atlas.hook.hive.maxThreads (remove this property) > * atlas.hook.hive.minThreads (remove this property) > * atlas.rest.address (remove this property) > > /etc/storm/conf/atlas-application.properties: > > * atlas.rest.address (add this property) > > /etc/storm/conf/storm.yaml: > > * atlas.cluster.name (remove this property) > > /etc/falcon/conf/atlas-application.properties: > > * atlas.rest.address (add this property) > * atlas.cluster.name (this value is empty; need to set to correct value) > > > /etc/sqoop/conf/atlas-application.properties: > > * atlas.jaas.KafkaClient.option.keyTab (remove this property) > * atlas.jaas.KafkaClient.option.principal (remove this property) > * atlas.jaas.KafkaClient.option.storeKey (remove this property) > * atlas.jaas.KafkaClient.option.useKeyTab (remove this property) > * atlas.jaas.KafkaClient.option.useTicketCache=true (add this property) > * atlas.jaas.KafkaClient.option.renewTicket=true (add this property) > * atlas.rest.address (add this property) > > /etc/sqoop/conf/sqoop-site.xml: > > * atlas.cluster.name (remove this property) > > Also, there is no 'Custom sqoop-atlas-application.properties' section in Sqoop > > > Diffs > ----- > > > ambari-common/src/main/python/resource_management/libraries/functions/setup_atlas_hook.py > a384f69 > > ambari-server/src/main/resources/common-services/ATLAS/0.1.0.2.3/package/scripts/params.py > f0a3f32 > > ambari-server/src/main/resources/common-services/ATLAS/0.7.0.2.5/configuration/application-properties.xml > 4305965 > > ambari-server/src/main/resources/common-services/FALCON/0.5.0.2.1/package/scripts/falcon.py > 6111c34 > > ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/configuration/hive-atlas-application.properties.xml > 0590244 > > ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/configuration/hive-site.xml > c53dd39 > > ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/metainfo.xml > ec14979 > > ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/hcat.py > f53625c > > ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/hive.py > 6b05134 > > ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/webhcat.py > 4f38055 > > ambari-server/src/main/resources/common-services/SQOOP/1.4.4.2.0/configuration/sqoop-site.xml > d78c5be > > ambari-server/src/main/resources/common-services/SQOOP/1.4.4.2.0/package/scripts/sqoop.py > 963d169 > > ambari-server/src/main/resources/common-services/STORM/0.9.1/package/scripts/storm.py > d765ca3 > > ambari-server/src/main/resources/common-services/STORM/1.0.1/configuration/storm-site.xml > 3a3879b > ambari-server/src/main/resources/stacks/HDP/2.3/services/stack_advisor.py > 8d5cdc9 > ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/config-upgrade.xml > bfdb3d3 > > ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/nonrolling-upgrade-2.5.xml > a1b93e3 > ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/upgrade-2.5.xml > 96b1400 > ambari-server/src/main/resources/stacks/HDP/2.4/upgrades/config-upgrade.xml > b2cc1c4 > > ambari-server/src/main/resources/stacks/HDP/2.4/upgrades/nonrolling-upgrade-2.5.xml > 86e0964 > ambari-server/src/main/resources/stacks/HDP/2.4/upgrades/upgrade-2.5.xml > 2099958 > > ambari-server/src/main/resources/stacks/HDP/2.5/services/HIVE/configuration/hive-atlas-application.properties.xml > PRE-CREATION > ambari-server/src/main/resources/stacks/HDP/2.5/services/HIVE/metainfo.xml > 88c2aae > > ambari-server/src/main/resources/stacks/HDP/2.5/services/SQOOP/configuration/sqoop-site.xml > PRE-CREATION > > ambari-server/src/main/resources/stacks/HDP/2.5/services/SQOOP/kerberos.json > PRE-CREATION > ambari-server/src/main/resources/stacks/HDP/2.5/services/stack_advisor.py > 4972972 > > ambari-server/src/test/java/org/apache/ambari/server/controller/internal/BlueprintConfigurationProcessorTest.java > 7aa4a4e > ambari-server/src/test/python/stacks/2.3/common/test_stack_advisor.py > 34bebe8 > ambari-server/src/test/python/stacks/2.5/common/test_stack_advisor.py > ebd2f2a > > Diff: https://reviews.apache.org/r/50350/diff/ > > > Testing > ------- > > Verified the following test cases. > > > 1. Install Ambari 2.4, HDP 2.5 along with Hive, Storm, Kafka, Sqoop and all > of their dependencies. > Verify that none of the atlas.* configs exist nor the > $HOOK-atlas-application.properties file > Ensure that hive-site and storm.yaml don't have any Atlas properties. > > 2. After #1, add Atlas and verify that all of the hooks have > atlas.rest.address and atlas.cluster.name > > > 3. After #2, kerberize the cluster and ensure Sqoop has the right configs in > its sqoop-atlas-application.properties file. > > > 4. After #1, kerberize the cluster. This should add the application configs > to sqoop-atlas-application.properties in the DB, but the file will not be > saved to the local file system until Atlas is added. > After this, add Atlas service (and its dependencies), and ensure that all of > the configs in /etc/sqoop/conf/sqoop-atlas-application.properties are correct. > > > 5. Install Ambari 2.2.2 with HDP 2.4 along with Hive, Storm, Kafka, Sqoop, > and Atlas. Upgrade Ambari to 2.4.0 and ensure Atlas still works. > Install bits for HDP 2.5, remove Atlas, and perform either an EU/RU to HDP > 2.5. > This should remove atlas.cluster.name and atlas.rest.address from hive-site, > plus several other security-related properties from sqoop-site if the cluster > was kerberized. > > > Thanks, > > Alejandro Fernandez > >
