-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/50350/#review143541
-----------------------------------------------------------


Ship it!




Ship It!

- Tim Thorpe


On July 26, 2016, 1:22 a.m., Alejandro Fernandez wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/50350/
> -----------------------------------------------------------
> 
> (Updated July 26, 2016, 1:22 a.m.)
> 
> 
> Review request for Ambari, Di Li, Jonathan Hurley, Jayush Luniya, Madhan 
> Neethiraj, Nate Cole, Sumit Mohanty, Swapan Shridhar, Suma Shivaprasad, Sid 
> Wagle, and Tim Thorpe.
> 
> 
> Bugs: AMBARI-17782
>     https://issues.apache.org/jira/browse/AMBARI-17782
> 
> 
> Repository: ambari
> 
> 
> Description
> -------
> 
> To support Atlas in HDP 2.5, make several config changes:
> 
> /etc/hive/conf/atlas-application.properties:
> 
> * atlas.rest.address (add this property)
>  
> /etc/hive/conf/hive-site.xml:
> 
> * atlas.cluster.name (remove this property)
> * atlas.hook.hive.maxThreads (remove this property)
> * atlas.hook.hive.minThreads (remove this property)
> * atlas.rest.address (remove this property)
> 
> /etc/storm/conf/atlas-application.properties:
> 
> * atlas.rest.address (add this property)
>  
> /etc/storm/conf/storm.yaml:
> 
> * atlas.cluster.name (remove this property)
>  
> /etc/falcon/conf/atlas-application.properties:
> 
> * atlas.rest.address (add this property)
> * atlas.cluster.name (this value is empty; need to set to correct value)
>  
>  
> /etc/sqoop/conf/atlas-application.properties:
> 
> * atlas.jaas.KafkaClient.option.keyTab (remove this property)
> * atlas.jaas.KafkaClient.option.principal (remove this property)
> * atlas.jaas.KafkaClient.option.storeKey (remove this property)
> * atlas.jaas.KafkaClient.option.useKeyTab (remove this property)
> * atlas.jaas.KafkaClient.option.useTicketCache=true (add this property)
> * atlas.jaas.KafkaClient.option.renewTicket=true (add this property)
> * atlas.rest.address (add this property)
>  
> /etc/sqoop/conf/sqoop-site.xml:
> 
> * atlas.cluster.name (remove this property)
> 
> Also, there is no 'Custom sqoop-atlas-application.properties' section in Sqoop
> 
> 
> Diffs
> -----
> 
>   
> ambari-common/src/main/python/resource_management/libraries/functions/setup_atlas_hook.py
>  a384f69 
>   
> ambari-server/src/main/resources/common-services/ATLAS/0.1.0.2.3/package/scripts/params.py
>  f0a3f32 
>   
> ambari-server/src/main/resources/common-services/ATLAS/0.7.0.2.5/configuration/application-properties.xml
>  4305965 
>   
> ambari-server/src/main/resources/common-services/FALCON/0.5.0.2.1/package/scripts/falcon.py
>  6111c34 
>   
> ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/configuration/hive-atlas-application.properties.xml
>  0590244 
>   
> ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/configuration/hive-site.xml
>  c53dd39 
>   
> ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/metainfo.xml 
> ec14979 
>   
> ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/hcat.py
>  f53625c 
>   
> ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/hive.py
>  6b05134 
>   
> ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/webhcat.py
>  4f38055 
>   
> ambari-server/src/main/resources/common-services/SQOOP/1.4.4.2.0/configuration/sqoop-site.xml
>  d78c5be 
>   
> ambari-server/src/main/resources/common-services/SQOOP/1.4.4.2.0/package/scripts/sqoop.py
>  963d169 
>   
> ambari-server/src/main/resources/common-services/STORM/0.9.1/package/scripts/storm.py
>  d765ca3 
>   
> ambari-server/src/main/resources/common-services/STORM/1.0.1/configuration/storm-site.xml
>  3a3879b 
>   ambari-server/src/main/resources/stacks/HDP/2.3/services/stack_advisor.py 
> 8d5cdc9 
>   ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/config-upgrade.xml 
> bfdb3d3 
>   
> ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/nonrolling-upgrade-2.5.xml
>  a1b93e3 
>   ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/upgrade-2.5.xml 
> 96b1400 
>   ambari-server/src/main/resources/stacks/HDP/2.4/upgrades/config-upgrade.xml 
> b2cc1c4 
>   
> ambari-server/src/main/resources/stacks/HDP/2.4/upgrades/nonrolling-upgrade-2.5.xml
>  86e0964 
>   ambari-server/src/main/resources/stacks/HDP/2.4/upgrades/upgrade-2.5.xml 
> 2099958 
>   
> ambari-server/src/main/resources/stacks/HDP/2.5/services/HIVE/configuration/hive-atlas-application.properties.xml
>  PRE-CREATION 
>   ambari-server/src/main/resources/stacks/HDP/2.5/services/HIVE/metainfo.xml 
> 88c2aae 
>   
> ambari-server/src/main/resources/stacks/HDP/2.5/services/SQOOP/configuration/sqoop-site.xml
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/stacks/HDP/2.5/services/SQOOP/kerberos.json 
> PRE-CREATION 
>   ambari-server/src/main/resources/stacks/HDP/2.5/services/stack_advisor.py 
> 4972972 
>   
> ambari-server/src/test/java/org/apache/ambari/server/controller/internal/BlueprintConfigurationProcessorTest.java
>  7aa4a4e 
>   ambari-server/src/test/python/stacks/2.3/common/test_stack_advisor.py 
> 34bebe8 
>   ambari-server/src/test/python/stacks/2.5/common/test_stack_advisor.py 
> ebd2f2a 
> 
> Diff: https://reviews.apache.org/r/50350/diff/
> 
> 
> Testing
> -------
> 
> Verified the following test cases.
> 
> 
> 1. Install Ambari 2.4, HDP 2.5 along with Hive, Storm, Kafka, Sqoop and all 
> of their dependencies.
> Verify that none of the atlas.* configs exist nor the 
> $HOOK-atlas-application.properties file
> Ensure that hive-site and storm.yaml don't have any Atlas properties.
> 
> 2. After #1, add Atlas and verify that all of the hooks have 
> atlas.rest.address and atlas.cluster.name
> 
> 
> 3. After #2, kerberize the cluster and ensure Sqoop has the right configs in 
> its sqoop-atlas-application.properties file.
> 
> 
> 4. After #1, kerberize the cluster. This should add the application configs 
> to sqoop-atlas-application.properties in the DB, but the file will not be 
> saved to the local file system until Atlas is added.
> After this, add Atlas service (and its dependencies), and ensure that all of 
> the configs in /etc/sqoop/conf/sqoop-atlas-application.properties are correct.
> 
> 
> 5. Install Ambari 2.2.2 with HDP 2.4 along with Hive, Storm, Kafka, Sqoop, 
> and Atlas. Upgrade Ambari to 2.4.0 and ensure Atlas still works.
> Install bits for HDP 2.5, remove Atlas, and perform either an EU/RU to HDP 
> 2.5.
> This should remove atlas.cluster.name and atlas.rest.address from hive-site, 
> plus several other security-related properties from sqoop-site if the cluster 
> was kerberized.
> 
> 
> Thanks,
> 
> Alejandro Fernandez
> 
>

Reply via email to