-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/50350/#review143253
-----------------------------------------------------------




ambari-server/src/main/java/org/apache/ambari/server/controller/internal/BlueprintConfigurationProcessor.java
 (line 2495)
<https://reviews.apache.org/r/50350/#comment209032>

    If we don't support Atlas in HDP 2.3/2.4 with Ambari 2.4, then can remove 
this.



ambari-server/src/main/resources/stacks/HDP/2.3/services/stack_advisor.py (line 
229)
<https://reviews.apache.org/r/50350/#comment209034>

    Will remove these log statements



ambari-server/src/main/resources/stacks/HDP/2.3/services/stack_advisor.py (line 
244)
<https://reviews.apache.org/r/50350/#comment209035>

    The old code was looking up the wrong keys in the dictionary.


- Alejandro Fernandez


On July 22, 2016, 6:47 p.m., Alejandro Fernandez wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/50350/
> -----------------------------------------------------------
> 
> (Updated July 22, 2016, 6:47 p.m.)
> 
> 
> Review request for Ambari, Madhan Neethiraj, Sumit Mohanty, and Suma 
> Shivaprasad.
> 
> 
> Bugs: AMBARI-17782
>     https://issues.apache.org/jira/browse/AMBARI-17782
> 
> 
> Repository: ambari
> 
> 
> Description
> -------
> 
> To support Atlas in HDP 2.5, make several config changes:
> 
> /etc/hive/conf/atlas-application.properties:
> 
> * atlas.rest.address (add this property)
>  
> /etc/hive/conf/hive-site.xml:
> 
> * atlas.cluster.name (remove this property)
> * atlas.hook.hive.maxThreads (remove this property)
> * atlas.hook.hive.minThreads (remove this property)
> * atlas.rest.address (remove this property)
> 
> /etc/storm/conf/atlas-application.properties:
> 
> * atlas.rest.address (add this property)
>  
> /etc/storm/conf/storm.yaml:
> 
> * atlas.cluster.name (remove this property)
>  
> /etc/falcon/conf/atlas-application.properties:
> 
> * atlas.rest.address (add this property)
> * atlas.cluster.name (this value is empty; need to set to correct value)
>  
>  
> /etc/sqoop/conf/atlas-application.properties:
> 
> * atlas.jaas.KafkaClient.option.keyTab (remove this property)
> * atlas.jaas.KafkaClient.option.principal (remove this property)
> * atlas.jaas.KafkaClient.option.storeKey (remove this property)
> * atlas.jaas.KafkaClient.option.useKeyTab (remove this property)
> * atlas.jaas.KafkaClient.option.useTicketCache=true (add this property)
> * atlas.jaas.KafkaClient.option.renewTicket=true (add this property)
> * atlas.rest.address (add this property)
>  
> /etc/sqoop/conf/sqoop-site.xml:
> 
> * atlas.cluster.name (remove this property)
> 
> Also, there is no 'Custom sqoop-atlas-application.properties' section in Sqoop
> 
> 
> Diffs
> -----
> 
>   
> ambari-common/src/main/python/resource_management/libraries/functions/setup_atlas_hook.py
>  947d441 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/BlueprintConfigurationProcessor.java
>  939e232 
>   
> ambari-server/src/main/resources/common-services/ATLAS/0.1.0.2.3/package/scripts/params.py
>  e4d2392 
>   
> ambari-server/src/main/resources/common-services/ATLAS/0.7.0.2.5/configuration/application-properties.xml
>  c3bcbc3 
>   
> ambari-server/src/main/resources/common-services/FALCON/0.5.0.2.1/package/scripts/falcon.py
>  6111c34 
>   
> ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/hcat.py
>  f53625c 
>   
> ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/hive.py
>  6b05134 
>   
> ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/webhcat.py
>  4f38055 
>   
> ambari-server/src/main/resources/common-services/SQOOP/1.4.4.2.0/configuration/sqoop-site.xml
>  d78c5be 
>   
> ambari-server/src/main/resources/common-services/SQOOP/1.4.4.2.0/package/scripts/sqoop.py
>  963d169 
>   
> ambari-server/src/main/resources/common-services/STORM/0.9.1/package/scripts/storm.py
>  d765ca3 
>   
> ambari-server/src/main/resources/common-services/STORM/1.0.1/configuration/storm-site.xml
>  d4ca263 
>   ambari-server/src/main/resources/stacks/HDP/2.3/services/stack_advisor.py 
> 6081a04 
>   ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/config-upgrade.xml 
> bfdb3d3 
>   
> ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/nonrolling-upgrade-2.5.xml
>  7d96155 
>   ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/upgrade-2.5.xml 
> 09007a6 
>   ambari-server/src/main/resources/stacks/HDP/2.4/upgrades/config-upgrade.xml 
> bd1bef2 
>   
> ambari-server/src/main/resources/stacks/HDP/2.4/upgrades/nonrolling-upgrade-2.5.xml
>  c82e938 
>   ambari-server/src/main/resources/stacks/HDP/2.4/upgrades/upgrade-2.5.xml 
> a9336bb 
>   
> ambari-server/src/main/resources/stacks/HDP/2.5/services/HIVE/configuration/hive-site.xml
>  c0a9e64 
>   
> ambari-server/src/main/resources/stacks/HDP/2.5/services/SQOOP/configuration/sqoop-site.xml
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/stacks/HDP/2.5/services/SQOOP/kerberos.json 
> PRE-CREATION 
>   ambari-server/src/main/resources/stacks/HDP/2.5/services/stack_advisor.py 
> 65c21fc 
>   
> ambari-server/src/test/java/org/apache/ambari/server/controller/internal/BlueprintConfigurationProcessorTest.java
>  268bc29 
> 
> Diff: https://reviews.apache.org/r/50350/diff/
> 
> 
> Testing
> -------
> 
> Preview patch. Unit tests are broken right now.
> 
> Still need to test use cases.
> 1. Install Ambari 2.4, HDP 2.5 along with Hive, Storm, Kafka, Sqoop and all 
> of their dependencies.
> Verify that none of the atlas.* configs exist nor the 
> $HOOK-atlas-application.properties file
> Ensure that hive-site and storm.yaml don't have any Atlas properties.
> 
> 2. After #1, add Atlas and verify that all of the hooks have 
> atlas.rest.address and atlas.cluster.name
> 
> 
> 3. After #2, kerberize the cluster and ensure Sqoop has the right configs in 
> its sqoop-atlas-application.properties file.
> 
> 
> 4. After #1, kerberize the cluster. This should add the application configs 
> to sqoop-atlas-application.properties in the DB, but the file will not be 
> saved to the local file system until Atlas is added.
> After this, add Atlas service (and its dependencies), and ensure that all of 
> the configs in /etc/sqoop/conf/sqoop-atlas-application.properties are correct.
> 
> 
> 5. Install Ambari 2.2.2 with HDP 2.4 along with Hive, Storm, Kafka, Sqoop, 
> and Atlas. Upgrade Ambari to 2.4.0 and ensure Atlas still works.
> Install bits for HDP 2.5, remove Atlas, and perform either an EU/RU to HDP 
> 2.5.
> This should remove atlas.cluster.name and atlas.rest.address from hive-site, 
> plus several other security-related properties from sqoop-site if the cluster 
> was kerberized.
> 
> 
> Thanks,
> 
> Alejandro Fernandez
> 
>

Reply via email to