-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/50350/
-----------------------------------------------------------

(Updated July 22, 2016, 6:47 p.m.)


Review request for Ambari, Madhan Neethiraj, Sumit Mohanty, and Suma 
Shivaprasad.


Bugs: AMBARI-17782
    https://issues.apache.org/jira/browse/AMBARI-17782


Repository: ambari


Description
-------

To support Atlas in HDP 2.5, make several config changes:

/etc/hive/conf/atlas-application.properties:

* atlas.rest.address (add this property)
 
/etc/hive/conf/hive-site.xml:

* atlas.cluster.name (remove this property)
* atlas.hook.hive.maxThreads (remove this property)
* atlas.hook.hive.minThreads (remove this property)
* atlas.rest.address (remove this property)

/etc/storm/conf/atlas-application.properties:

* atlas.rest.address (add this property)
 
/etc/storm/conf/storm.yaml:

* atlas.cluster.name (remove this property)
 
/etc/falcon/conf/atlas-application.properties:

* atlas.rest.address (add this property)
* atlas.cluster.name (this value is empty; need to set to correct value)
 
 
/etc/sqoop/conf/atlas-application.properties:

* atlas.jaas.KafkaClient.option.keyTab (remove this property)
* atlas.jaas.KafkaClient.option.principal (remove this property)
* atlas.jaas.KafkaClient.option.storeKey (remove this property)
* atlas.jaas.KafkaClient.option.useKeyTab (remove this property)
* atlas.jaas.KafkaClient.option.useTicketCache=true (add this property)
* atlas.jaas.KafkaClient.option.renewTicket=true (add this property)
* atlas.rest.address (add this property)
 
/etc/sqoop/conf/sqoop-site.xml:

* atlas.cluster.name (remove this property)

Also, there is no 'Custom sqoop-atlas-application.properties' section in Sqoop


Diffs
-----

  
ambari-common/src/main/python/resource_management/libraries/functions/setup_atlas_hook.py
 947d441 
  
ambari-server/src/main/java/org/apache/ambari/server/controller/internal/BlueprintConfigurationProcessor.java
 939e232 
  
ambari-server/src/main/resources/common-services/ATLAS/0.1.0.2.3/package/scripts/params.py
 e4d2392 
  
ambari-server/src/main/resources/common-services/ATLAS/0.7.0.2.5/configuration/application-properties.xml
 c3bcbc3 
  
ambari-server/src/main/resources/common-services/FALCON/0.5.0.2.1/package/scripts/falcon.py
 6111c34 
  
ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/hcat.py
 f53625c 
  
ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/hive.py
 6b05134 
  
ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/webhcat.py
 4f38055 
  
ambari-server/src/main/resources/common-services/SQOOP/1.4.4.2.0/configuration/sqoop-site.xml
 d78c5be 
  
ambari-server/src/main/resources/common-services/SQOOP/1.4.4.2.0/package/scripts/sqoop.py
 963d169 
  
ambari-server/src/main/resources/common-services/STORM/0.9.1/package/scripts/storm.py
 d765ca3 
  
ambari-server/src/main/resources/common-services/STORM/1.0.1/configuration/storm-site.xml
 d4ca263 
  ambari-server/src/main/resources/stacks/HDP/2.3/services/stack_advisor.py 
6081a04 
  ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/config-upgrade.xml 
bfdb3d3 
  
ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/nonrolling-upgrade-2.5.xml
 7d96155 
  ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/upgrade-2.5.xml 
09007a6 
  ambari-server/src/main/resources/stacks/HDP/2.4/upgrades/config-upgrade.xml 
bd1bef2 
  
ambari-server/src/main/resources/stacks/HDP/2.4/upgrades/nonrolling-upgrade-2.5.xml
 c82e938 
  ambari-server/src/main/resources/stacks/HDP/2.4/upgrades/upgrade-2.5.xml 
a9336bb 
  
ambari-server/src/main/resources/stacks/HDP/2.5/services/HIVE/configuration/hive-site.xml
 c0a9e64 
  
ambari-server/src/main/resources/stacks/HDP/2.5/services/SQOOP/configuration/sqoop-site.xml
 PRE-CREATION 
  ambari-server/src/main/resources/stacks/HDP/2.5/services/SQOOP/kerberos.json 
PRE-CREATION 
  ambari-server/src/main/resources/stacks/HDP/2.5/services/stack_advisor.py 
65c21fc 
  
ambari-server/src/test/java/org/apache/ambari/server/controller/internal/BlueprintConfigurationProcessorTest.java
 268bc29 

Diff: https://reviews.apache.org/r/50350/diff/


Testing (updated)
-------

Preview patch. Unit tests are broken right now.

Still need to test use cases.
1. Install Ambari 2.4, HDP 2.5 along with Hive, Storm, Kafka, Sqoop and all of 
their dependencies.
Verify that none of the atlas.* configs exist nor the 
$HOOK-atlas-application.properties file
Ensure that hive-site and storm.yaml don't have any Atlas properties.

2. After #1, add Atlas and verify that all of the hooks have atlas.rest.address 
and atlas.cluster.name


3. After #2, kerberize the cluster and ensure Sqoop has the right configs in 
its sqoop-atlas-application.properties file.


4. After #1, kerberize the cluster. This should add the application configs to 
sqoop-atlas-application.properties in the DB, but the file will not be saved to 
the local file system until Atlas is added.
After this, add Atlas service (and its dependencies), and ensure that all of 
the configs in /etc/sqoop/conf/sqoop-atlas-application.properties are correct.


5. Install Ambari 2.2.2 with HDP 2.4 along with Hive, Storm, Kafka, Sqoop, and 
Atlas. Upgrade Ambari to 2.4.0 and ensure Atlas still works.
Install bits for HDP 2.5, remove Atlas, and perform either an EU/RU to HDP 2.5.
This should remove atlas.cluster.name and atlas.rest.address from hive-site, 
plus several other security-related properties from sqoop-site if the cluster 
was kerberized.


Thanks,

Alejandro Fernandez

Reply via email to