-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/49676/
-----------------------------------------------------------

(Updated July 6, 2016, 8:29 p.m.)


Review request for Ambari, Madhan Neethiraj, Robert Levas, Sumit Mohanty, 
Swapan Shridhar, and Suma Shivaprasad.


Changes
-------

Added addendum patch


Bugs: AMBARI-17573
    https://issues.apache.org/jira/browse/AMBARI-17573


Repository: ambari


Description
-------

Currently, Atlas hooks that run in Hive, Falconm Storm, Sqoop processes 
reference atlas-application.properties file from Atlas server config location - 
/etc/atlas/conf/atlas-application.properties.
Not all properties in /etc/atlas/conf/atlas-application.properties are required 
in hooks and some of these properties are sensitive enough not to expose them 
to hooks/clients.

To address this concern:
1. atlas-application.properties should be added as a config section in each of 
the host component's that run Atlas hook - Hive, Storm, Falcon, Sqoop
2. These new config sections will only include properties that are required to 
the respective hooks
3. During initial deployment, Ambari will initialize these properties with 
values in Atlas server configuration.
For each one of those services, create a config type called $
{service}-atlas-application.properties that will be saved to /etc/${service}
/conf/application.properties

These are the default values,

Falcon
atlas.hook.falcon.synchronous=false
atlas.hook.falcon.numRetries=3
atlas.hook.falcon.minThreads=5
atlas.hook.falcon.maxThreads=5
atlas.hook.falcon.keepAliveTime=10
atlas.hook.falcon.queueSize

Storm
atlas.hook.storm.numRetries=3

Hive
atlas.hook.hive.synchronous=false
atlas.hook.hive.numRetries=3
atlas.hook.hive.minThreads=5
atlas.hook.hive.maxThreads=5
atlas.hook.hive.keepAliveTime=10
atlas.hook.hive.queueSize=10000

Common for all hooks
atlas.kafka.zookeeper.connect=
atlas.kafka.bootstrap.servers=
atlas.kafka.zookeeper.session.timeout.ms=400
atlas.kafka.zookeeper.connection.timeout.ms=200
atlas.kafka.zookeeper.sync.time.ms=20
atlas.kafka.hook.group.id=atlas
atlas.notification.create.topics=true
atlas.notification.replicas=1
atlas.notification.topics=ATLAS_HOOK,ATLAS_ENTITIES
atlas.notification.kafka.service.principal=kafka/_h...@example.com
atlas.notification.kafka.keytab.location=/etc/security/keytabs/kafka.service.keytab
atlas.jaas.KafkaClient.loginModuleName = 
com.sun.security.auth.module.Krb5LoginModule
atlas.jaas.KafkaClient.loginModuleControlFlag = required
atlas.jaas.KafkaClient.option.useKeyTab = true
atlas.jaas.KafkaClient.option.storeKey = true
atlas.jaas.KafkaClient.option.serviceName = kafka
atlas.jaas.KafkaClient.option.keyTab = 
/etc/security/keytabs/atlas.service.keytab
atlas.jaas.KafkaClient.option.principal = atlas/_h...@example.com
atlas.cluster.name=


Diffs
-----

  
ambari-common/src/main/python/resource_management/libraries/functions/setup_atlas_hook.py
 PRE-CREATION 
  
ambari-server/src/main/resources/common-services/ATLAS/0.1.0.2.3/configuration/application-properties.xml
 1437251 
  
ambari-server/src/main/resources/common-services/FALCON/0.5.0.2.1/configuration/falcon-atlas-application.properties.xml
 PRE-CREATION 
  
ambari-server/src/main/resources/common-services/FALCON/0.5.0.2.1/metainfo.xml 
602144b 
  
ambari-server/src/main/resources/common-services/FALCON/0.5.0.2.1/package/scripts/falcon.py
 c2f1f53 
  
ambari-server/src/main/resources/common-services/FALCON/0.5.0.2.1/package/scripts/params_linux.py
 fc9d8b9 
  
ambari-server/src/main/resources/common-services/FALCON/0.5.0.2.1/package/scripts/setup_atlas_falcon.py
 1dce515 
  
ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/configuration/hive-atlas-application.properties.xml
 PRE-CREATION 
  ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/metainfo.xml 
273133a 
  
ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/hcat.py
 839ab04 
  
ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/hive.py
 ea2af62 
  
ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/params_linux.py
 17f7380 
  
ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/setup_atlas_hive.py
 d1bd8ea 
  
ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/webhcat.py
 816b6af 
  ambari-server/src/main/resources/common-services/SQOOP/1.4.4.2.0/metainfo.xml 
e3aa5ef 
  
ambari-server/src/main/resources/common-services/SQOOP/1.4.4.2.0/package/scripts/params_linux.py
 b2a6802 
  
ambari-server/src/main/resources/common-services/SQOOP/1.4.4.2.0/package/scripts/setup_atlas_sqoop.py
 76c1cda 
  
ambari-server/src/main/resources/common-services/SQOOP/1.4.4.2.0/package/scripts/sqoop.py
 bac836c 
  
ambari-server/src/main/resources/common-services/STORM/0.9.1/package/scripts/params_linux.py
 fac6331 
  
ambari-server/src/main/resources/common-services/STORM/0.9.1/package/scripts/setup_atlas_storm.py
 76ca1d3 
  
ambari-server/src/main/resources/common-services/STORM/0.9.1/package/scripts/storm.py
 2d50767 
  
ambari-server/src/main/resources/common-services/STORM/1.0.1/configuration/storm-atlas-application.properties.xml
 PRE-CREATION 
  ambari-server/src/main/resources/common-services/STORM/1.0.1/metainfo.xml 
3c54a1e 
  
ambari-server/src/main/resources/stacks/HDP/2.5/services/ATLAS/configuration/application-properties.xml
 5f3d603 

Diff: https://reviews.apache.org/r/49676/diff/


Testing
-------

Verified on a live cluster with HDP 2.5 and Atlas, Hive, Storm, Sqoop, and 
Storm.

Python unit tests passed,
----------------------------------------------------------------------
Total run:1094
Total errors:0
Total failures:0
OK


File Attachments (updated)
----------------

Addendum Patch
  
https://reviews.apache.org/media/uploaded/files/2016/07/06/8bc59fb8-426d-441b-b87d-d79728628ff7__AMBARI-17573.trunk.additional.patch


Thanks,

Alejandro Fernandez

Reply via email to