-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/49676/#review141060
-----------------------------------------------------------


Ship it!




Ship It!

- Robert Levas


On July 6, 2016, 2:48 p.m., Alejandro Fernandez wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/49676/
> -----------------------------------------------------------
> 
> (Updated July 6, 2016, 2:48 p.m.)
> 
> 
> Review request for Ambari, Madhan Neethiraj, Robert Levas, Sumit Mohanty, 
> Swapan Shridhar, and Suma Shivaprasad.
> 
> 
> Bugs: AMBARI-17573
>     https://issues.apache.org/jira/browse/AMBARI-17573
> 
> 
> Repository: ambari
> 
> 
> Description
> -------
> 
> Currently, Atlas hooks that run in Hive, Falconm Storm, Sqoop processes 
> reference atlas-application.properties file from Atlas server config location 
> - /etc/atlas/conf/atlas-application.properties.
> Not all properties in /etc/atlas/conf/atlas-application.properties are 
> required in hooks and some of these properties are sensitive enough not to 
> expose them to hooks/clients.
> 
> To address this concern:
> 1. atlas-application.properties should be added as a config section in each 
> of the host component's that run Atlas hook - Hive, Storm, Falcon, Sqoop
> 2. These new config sections will only include properties that are required 
> to the respective hooks
> 3. During initial deployment, Ambari will initialize these properties with 
> values in Atlas server configuration.
> For each one of those services, create a config type called $
> {service}-atlas-application.properties that will be saved to /etc/${service}
> /conf/application.properties
> 
> These are the default values,
> 
> Falcon
> atlas.hook.falcon.synchronous=false
> atlas.hook.falcon.numRetries=3
> atlas.hook.falcon.minThreads=5
> atlas.hook.falcon.maxThreads=5
> atlas.hook.falcon.keepAliveTime=10
> atlas.hook.falcon.queueSize
> 
> Storm
> atlas.hook.storm.numRetries=3
> 
> Hive
> atlas.hook.hive.synchronous=false
> atlas.hook.hive.numRetries=3
> atlas.hook.hive.minThreads=5
> atlas.hook.hive.maxThreads=5
> atlas.hook.hive.keepAliveTime=10
> atlas.hook.hive.queueSize=10000
> 
> Common for all hooks
> atlas.kafka.zookeeper.connect=
> atlas.kafka.bootstrap.servers=
> atlas.kafka.zookeeper.session.timeout.ms=400
> atlas.kafka.zookeeper.connection.timeout.ms=200
> atlas.kafka.zookeeper.sync.time.ms=20
> atlas.kafka.hook.group.id=atlas
> atlas.notification.create.topics=true
> atlas.notification.replicas=1
> atlas.notification.topics=ATLAS_HOOK,ATLAS_ENTITIES
> atlas.notification.kafka.service.principal=kafka/_h...@example.com
> atlas.notification.kafka.keytab.location=/etc/security/keytabs/kafka.service.keytab
> atlas.jaas.KafkaClient.loginModuleName = 
> com.sun.security.auth.module.Krb5LoginModule
> atlas.jaas.KafkaClient.loginModuleControlFlag = required
> atlas.jaas.KafkaClient.option.useKeyTab = true
> atlas.jaas.KafkaClient.option.storeKey = true
> atlas.jaas.KafkaClient.option.serviceName = kafka
> atlas.jaas.KafkaClient.option.keyTab = 
> /etc/security/keytabs/atlas.service.keytab
> atlas.jaas.KafkaClient.option.principal = atlas/_h...@example.com
> atlas.cluster.name=
> 
> 
> Diffs
> -----
> 
>   
> ambari-common/src/main/python/resource_management/libraries/functions/setup_atlas_hook.py
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/common-services/ATLAS/0.1.0.2.3/configuration/application-properties.xml
>  1437251 
>   
> ambari-server/src/main/resources/common-services/FALCON/0.5.0.2.1/configuration/falcon-atlas-application.properties.xml
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/common-services/FALCON/0.5.0.2.1/metainfo.xml
>  602144b 
>   
> ambari-server/src/main/resources/common-services/FALCON/0.5.0.2.1/package/scripts/falcon.py
>  c2f1f53 
>   
> ambari-server/src/main/resources/common-services/FALCON/0.5.0.2.1/package/scripts/params_linux.py
>  fc9d8b9 
>   
> ambari-server/src/main/resources/common-services/FALCON/0.5.0.2.1/package/scripts/setup_atlas_falcon.py
>  1dce515 
>   
> ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/configuration/hive-atlas-application.properties.xml
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/metainfo.xml 
> 273133a 
>   
> ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/hcat.py
>  839ab04 
>   
> ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/hive.py
>  ea2af62 
>   
> ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/params_linux.py
>  17f7380 
>   
> ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/setup_atlas_hive.py
>  d1bd8ea 
>   
> ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/webhcat.py
>  816b6af 
>   
> ambari-server/src/main/resources/common-services/SQOOP/1.4.4.2.0/metainfo.xml 
> e3aa5ef 
>   
> ambari-server/src/main/resources/common-services/SQOOP/1.4.4.2.0/package/scripts/params_linux.py
>  b2a6802 
>   
> ambari-server/src/main/resources/common-services/SQOOP/1.4.4.2.0/package/scripts/setup_atlas_sqoop.py
>  76c1cda 
>   
> ambari-server/src/main/resources/common-services/SQOOP/1.4.4.2.0/package/scripts/sqoop.py
>  bac836c 
>   
> ambari-server/src/main/resources/common-services/STORM/0.9.1/package/scripts/params_linux.py
>  fac6331 
>   
> ambari-server/src/main/resources/common-services/STORM/0.9.1/package/scripts/setup_atlas_storm.py
>  76ca1d3 
>   
> ambari-server/src/main/resources/common-services/STORM/0.9.1/package/scripts/storm.py
>  2d50767 
>   
> ambari-server/src/main/resources/common-services/STORM/1.0.1/configuration/storm-atlas-application.properties.xml
>  PRE-CREATION 
>   ambari-server/src/main/resources/common-services/STORM/1.0.1/metainfo.xml 
> 3c54a1e 
>   
> ambari-server/src/main/resources/stacks/HDP/2.5/services/ATLAS/configuration/application-properties.xml
>  5f3d603 
> 
> Diff: https://reviews.apache.org/r/49676/diff/
> 
> 
> Testing
> -------
> 
> Verified on a live cluster with HDP 2.5 and Atlas, Hive, Storm, Sqoop, and 
> Storm.
> 
> Python unit tests passed,
> ----------------------------------------------------------------------
> Total run:1094
> Total errors:0
> Total failures:0
> OK
> 
> 
> Thanks,
> 
> Alejandro Fernandez
> 
>

Reply via email to