-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/49676/
-----------------------------------------------------------
(Updated July 6, 2016, 6:48 p.m.)
Review request for Ambari, Madhan Neethiraj, Robert Levas, Sumit Mohanty,
Swapan Shridhar, and Suma Shivaprasad.
Changes
-------
Addressed review comments
Bugs: AMBARI-17573
https://issues.apache.org/jira/browse/AMBARI-17573
Repository: ambari
Description
-------
Currently, Atlas hooks that run in Hive, Falconm Storm, Sqoop processes
reference atlas-application.properties file from Atlas server config location -
/etc/atlas/conf/atlas-application.properties.
Not all properties in /etc/atlas/conf/atlas-application.properties are required
in hooks and some of these properties are sensitive enough not to expose them
to hooks/clients.
To address this concern:
1. atlas-application.properties should be added as a config section in each of
the host component's that run Atlas hook - Hive, Storm, Falcon, Sqoop
2. These new config sections will only include properties that are required to
the respective hooks
3. During initial deployment, Ambari will initialize these properties with
values in Atlas server configuration.
For each one of those services, create a config type called $
{service}-atlas-application.properties that will be saved to /etc/${service}
/conf/application.properties
These are the default values,
Falcon
atlas.hook.falcon.synchronous=false
atlas.hook.falcon.numRetries=3
atlas.hook.falcon.minThreads=5
atlas.hook.falcon.maxThreads=5
atlas.hook.falcon.keepAliveTime=10
atlas.hook.falcon.queueSize
Storm
atlas.hook.storm.numRetries=3
Hive
atlas.hook.hive.synchronous=false
atlas.hook.hive.numRetries=3
atlas.hook.hive.minThreads=5
atlas.hook.hive.maxThreads=5
atlas.hook.hive.keepAliveTime=10
atlas.hook.hive.queueSize=10000
Common for all hooks
atlas.kafka.zookeeper.connect=
atlas.kafka.bootstrap.servers=
atlas.kafka.zookeeper.session.timeout.ms=400
atlas.kafka.zookeeper.connection.timeout.ms=200
atlas.kafka.zookeeper.sync.time.ms=20
atlas.kafka.hook.group.id=atlas
atlas.notification.create.topics=true
atlas.notification.replicas=1
atlas.notification.topics=ATLAS_HOOK,ATLAS_ENTITIES
atlas.notification.kafka.service.principal=kafka/_h...@example.com
atlas.notification.kafka.keytab.location=/etc/security/keytabs/kafka.service.keytab
atlas.jaas.KafkaClient.loginModuleName =
com.sun.security.auth.module.Krb5LoginModule
atlas.jaas.KafkaClient.loginModuleControlFlag = required
atlas.jaas.KafkaClient.option.useKeyTab = true
atlas.jaas.KafkaClient.option.storeKey = true
atlas.jaas.KafkaClient.option.serviceName = kafka
atlas.jaas.KafkaClient.option.keyTab =
/etc/security/keytabs/atlas.service.keytab
atlas.jaas.KafkaClient.option.principal = atlas/_h...@example.com
atlas.cluster.name=
Diffs (updated)
-----
ambari-common/src/main/python/resource_management/libraries/functions/setup_atlas_hook.py
PRE-CREATION
ambari-server/src/main/resources/common-services/ATLAS/0.1.0.2.3/configuration/application-properties.xml
1437251
ambari-server/src/main/resources/common-services/FALCON/0.5.0.2.1/configuration/falcon-atlas-application.properties.xml
PRE-CREATION
ambari-server/src/main/resources/common-services/FALCON/0.5.0.2.1/metainfo.xml
602144b
ambari-server/src/main/resources/common-services/FALCON/0.5.0.2.1/package/scripts/falcon.py
c2f1f53
ambari-server/src/main/resources/common-services/FALCON/0.5.0.2.1/package/scripts/params_linux.py
fc9d8b9
ambari-server/src/main/resources/common-services/FALCON/0.5.0.2.1/package/scripts/setup_atlas_falcon.py
1dce515
ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/configuration/hive-atlas-application.properties.xml
PRE-CREATION
ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/metainfo.xml
273133a
ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/hcat.py
839ab04
ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/hive.py
ea2af62
ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/params_linux.py
17f7380
ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/setup_atlas_hive.py
d1bd8ea
ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/webhcat.py
816b6af
ambari-server/src/main/resources/common-services/SQOOP/1.4.4.2.0/metainfo.xml
e3aa5ef
ambari-server/src/main/resources/common-services/SQOOP/1.4.4.2.0/package/scripts/params_linux.py
b2a6802
ambari-server/src/main/resources/common-services/SQOOP/1.4.4.2.0/package/scripts/setup_atlas_sqoop.py
76c1cda
ambari-server/src/main/resources/common-services/SQOOP/1.4.4.2.0/package/scripts/sqoop.py
bac836c
ambari-server/src/main/resources/common-services/STORM/0.9.1/package/scripts/params_linux.py
fac6331
ambari-server/src/main/resources/common-services/STORM/0.9.1/package/scripts/setup_atlas_storm.py
76ca1d3
ambari-server/src/main/resources/common-services/STORM/0.9.1/package/scripts/storm.py
2d50767
ambari-server/src/main/resources/common-services/STORM/1.0.1/configuration/storm-atlas-application.properties.xml
PRE-CREATION
ambari-server/src/main/resources/common-services/STORM/1.0.1/metainfo.xml
3c54a1e
ambari-server/src/main/resources/stacks/HDP/2.5/services/ATLAS/configuration/application-properties.xml
5f3d603
Diff: https://reviews.apache.org/r/49676/diff/
Testing
-------
Verified on a live cluster with HDP 2.5 and Atlas, Hive, Storm, Sqoop, and
Storm.
Python unit tests passed,
----------------------------------------------------------------------
Total run:1094
Total errors:0
Total failures:0
OK
Thanks,
Alejandro Fernandez
