[
https://issues.apache.org/jira/browse/AMBARI-17782?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alejandro Fernandez updated AMBARI-17782:
-----------------------------------------
Description:
To support Atlas in HDP 2.5, make several config changes:
/etc/hive/conf/atlas-application.properties:
* atlas.rest.address (add this property)
/etc/hive/conf/hive-site.xml:
* atlas.cluster.name (remove this property)
* atlas.hook.hive.maxThreads (remove this property)
* atlas.hook.hive.minThreads (remove this property)
* atlas.rest.address (remove this property)
/etc/storm/conf/atlas-application.properties:
* atlas.rest.address (add this property)
/etc/storm/conf/storm.yaml:
* atlas.cluster.name (remove this property)
/etc/falcon/conf/atlas-application.properties:
* atlas.rest.address (add this property)
* atlas.cluster.name (this value is empty; need to set to correct value)
/etc/sqoop/conf/atlas-application.properties:
* atlas.jaas.KafkaClient.option.keyTab (remove this property)
* atlas.jaas.KafkaClient.option.principal (remove this property)
* atlas.jaas.KafkaClient.option.storeKey (remove this property)
* atlas.jaas.KafkaClient.option.useKeyTab (remove this property)
* atlas.jaas.KafkaClient.option.useTicketCache=true (add this property)
* atlas.jaas.KafkaClient.option.renewTicket=true (add this property)
* atlas.rest.address (add this property)
/etc/sqoop/conf/sqoop-site.xml:
* atlas.cluster.name (remove this property)
Also, there is no 'Custom sqoop-atlas-application.properties' section in Sqoop
Test Plan:
1. Install Ambari 2.4, HDP 2.5 along with Hive, Storm, Kafka, Sqoop and all of
their dependencies.
Verify that none of the atlas.* configs exist nor the
$HOOK-atlas-application.properties file
Ensure that hive-site and storm.yaml don't have any Atlas properties.
2. After #1, add Atlas and verify that all of the hooks have atlas.rest.address
and atlas.cluster.name
3. After #2, kerberize the cluster and ensure Sqoop has the right configs in
its sqoop-atlas-application.properties file.
4. After #1, kerberize the cluster. This should add the application configs to
sqoop-atlas-application.properties in the DB, but the file will not be saved to
the local file system until Atlas is added.
After this, add Atlas service (and its dependencies), and ensure that all of
the configs in /etc/sqoop/conf/sqoop-atlas-application.properties are correct.
5. Install Ambari 2.2.2 with HDP 2.4 along with Hive, Storm, Kafka, Sqoop, and
Atlas. Upgrade Ambari to 2.4.0 and ensure Atlas still works.
Install bits for HDP 2.5, remove Atlas, and perform either an EU/RU to HDP 2.5.
This should remove atlas.cluster.name and atlas.rest.address from hive-site,
plus several other security-related properties from sqoop-site if the cluster
was kerberized.
was:
To support Atlas in HDP 2.5, make several config changes:
/etc/hive/conf/atlas-application.properties:
* atlas.rest.address (add this property)
/etc/hive/conf/hive-site.xml:
* atlas.cluster.name (remove this property)
* atlas.hook.hive.maxThreads (remove this property)
* atlas.hook.hive.minThreads (remove this property)
* atlas.rest.address (remove this property)
/etc/storm/conf/atlas-application.properties:
* atlas.rest.address (add this property)
/etc/storm/conf/storm.yaml:
* atlas.cluster.name (remove this property)
/etc/falcon/conf/atlas-application.properties:
* atlas.rest.address (add this property)
* atlas.cluster.name (this value is empty; need to set to correct value)
/etc/sqoop/conf/atlas-application.properties:
* atlas.jaas.KafkaClient.option.keyTab (remove this property)
* atlas.jaas.KafkaClient.option.principal (remove this property)
* atlas.jaas.KafkaClient.option.storeKey (remove this property)
* atlas.jaas.KafkaClient.option.useKeyTab (remove this property)
* atlas.jaas.KafkaClient.option.useTicketCache=true (add this property)
* atlas.jaas.KafkaClient.option.renewTicket=true (add this property)
* atlas.rest.address (add this property)
/etc/sqoop/conf/sqoop-site.xml:
* atlas.cluster.name (remove this property)
Also, there is no 'Custom sqoop-atlas-application.properties' section in Sqoop
> Config changes for Atlas in HDP 2.5 related to atlas.rest.address,
> atlas.cluster.name, etc
> ------------------------------------------------------------------------------------------
>
> Key: AMBARI-17782
> URL: https://issues.apache.org/jira/browse/AMBARI-17782
> Project: Ambari
> Issue Type: Bug
> Components: stacks
> Affects Versions: 2.4.0
> Reporter: Alejandro Fernandez
> Assignee: Alejandro Fernandez
> Priority: Critical
> Fix For: 2.4.0
>
> Attachments: AMBARI-17782.v1.patch
>
>
> To support Atlas in HDP 2.5, make several config changes:
> /etc/hive/conf/atlas-application.properties:
> * atlas.rest.address (add this property)
>
> /etc/hive/conf/hive-site.xml:
> * atlas.cluster.name (remove this property)
> * atlas.hook.hive.maxThreads (remove this property)
> * atlas.hook.hive.minThreads (remove this property)
> * atlas.rest.address (remove this property)
> /etc/storm/conf/atlas-application.properties:
> * atlas.rest.address (add this property)
>
> /etc/storm/conf/storm.yaml:
> * atlas.cluster.name (remove this property)
>
> /etc/falcon/conf/atlas-application.properties:
> * atlas.rest.address (add this property)
> * atlas.cluster.name (this value is empty; need to set to correct value)
>
> /etc/sqoop/conf/atlas-application.properties:
> * atlas.jaas.KafkaClient.option.keyTab (remove this property)
> * atlas.jaas.KafkaClient.option.principal (remove this property)
> * atlas.jaas.KafkaClient.option.storeKey (remove this property)
> * atlas.jaas.KafkaClient.option.useKeyTab (remove this property)
> * atlas.jaas.KafkaClient.option.useTicketCache=true (add this property)
> * atlas.jaas.KafkaClient.option.renewTicket=true (add this property)
> * atlas.rest.address (add this property)
>
> /etc/sqoop/conf/sqoop-site.xml:
> * atlas.cluster.name (remove this property)
> Also, there is no 'Custom sqoop-atlas-application.properties' section in Sqoop
> Test Plan:
> 1. Install Ambari 2.4, HDP 2.5 along with Hive, Storm, Kafka, Sqoop and all
> of their dependencies.
> Verify that none of the atlas.* configs exist nor the
> $HOOK-atlas-application.properties file
> Ensure that hive-site and storm.yaml don't have any Atlas properties.
> 2. After #1, add Atlas and verify that all of the hooks have
> atlas.rest.address and atlas.cluster.name
> 3. After #2, kerberize the cluster and ensure Sqoop has the right configs in
> its sqoop-atlas-application.properties file.
> 4. After #1, kerberize the cluster. This should add the application configs
> to sqoop-atlas-application.properties in the DB, but the file will not be
> saved to the local file system until Atlas is added.
> After this, add Atlas service (and its dependencies), and ensure that all of
> the configs in /etc/sqoop/conf/sqoop-atlas-application.properties are correct.
> 5. Install Ambari 2.2.2 with HDP 2.4 along with Hive, Storm, Kafka, Sqoop,
> and Atlas. Upgrade Ambari to 2.4.0 and ensure Atlas still works.
> Install bits for HDP 2.5, remove Atlas, and perform either an EU/RU to HDP
> 2.5.
> This should remove atlas.cluster.name and atlas.rest.address from hive-site,
> plus several other security-related properties from sqoop-site if the cluster
> was kerberized.
>
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
