[
https://issues.apache.org/jira/browse/ATLAS-515?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15320301#comment-15320301
]
Hemanth Yamijala commented on ATLAS-515:
----------------------------------------
Just picking this thread up once more. From some analysis, it appears that the
secure Kafka restrictions do impose a huge inconvenience in how this would
work. An option, therefore, is to convert this to a utility (rather than doing
it automatically during regular runtime work of the hooks or server). This
utility can either be invoked as the right user, or potentially the
authentication can happen in line by the utility itself when launched with the
right parameters (for e.g. it can do a kinit programmatically as a service
user).
The reason and way to run this utility should be documented. Running it can be
automated via deployment tools like Ambari. I will convert the patch to be like
this, by:
* Removing calling the AtlasTopicCreator from initialization in hooks, server
side
* Creating a shell script that launches this.
I still have a few questions about how the login should be done automatically
for secure Kafka deployments. Will update once I find out.
> Ability to initialize Kafka topics with more than 1 replica
> -----------------------------------------------------------
>
> Key: ATLAS-515
> URL: https://issues.apache.org/jira/browse/ATLAS-515
> Project: Atlas
> Issue Type: Sub-task
> Reporter: Hemanth Yamijala
> Assignee: Hemanth Yamijala
> Fix For: 0.7-incubating
>
> Attachments: ATLAS-515-poc.patch, ATLAS-515.patch
>
>
> Atlas uses two Kafka topics - ATLAS_HOOK and ATLAS_ENTITIES for communicating
> with its upstream and downstream dependencies. If not explicitly created,
> these topics are setup with a single replica. From a HA perspective, this
> becomes a problem. This JIRA is to request for a mechanism for creating
> topics with more than 1 replica.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
