[
https://issues.apache.org/jira/browse/ATLAS-512?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15171779#comment-15171779
]
Hemanth Yamijala edited comment on ATLAS-512 at 2/29/16 1:59 PM:
-----------------------------------------------------------------
Uploaded a patch for review here: https://reviews.apache.org/r/44165/
Copying the description here for reference. Let's take code related
conversation on the review board.
This patch implements the modified option 1 discussed in this comment:
https://issues.apache.org/jira/browse/ATLAS-512?focusedCommentId=15167236&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-15167236
The changes are as follows:
* Introduces a build step that generates the JSON definitions of the reserved
models for Hive, Sqoop, Falcon and Storm and copies them into the binary
distribution into a well defined folder relative to ATLAS_HOME.
* Modifies the startup of Atlas server to load the JSON definitions and create
the types during load of DefaultMetadataService.
* Removes the hooks making calls to register types (thus removing the
dependency on Atlas service being available - which is the core issue being
addressed).
* Implements unit tests for testing the functionality.
One change not implemented is the clean-up of dependencies on Hooks for the
Atlas components. This seems a bit more involved than simply removing the
dependencies. I will take that up in a separate JIRA.
was (Author: yhemanth):
Uploaded a patch for review here: https://reviews.apache.org/r/44165/
Copying the description here for reference. Let's take code related
conversation on the review board.
The changes are as follows:
* Introduces a build step that generates the JSON definitions of the reserved
models for Hive, Sqoop, Falcon and Storm and copies them into the binary
distribution into a well defined folder relative to ATLAS_HOME.
* Modifies the startup of Atlas server to load the JSON definitions and create
the types during load of DefaultMetadataService.
* Removes the hooks making calls to register types (thus removing the
dependency on Atlas service being available - which is the core issue being
addressed).
* Implements unit tests for testing the functionality.
One change not implemented is the clean-up of dependencies on Hooks for the
Atlas components. This seems a bit more involved than simply removing the
dependencies. I will take that up in a separate JIRA.
> Decouple currently integrating components from availability of Atlas service
> for raising metadata events
> ---------------------------------------------------------------------------------------------------------
>
> Key: ATLAS-512
> URL: https://issues.apache.org/jira/browse/ATLAS-512
> Project: Atlas
> Issue Type: Sub-task
> Reporter: Hemanth Yamijala
> Assignee: Hemanth Yamijala
>
> The components that currently integrate with Atlas (Hive, Sqoop, Falcon,
> Storm) all communicate their metadata events using Kafka as a messaging
> layer. This effectively decouples these components from the Atlas server.
> However, all of these components have some initialization that checks if
> their respective models are registered with Atlas. For components that
> integrate on the server, like HiveServer2 and Falcon, this initialization is
> a one time check and hence, is manageable. Others like Sqoop, Storm and the
> Hive CLI are client side components and hence the initialization happens for
> every run or session of these components. Invoking the initialization (and
> the one time check) every time like this effectively means that the Atlas
> server should be always available.
> This JIRA is to try and remove this dependency and thus truly decouple these
> components.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
