[
https://issues.apache.org/jira/browse/ATLAS-3427?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17391990#comment-17391990
]
ASF subversion and git services commented on ATLAS-3427:
--------------------------------------------------------
Commit 6afeef4cc102931a51b12b9d365b23787b302f19 in atlas's branch
refs/heads/branch-2.0 from Ashutosh Mestry
[ https://gitbox.apache.org/repos/asf?p=atlas.git;h=6afeef4 ]
ATLAS-3427: Atlas Hook Enhancements for improved resiliancy.
(cherry picked from commit de87bc5022627d82cb8e6048b6728e7028a4af25)
> Atlas hooks enhancements for better fault-tolerance i.e. Kafka unavailability
> -----------------------------------------------------------------------------
>
> Key: ATLAS-3427
> URL: https://issues.apache.org/jira/browse/ATLAS-3427
> Project: Atlas
> Issue Type: Improvement
> Reporter: Nixon Rodrigues
> Assignee: Ashutosh Mestry
> Priority: Major
>
> *Background*
> Existing implementation for sending messages from the hooks relies on Kafka.
> Which means that if Kafka is not available for any reasons, the messages do
> not make it to Atlas. They are simply added to the _failed_._log_.
> *Solution*
> Introduce a mechanism which preserves messages if destination for the
> messages is not reachable. This new mechanism will also have logic to retry
> these messages.
> *Requirements*
> The mechanism should be:
> * Transparent: Existing hooks should not have to rework their
> implementation.
> * Configurable: This should be introduced optionally. If hook chooses not to
> use this, it should fallback to existing implementation. The behavior should
> be configurable via properties.
> * High-performance: Using this mechanism should not introduce additional
> overhead. Ideally, there should not be additional serialization introduced.
> * Reliable: The new mechanism should relay these messages reliably. Existing
> message formats should be supported viz. plain, compressed, compress &
> multi-part.
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
