[
https://issues.apache.org/jira/browse/ATLAS-3427?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ashutosh Mestry updated ATLAS-3427:
-----------------------------------
Description:
*Background*
Existing implementation for sending messages from the hooks relies on Kafka.
Which means that if Kafka is not available for any reasons, the messages do not
make it to Atlas. They are simply added to the _failed_._log_.
*Solution*
Introduce a mechanism which preserves messages if destination for the messages
is not reachable. This new mechanism will also have logic to retry these
messages.
was:
Currently if hooks are not able to reach Kafka, metadata updates are not posted
and received by Atlas service.
Add a mechanism to make Hooks fault tolerant.
> Atlas hooks enhancements for better fault-tolerance i.e. Kafka unavailability
> -----------------------------------------------------------------------------
>
> Key: ATLAS-3427
> URL: https://issues.apache.org/jira/browse/ATLAS-3427
> Project: Atlas
> Issue Type: Improvement
> Reporter: Nixon Rodrigues
> Assignee: Ashutosh Mestry
> Priority: Major
>
> *Background*
> Existing implementation for sending messages from the hooks relies on Kafka.
> Which means that if Kafka is not available for any reasons, the messages do
> not make it to Atlas. They are simply added to the _failed_._log_.
>
> *Solution*
> Introduce a mechanism which preserves messages if destination for the
> messages is not reachable. This new mechanism will also have logic to retry
> these messages.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
