Abhay Kulkarni created RANGER-4130:
--------------------------------------
Summary: Improve performance of event processing in agsync by
optimizing number of commits to Kafka broker
Key: RANGER-4130
URL: https://issues.apache.org/jira/browse/RANGER-4130
Project: Ranger
Issue Type: Improvement
Components: tagsync
Reporter: Abhay Kulkarni
Assignee: Abhay Kulkarni
tagsync, currently, commits to Kafka every handled event after successfully
uploading to Ranger admin. The commits are synchronous and may impose
performance penalty.
This Jira is to update event handling in tagsync by optmizing on the commits to
Kafka while reducing the chances of Kafka delivering same message multiple
times to tagsync.
This Jira also fixes a potential issue if the Kafka queue contains only a large
number of notifications not handled by tagsync. In such cases, tagsync does not
do any commit to kafka, and that may cause back-pressure on Kafka queue.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
