Ashutosh Mestry created ATLAS-4464:
--------------------------------------
Summary: Ingest: Improve Rate of Ingest
Key: ATLAS-4464
URL: https://issues.apache.org/jira/browse/ATLAS-4464
Project: Atlas
Issue Type: Improvement
Components: atlas-core
Affects Versions: trunk
Reporter: Ashutosh Mestry
Assignee: Ashutosh Mestry
Fix For: trunk
*Background*
Existing implementation of _NotificationHookConsumer_ has linear complexity for
ingestion. This has several impacts:
* Authorization policies will take longer to get enforced.
* Unpredictable wait times for metadata showing up in Atlas.
*Solution*
Implement a mechanism for processing messages such that:
* Determine dependencies within incoming messages.
* Dependent messages should get processed serially.
* Messages without dependencies are processed concurrently.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
