[
https://issues.apache.org/jira/browse/ATLAS-3198?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Madhan Neethiraj updated ATLAS-3198:
------------------------------------
Fix Version/s: (was: 2.0.0)
trunk
> update/delete/truncate operations ATLAS_HOOK and ATLAS_ENTITIES improvements
> -----------------------------------------------------------------------------
>
> Key: ATLAS-3198
> URL: https://issues.apache.org/jira/browse/ATLAS-3198
> Project: Atlas
> Issue Type: Improvement
> Affects Versions: 2.0.0
> Reporter: Umesh Padashetty
> Assignee: Yu-Hsin Chang
> Priority: Critical
> Fix For: trunk
>
> Attachments: ATLAS_HOOK and ATLAS_ENTITIES messages.xlsx
>
>
> With the recent changes, we have now stopped ingesting simple insert
> operations since with this operation, there is no change in the metadata.
> Ideally, we should be doing similar improvements w.r.t. update, delete and
> truncate operations too.
> Currently, for update/delete/truncate, we are pushing messages in both
> ATLAS_HOOK as well as ATLAS_ENTITIES topic, which can be avoided.
> If we stop ingesting updates/deletes/truncates, it will further help us
> reduce the burden on Kafka and save CPU cycles
> I have created and attached an excel sheet, with the list of operations and
> the corresponding messages generated in ATLAS_HOOK and ATLAS_ENTITIES topic,
> along with suitable comments.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
