[
https://issues.apache.org/jira/browse/ATLAS-3198?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16865889#comment-16865889
]
ASF subversion and git services commented on ATLAS-3198:
--------------------------------------------------------
Commit 4e1eeb4cdcd8bc0601564434176eb200b2811c2e in atlas's branch
refs/heads/master from Yu-Hsin Shih
[ https://gitbox.apache.org/repos/asf?p=atlas.git;h=4e1eeb4 ]
ATLAS-3198: Stop pushing message to ATLAS_HOOK for Truncate Table/Delete
data/Update data
Both ATLAS_HOOK and ATLAS_ENTITIES should stop receiving message from Truncate
Table/Delete data/Update data
Signed-off-by: Sarath Subramanian <sar...@apache.org>
> update/delete/truncate operations ATLAS_HOOK and ATLAS_ENTITIES improvements
> -----------------------------------------------------------------------------
>
> Key: ATLAS-3198
> URL: https://issues.apache.org/jira/browse/ATLAS-3198
> Project: Atlas
> Issue Type: Improvement
> Affects Versions: 2.0.0
> Reporter: Umesh Padashetty
> Assignee: Yu-Hsin Chang
> Priority: Critical
> Fix For: trunk
>
> Attachments: ATLAS-3198.patch, ATLAS_HOOK and ATLAS_ENTITIES
> messages.xlsx
>
>
> With the recent changes, we have now stopped ingesting simple insert
> operations since with this operation, there is no change in the metadata.
> Ideally, we should be doing similar improvements w.r.t. update, delete and
> truncate operations too.
> Currently, for update/delete/truncate, we are pushing messages in both
> ATLAS_HOOK as well as ATLAS_ENTITIES topic, which can be avoided.
> If we stop ingesting updates/deletes/truncates, it will further help us
> reduce the burden on Kafka and save CPU cycles
> I have created and attached an excel sheet, with the list of operations and
> the corresponding messages generated in ATLAS_HOOK and ATLAS_ENTITIES topic,
> along with suitable comments.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
