Radhika Kundam created ATLAS-4348:
-------------------------------------
Summary: Atlas-Kafka Hook : When a producer publishes messages to
multiple topics, the latest relationship is marked ACTIVE , rest are marked
DELETED
Key: ATLAS-4348
URL: https://issues.apache.org/jira/browse/ATLAS-4348
Project: Atlas
Issue Type: Task
Components: atlas-core
Affects Versions: 2.1.0
Reporter: Radhika Kundam
Assignee: Radhika Kundam
*Steps to reproduce the issue*:
Created 2 topics : test1, test2
Using console consumer produced messages to test1 and test2.
Relationship between console-producer-lineage and test1 is ACTIVE now.
Relationship between console-producer-lineage and test1 is set to DELETED when
messages are published to test2.
*Root cause of issue:*
When messages published on topic, Atlas hook receives partial_update request
with only one topic as output and by default partial update deletes
relationships between previous topics by considering the only topic(which is in
partial_update request) as the active.
*Proposed Solution:*
Introducing typedef option "appendRelationshipsOnPartialUpdate" which will hold
list of relationship attribute names. While Atlas startup, it will resolve
entity type definition and all the specified relationship attributes
definitions will be updated with typedef property "isAppendOnPartialUpdate"
only for that entity type.
With this solution, we can keep all the previous relationship entries active
along with latest relationship entry for the specific entity by providing
typedef options as below.
"typeDefOptions": \{ "appendRelationshipsOnPartialUpdate": "[\"inputs\",
\"outputs\"]"}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
