-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/69392/
-----------------------------------------------------------
(Updated Jan. 4, 2019, 1:21 p.m.)
Review request for atlas, Ashutosh Mestry, Nixon Rodrigues, and Sarath
Subramanian.
Changes
-------
Added support for disabling/enabling relationship notifications by specifying
property.
Bugs: ATLAS-2810
https://issues.apache.org/jira/browse/ATLAS-2810
Repository: atlas
Description (updated)
-------
Notifications for relationship mutations.
To enable notifications, set property
"atlas.notification.relationships.enabled" to true.
If property is not specified, notifications will be disabled by default.
If enabled, this will add message to kafka topic when relationship is
added/updated/deleted.
Sample messages :
ADD
{"version":{"version":"1.0.0","versionParts":[1]},"msgCompressionKind":"NONE","msgSplitIdx":1,"msgSplitCount":1,"msgSourceIP":"127.0.1.1","msgCreatedBy":"","msgCreationTime":1544787083448,"message":{"type":"ENTITY_NOTIFICATION_V2","relationship":{"typeName":"hive_table_db","guid":"e4949325-562a-43bf-a980-c0c562c8759e","status":"ACTIVE","label":"__hive_table.db","end1":{"guid":"6da63233-d449-42d5-8449-b6ad243f6af2","typeName":"hive_table","uniqueAttributes":{"qualifiedName":"tbl2@qn"}},"end2":{"guid":"cea76431-af27-43ae-ab11-98a2ef989bb8","typeName":"hive_db","uniqueAttributes":{"qualifiedName":"db2@qn"}}},"operationType":"RELATIONSHIP_ADD","eventTime":1544787072298}}
UPDATE
{"version":{"version":"1.0.0","versionParts":[1]},"msgCompressionKind":"NONE","msgSplitIdx":1,"msgSplitCount":1,"msgSourceIP":"127.0.1.1","msgCreatedBy":"","msgCreationTime":1544787404692,"message":{"type":"ENTITY_NOTIFICATION_V2","relationship":{"typeName":"hive_table_db","guid":"e4949325-562a-43bf-a980-c0c562c8759e","status":"ACTIVE","label":"__hive_table.db","end1":{"guid":"6da63233-d449-42d5-8449-b6ad243f6af2","typeName":"hive_table","uniqueAttributes":{"qualifiedName":"tbl2@qn"}},"end2":{"guid":"cea76431-af27-43ae-ab11-98a2ef989bb8","typeName":"hive_db","uniqueAttributes":{"qualifiedName":"db2@qn"}}},"operationType":"RELATIONSHIP_UPDATE","eventTime":1544787404679}}
DELETE
{"version":{"version":"1.0.0","versionParts":[1]},"msgCompressionKind":"NONE","msgSplitIdx":1,"msgSplitCount":1,"msgSourceIP":"127.0.1.1","msgCreatedBy":"","msgCreationTime":1544787460081,"message":{"type":"ENTITY_NOTIFICATION_V2","relationship":{"typeName":"hive_table_db","guid":"e4949325-562a-43bf-a980-c0c562c8759e","status":"DELETED","label":"__hive_table.db","end1":{"guid":"6da63233-d449-42d5-8449-b6ad243f6af2","typeName":"hive_table","uniqueAttributes":{"qualifiedName":"tbl2@qn"}},"end2":{"guid":"cea76431-af27-43ae-ab11-98a2ef989bb8","typeName":"hive_db","uniqueAttributes":{"qualifiedName":"db2@qn"}}},"operationType":"RELATIONSHIP_DELETE","eventTime":1544787459708}}
Diffs (updated)
-----
intg/src/main/java/org/apache/atlas/AtlasConfiguration.java c5357f5
intg/src/main/java/org/apache/atlas/listener/EntityChangeListenerV2.java
cccf0d4
intg/src/main/java/org/apache/atlas/model/instance/AtlasRelationshipHeader.java
PRE-CREATION
intg/src/main/java/org/apache/atlas/model/notification/EntityNotification.java
1eae100
repository/src/main/java/org/apache/atlas/repository/audit/EntityAuditListenerV2.java
16d8879
repository/src/main/java/org/apache/atlas/repository/store/graph/v2/AtlasEntityChangeNotifier.java
ca3179a
repository/src/main/java/org/apache/atlas/repository/store/graph/v2/AtlasRelationshipStoreV2.java
86cc98c
repository/src/main/java/org/apache/atlas/repository/store/graph/v2/EntityGraphRetriever.java
066abc1
webapp/src/main/java/org/apache/atlas/notification/EntityNotificationListenerV2.java
ef9ebab
Diff: https://reviews.apache.org/r/69392/diff/10/
Changes: https://reviews.apache.org/r/69392/diff/9-10/
Testing
-------
Tested from REST client(Postman).
1) Added/updated/deleted relationship by hitting REST APIs(see details below)
2) Verified message is sent to kafka using kafka-console-consumer
./kafka-console-consumer.sh --bootstrap-server localhost:9027 --topic
ATLAS_ENTITIES --from-beginning
## Add relationship - create hive_db then create hive_table. This will create
new relationship
API - http://localhost:21000/api/atlas/v2/entity - POST
hive_db
request body -
{"entity":{"typeName":"hive_db","attributes":{"description":null,"name":"db2","owner":null,"qualifiedName":"db2@qn","replicatedFrom":[],"replicatedTo":[],"clusterName":"cluster0","location":null,"ownerType":null},"guid":-1},"referredEntities":{}}
hive_table
request body -
{"entity":{"typeName":"hive_table","attributes":{"description":null,"name":"tbl2","owner":null,"qualifiedName":"tbl2@qn","replicatedFrom":[],"replicatedTo":[],"aliases":[],"columns":[],"comment":null,"createTime":1544553000000,"db":{"guid":"cea76431-af27-43ae-ab11-98a2ef989bb8","typeName":"hive_db"},"lastAccessTime":1544553000000,"partitionKeys":[],"retention":null,"sd":null,"tableType":null,"temporary":null,"viewExpandedText":null,"viewOriginalText":null},"guid":-1},"referredEntities":{}}
## Update relationship -
API - http://localhost:21000/api/atlas/v2/relationship - PUT
request body -
{"typeName":"hive_table_db","guid":"8a8b8dc2-a163-42f0-b706-01c30cb0a688","provenanceType":0,"end1":{"guid":"6da63233-d449-42d5-8449-b6ad243f6af2","typeName":"hive_table"},"end2":{"guid":"cea76431-af27-43ae-ab11-98a2ef989bb8","typeName":"hive_db"},"label":"__hive_table.db","propagateTags":"NONE","status":"ACTIVE","createdBy":"admin","updatedBy":"admin","createTime":1542630658904,"updateTime":1542630658904,"version":0,"propagatedClassifications":[],"blockedPropagatedClassifications":[]}
## Delete relationship -
API -
http://localhost:21000/api/atlas/v2/relationship/guid/8a8b8dc2-a163-42f0-b706-01c30cb0a688
- DELETE
Thanks,
Nikhil Bonte
