-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/71737/
-----------------------------------------------------------
(Updated Nov. 12, 2019, 11:54 p.m.)
Review request for atlas, Ashutosh Mestry, Madhan Neethiraj, and Sarath
Subramanian.
Changes
-------
Addressed the review comments
Bugs: ATLAS-3477
https://issues.apache.org/jira/browse/ATLAS-3477
Repository: atlas
Description
-------
As part of this solution users can purge the entities which are already soft
deleted. This will be useful as someone may be interested to purge some list of
soft deleted entities from Atlas. Due to governance compliances, only admin can
perform purge of entities.
Purge will also create separate Audit entry at HBase similar to delete entity
operation.
Diffs (updated)
-----
authorization/src/main/java/org/apache/atlas/authorize/AtlasPrivilege.java
59c596d66
client/client-v2/src/main/java/org/apache/atlas/AtlasClientV2.java 199d6bf58
intg/src/main/java/org/apache/atlas/listener/EntityChangeListenerV2.java
444167eae
intg/src/main/java/org/apache/atlas/model/audit/EntityAuditEventV2.java
e9cc7cd3d
intg/src/main/java/org/apache/atlas/model/instance/EntityMutationResponse.java
7ace00dff
intg/src/main/java/org/apache/atlas/model/instance/EntityMutations.java
daf4ca88c
repository/src/main/java/org/apache/atlas/repository/audit/EntityAuditListenerV2.java
43a9b842c
repository/src/main/java/org/apache/atlas/repository/store/graph/AtlasEntityStore.java
b94590bf1
repository/src/main/java/org/apache/atlas/repository/store/graph/v1/DeleteHandlerV1.java
d2544df4f
repository/src/main/java/org/apache/atlas/repository/store/graph/v2/AtlasEntityChangeNotifier.java
3389d2465
repository/src/main/java/org/apache/atlas/repository/store/graph/v2/AtlasEntityStoreV2.java
17a96493f
repository/src/main/java/org/apache/atlas/repository/store/graph/v2/AtlasGraphUtilsV2.java
4d57d8bc8
server-api/src/main/java/org/apache/atlas/RequestContext.java 3d2a18f0c
webapp/src/main/java/org/apache/atlas/notification/EntityNotificationListenerV2.java
48f0cd3c2
webapp/src/main/java/org/apache/atlas/web/resources/AdminResource.java
464d46f1e
webapp/src/test/java/org/apache/atlas/web/integration/EntityV2JerseyResourceIT.java
5d506bbc2
Diff: https://reviews.apache.org/r/71737/diff/2/
Changes: https://reviews.apache.org/r/71737/diff/1-2/
Testing
-------
IT has been added.
Validated manually by running below mentioned queries and REST:
a) create table thursday_07_11_1 (id int);
create table thursday_07_11_1_ctas as select * from thursday_07_11_1;
create table thursday_07_11_2 (id int);
create table thursday_07_11_2_ctas as select * from thursday_07_11_2;
b) The above will create two lineages. Go to Atlas UI and get the vertices for
atleast any two.
c) Using rest delete those two entities or use below query:
drop table thursday_07_11_1;
drop table thursday_07_11_1_ctas;
drop table thursday_07_11_2;
drop table thursday_07_11_2_ctas;
REST delete example:
DELETE
http://localhopst:31000/api/atlas/v2/entity/guid/7dd7fd9c-0185-41a5-aff0-4212fa7eb87c
d) Now use the same guids and call purge REST:
DELETE
http://localhost:31000/api/atlas/admin/purge/?guid=75983487-9e29-41ee-9bae-53f0a0b54c05&guid=98bee463-8810-402a-bcdc-3eb00bf50365
Check the response which will return all the successfully purged entities
e) For each guid check if the audit has separate new entry for purge. REST used:
GET
http://localhost:31000/api/atlas/v2/entity/7b6806c1-8daf-4e0f-8d10-672a192968f1/audit
Example Response:
[
{
"entityId": "7b6806c1-8daf-4e0f-8d10-672a192968f1",
"timestamp": 1573161789065,
"user": "admin",
"action": "ENTITY_PURGE",
"details": "Purged entity",
"eventKey": "7b6806c1-8daf-4e0f-8d10-672a192968f1:1573161789065:1",
"entity": null,
"type": null
},
{
"entityId": "7b6806c1-8daf-4e0f-8d10-672a192968f1",
"timestamp": 1573161729972,
"user": "impala",
"action": "ENTITY_DELETE",
"details": "Deleted entity",
"eventKey": "7b6806c1-8daf-4e0f-8d10-672a192968f1:1573161729972:1",
"entity": null,
"type": null
},
{
"entityId": "7b6806c1-8daf-4e0f-8d10-672a192968f1",
"timestamp": 1573161727305,
"user": "admin",
"action": "ENTITY_UPDATE",
"details": "Updated:
{\"typeName\":\"hive_column\",\"attributes\":{\"owner\":\"admin\",\"replicatedTo\":null,\"replicatedFrom\":null,\"qualifiedName\":\"default.thursday_07_11_5.id@cm\",\"name\":\"id\",\"description\":null,\"comment\":null,\"position\":0,\"type\":\"int\"},\"guid\":\"7b6806c1-8daf-4e0f-8d10-672a192968f1\",\"isIncomplete\":false,\"status\":\"ACTIVE\",\"createdBy\":\"impala\",\"updatedBy\":\"admin\",\"createTime\":1573161723180,\"updateTime\":1573161727305,\"version\":0,\"labels\":[]}",
"eventKey": "7b6806c1-8daf-4e0f-8d10-672a192968f1:1573161727305:0",
"entity": null,
"type": null
},
{
"entityId": "7b6806c1-8daf-4e0f-8d10-672a192968f1",
"timestamp": 1573161723180,
"user": "impala",
"action": "ENTITY_CREATE",
"details": "Created:
{\"typeName\":\"hive_column\",\"attributes\":{\"owner\":\"admin\",\"qualifiedName\":\"default.thursday_07_11_5.id@cm\",\"name\":\"id\",\"comment\":null,\"position\":0,\"type\":\"int\"},\"guid\":\"7b6806c1-8daf-4e0f-8d10-672a192968f1\",\"isIncomplete\":false,\"provenanceType\":0,\"version\":0,\"relationshipAttributes\":{\"table\":{\"guid\":\"c5ae4503-b498-493c-86b5-d8d909c6b16a\",\"typeName\":\"hive_table\",\"uniqueAttributes\":{\"qualifiedName\":\"default.thursday_07_11_5@cm\"},\"relationshipType\":\"hive_table_columns\"}},\"proxy\":false}",
"eventKey": "7b6806c1-8daf-4e0f-8d10-672a192968f1:1573161723180:2",
"entity": null,
"type": null
}
]
Thanks,
Sidharth Mishra
