> On Dec. 5, 2019, 12:39 a.m., Madhan Neethiraj wrote: > > repository/src/main/java/org/apache/atlas/repository/store/graph/v1/DeleteHandlerV1.java > > Lines 1057 (patched) > > <https://reviews.apache.org/r/71791/diff/7/?file=2182064#file2182064line1095> > > > > This will cause 'DELETED' entities to be skipped even though purge is > > in progress. This isn't correct. Please review and update. > > > > final String guid = AtlasGraphUtilsV2.getIdFromVertex(vertex); > > > > ret = guid == null; > > > > if (!ret) { > > if (reqContext.isPurgeRequested()) { > > ret = requestContext.isPurgedEntity(guid) || getState(vertex) == > > ACTIVE; > > } else { > > ret = requestContext.isDeletedEntity(guid) || getState(vertex) == > > DELETED; > > } > > }
As the code for purge, hard delete is same, the requestContext still stores the purged entities as part of deleteEntity List. > On Dec. 5, 2019, 12:39 a.m., Madhan Neethiraj wrote: > > webapp/src/main/java/org/apache/atlas/web/resources/AdminResource.java > > Lines 436 (patched) > > <https://reviews.apache.org/r/71791/diff/7/?file=2182093#file2182093line461> > > > > Audit generation should be done inside entityStore.purgeByIds(), in the > > same graph transaction. Avoid spilling such details into REST layer. As I earlier replied to your previous comment, Moving auditing to inside AtlasEntityStore.purgeByIds() is causing circular Injection issue. AtlasAuditService depends on DataAccess and DataAccess depends on AtlasEntityStore. So AtlasEntityStore can't depends on AtlasAuditService again. KIndly suggest is there any other option to avoid this circular injection issue. - Sidharth ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/71791/#review218929 ----------------------------------------------------------- On Dec. 4, 2019, 9:57 p.m., Sidharth Mishra wrote: > > ----------------------------------------------------------- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/71791/ > ----------------------------------------------------------- > > (Updated Dec. 4, 2019, 9:57 p.m.) > > > Review request for atlas, Ashutosh Mestry, Madhan Neethiraj, and Sarath > Subramanian. > > > Bugs: ATLAS-3518 > https://issues.apache.org/jira/browse/ATLAS-3518 > > > Repository: atlas > > > Description > ------- > > Right now for purge, delete entity, etc. at Atlas we create audit entry at > HBase. User can go to entity and click on audit to see the audit information > for that particular entity. But if user purge one entity there will not be > any vertex at Janus graph and user will not able to get the purge audit which > is important for governance purpose. > > As part of fix, implemented Atlas Audit Framework which is being used for > Purge operation and can be extended for other operations. In Future once we > change the AtlasAudit type as super type for Import export as well we can use > this framework for all Atlas operations. > > admin/audit rest has been added which will return all the Atlas audit > operations including import export at Atlas. > > > Diffs > ----- > > addons/models/0000-Area0/0010-base_model.json 2f5fdaf14 > client/client-v2/src/main/java/org/apache/atlas/AtlasClientV2.java > 8c0a6407c > dashboardv2/public/js/utils/Enums.js 58542ec59 > dashboardv3/public/js/utils/Enums.js 58542ec59 > intg/src/main/java/org/apache/atlas/model/audit/AtlasAuditEntry.java > PRE-CREATION > > intg/src/main/java/org/apache/atlas/model/instance/EntityMutationResponse.java > b448d510b > intg/src/main/java/org/apache/atlas/model/typedef/AtlasBaseTypeDef.java > eb8634533 > > repository/src/main/java/org/apache/atlas/repository/audit/AtlasAuditService.java > PRE-CREATION > > repository/src/main/java/org/apache/atlas/repository/ogm/AtlasAuditEntryDTO.java > PRE-CREATION > > repository/src/main/java/org/apache/atlas/repository/store/graph/v1/DeleteHandlerV1.java > c9ed79750 > repository/src/test/java/org/apache/atlas/TestModules.java 94b08a187 > repository/src/test/java/org/apache/atlas/glossary/GlossaryServiceTest.java > be1698fb4 > repository/src/test/java/org/apache/atlas/query/BasicTestSetup.java > 02f78b369 > > repository/src/test/java/org/apache/atlas/repository/audit/AdminPurgeTest.java > PRE-CREATION > > repository/src/test/java/org/apache/atlas/repository/audit/AtlasAuditServiceTest.java > PRE-CREATION > > repository/src/test/java/org/apache/atlas/repository/impexp/AtlasServerServiceTest.java > 78865c569 > > repository/src/test/java/org/apache/atlas/repository/impexp/ExportImportAuditServiceTest.java > ba7a8a04b > > repository/src/test/java/org/apache/atlas/repository/impexp/ExportImportTestBase.java > 925b2a007 > > repository/src/test/java/org/apache/atlas/repository/impexp/ExportIncrementalTest.java > 4d438525a > > repository/src/test/java/org/apache/atlas/repository/impexp/ExportServiceTest.java > db8c6c83c > > repository/src/test/java/org/apache/atlas/repository/impexp/ExportSkipLineageTest.java > 25e0a5362 > > repository/src/test/java/org/apache/atlas/repository/impexp/HdfsPathEntityCreatorTest.java > 1863b8d74 > > repository/src/test/java/org/apache/atlas/repository/impexp/ImportReactivateTableTest.java > d0c06a107 > > repository/src/test/java/org/apache/atlas/repository/impexp/ImportServiceTest.java > c14850f43 > > repository/src/test/java/org/apache/atlas/repository/impexp/ImportTransformsShaperTest.java > 78fdaca8d > > repository/src/test/java/org/apache/atlas/repository/impexp/IncrementalExportEntityProviderTest.java > 42b63535b > > repository/src/test/java/org/apache/atlas/repository/impexp/RelationshipAttributesExtractorTest.java > a1b512fb9 > > repository/src/test/java/org/apache/atlas/repository/impexp/ReplicationEntityAttributeTest.java > 7a1ed18cb > > repository/src/test/java/org/apache/atlas/repository/impexp/StartEntityFetchByExportRequestTest.java > 48b958271 > > repository/src/test/java/org/apache/atlas/repository/impexp/TableReplicationRequestProcessorTest.java > c9bb11c56 > > repository/src/test/java/org/apache/atlas/repository/impexp/ZipFileResourceTestUtils.java > 0ffc3d595 > > repository/src/test/java/org/apache/atlas/repository/migration/MigrationBaseAsserts.java > 18e950e8c > > repository/src/test/java/org/apache/atlas/repository/migration/RelationshipCacheGeneratorTest.java > 050bda3ae > > repository/src/test/java/org/apache/atlas/repository/tagpropagation/ClassificationPropagationTest.java > 6f9c05e7a > > repository/src/test/java/org/apache/atlas/repository/userprofile/UserProfileServiceTest.java > e56e48308 > repository/src/test/java/org/apache/atlas/services/MetricsServiceTest.java > baeafd4b7 > repository/src/test/java/org/apache/atlas/utils/TestLoadModelUtils.java > PRE-CREATION > server-api/src/main/java/org/apache/atlas/RequestContext.java f9ca7a26d > webapp/src/main/java/org/apache/atlas/web/resources/AdminResource.java > 40fd6efdc > webapp/src/test/java/org/apache/atlas/web/adapters/TestEntitiesREST.java > afc4c50d7 > > webapp/src/test/java/org/apache/atlas/web/integration/EntityV2JerseyResourceIT.java > a4b7b0d49 > webapp/src/test/java/org/apache/atlas/web/resources/AdminResourceTest.java > 563a16f64 > > > Diff: https://reviews.apache.org/r/71791/diff/7/ > > > Testing > ------- > > Two UT and one IT has been added to test functionality like: > 1. Atlas Audit entry save and retrive > 2. Atlas Purge Audit store, retrive and search (UT and IT) > > > Thanks, > > Sidharth Mishra > >
