-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/73423/
-----------------------------------------------------------
Review request for atlas, Ashutosh Mestry, Jayendra Parab, madhan, Nixon
Rodrigues, Sarath Subramanian, and Sidharth Mishra.
Bugs: ATLAS-4338
https://issues.apache.org/jira/browse/ATLAS-4338
Repository: atlas
Description
-------
Currently Migration import feature (MigrationImport.java) does not support
enabling notifiers (audits writer, notification to Atlas entities topic).
With this improvement, Atlas now will enable notifiers for Migration import,
based on a flag.
Approach:
* New option in AtlasImportRequest - sendNotifications (default : false)
* MigrationImport will use the flag to select appropriate implementation of
IAtlasEntityChangeNotifier to create EntityCreationManager.
* EntityCreationManager will then notify listeners.
Diffs
-----
intg/src/main/java/org/apache/atlas/model/impexp/AtlasImportRequest.java
2c1870444
repository/src/main/java/org/apache/atlas/repository/impexp/ImportService.java
1d29bf833
repository/src/main/java/org/apache/atlas/repository/store/graph/v2/AtlasEntityChangeNotifier.java
99be169f2
repository/src/main/java/org/apache/atlas/repository/store/graph/v2/BulkImporterImpl.java
8e17fd410
repository/src/main/java/org/apache/atlas/repository/store/graph/v2/bulkimport/MigrationImport.java
d6f23d6e2
repository/src/main/java/org/apache/atlas/repository/store/graph/v2/bulkimport/pc/EntityConsumer.java
b73988fd7
repository/src/main/java/org/apache/atlas/repository/store/graph/v2/bulkimport/pc/EntityCreationManager.java
734add6d7
Diff: https://reviews.apache.org/r/73423/diff/1/
Testing
-------
* Tested Manually
* Attached sample zip, sample request
{
"options": {
"migration": "true",
"size": "100",
"sendNotifications": "true",
"batchSize": 500,
"numWorkers": 40
}
}
* took ~270 secs to import 12987 entities with notifiers enabled.
* PC build :
File Attachments
----------------
largedb.zip
https://reviews.apache.org/media/uploaded/files/2021/06/15/c84f450b-c2b3-4cac-8b4a-76942a2c14e3__largedb.zip
Thanks,
Nikhil Bonte
