-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/71025/
-----------------------------------------------------------
Review request for atlas, Madhan Neethiraj, Nikhil Bonte, Nixon Rodrigues, and
Sarath Subramanian.
Bugs: ATLAS-3320
https://issues.apache.org/jira/browse/ATLAS-3320
Repository: atlas
Description
-------
**Approach**
- Use existing producer-consumer (PC) framework.
- Modify _BulkImporterImpl_ to implement _WorkItemConsumer_.
- Add support for configuring number of workers and batch size within
_AtlasImportRequest_.
Diffs
-----
graphdb/janus/src/main/java/org/apache/atlas/repository/graphdb/janus/AtlasJanusGraph.java
499e8d1af
intg/src/main/java/org/apache/atlas/model/impexp/AtlasImportRequest.java
0b3ede93f
intg/src/main/resources/atlas-log4j.xml 4f74c2abb
repository/src/main/java/org/apache/atlas/repository/impexp/ImportService.java
3ded79842
repository/src/main/java/org/apache/atlas/repository/store/graph/v2/BulkImporterImpl.java
2f330c093
Diff: https://reviews.apache.org/r/71025/diff/1/
Testing
-------
**Unit tests**
Existing tests.
**Functional tests**
- Verified import for pre-1.0 and post-1.0 exported ZIP files.
**Volume tests**
- Measure performance with large data.
+----------+----------+----------+------------------------+
| File | Before | After | Configuration |
+----------+----------+----------+------------------------+
| smalldb | 6 min | 3 min | Shards: 4, Threads: 8 |
| (2.2 MB) | | | |
+----------+----------+----------+------------------------+
| largedb | 3 hrs | 20 mins | Shards: 4, Threads: 16 |
| (40 MB) | | | |
+----------+----------+----------+------------------------+
Thanks,
Ashutosh Mestry
