-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/71426/
-----------------------------------------------------------
(Updated Sept. 9, 2019, 6:23 p.m.)
Review request for atlas, Madhan Neethiraj, Nikhil Bonte, Nixon Rodrigues, and
Sarath Subramanian.
Changes
-------
Updates include: Addressed review comments.
Bugs: ATLAS-3396
https://issues.apache.org/jira/browse/ATLAS-3396
Repository: atlas
Description
-------
**Background**
The approach adds another option to _ImportService_ to be able to optionally
use a directory on the server for storing data to be imported.
**Approach**
This is a backward compatible.
- - New: _ZipSourceWithBackingDirectory_: Uses a temporary directory for
storing contents of the zip file.
- Modified: _ImportService_ Now takes _InputStream_ as parameter instead of
_ZipSource_.
- Modified: _ImportRequest_ Additionl option takes _backingDirectory_ as
additional option. Specifying this will use the new
_ZipSourceWithBackingDirectory_ during import.
**CURL**
Use pre-configured temporary directory, set _atlas.import.temporaryDirectory_
in the applicatino properties.
_CURL_
```
curl -v -X POST -u admin:admin -H "Content-Type: multipart/form-data" -H
"Cache-Control: no-cache" -F request=@../docs/import-options.json -F
data=@../docs/smalldb.zip http://localhost:21000/api/atlas/admin/import
```
**Documentation**
Updated.
Diffs (updated)
-----
common/src/main/java/org/apache/atlas/AtlasConfiguration.java c5357f5a4
distro/src/conf/atlas-application.properties 654a9304e
docs/src/site/twiki/Import-API-Options.twiki 4004e7013
repository/src/main/java/org/apache/atlas/repository/impexp/ImportService.java
b5d8b7c39
repository/src/main/java/org/apache/atlas/repository/impexp/ZipSource.java
016acd743
repository/src/main/java/org/apache/atlas/repository/impexp/ZipSourceWithBackingDirectory.java
PRE-CREATION
repository/src/main/java/org/apache/atlas/repository/store/graph/v1/AtlasEntityStreamForImport.java
90ae15d1e
repository/src/main/java/org/apache/atlas/repository/store/graph/v1/BulkImporterImpl.java
26066920a
repository/src/main/java/org/apache/atlas/repository/store/graph/v1/EntityImportStream.java
d4b6c5505
repository/src/test/java/org/apache/atlas/repository/impexp/ExportIncrementalTest.java
a355297a8
repository/src/test/java/org/apache/atlas/repository/impexp/ExportSkipLineageTest.java
eaf4602d5
repository/src/test/java/org/apache/atlas/repository/impexp/ImportServiceTest.java
693a163f8
repository/src/test/java/org/apache/atlas/repository/impexp/ImportTransformsShaperTest.java
06bdaa6d6
repository/src/test/java/org/apache/atlas/repository/impexp/ReplicationEntityAttributeTest.java
868b732d3
repository/src/test/java/org/apache/atlas/repository/impexp/ZipFileResourceTestUtils.java
a2a5f58dc
webapp/src/main/java/org/apache/atlas/web/resources/AdminResource.java
8417e7eb0
Diff: https://reviews.apache.org/r/71426/diff/7/
Changes: https://reviews.apache.org/r/71426/diff/6-7/
Testing
-------
**Unit tests**
Updated to exercise the new implementation.
**Volume tests**
Imported 4GB ZIP file that took 26 hours. Memory and CPU stayed in normal range
during this operation.
**Functional tests**
Accuracy tests preformed on small size data.
**Pre-commit Build**
https://builds.apache.org/view/A/view/Atlas/job/PreCommit-ATLAS-Build-Test/1380/
Thanks,
Ashutosh Mestry
