-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/67302/
-----------------------------------------------------------
(Updated May 25, 2018, 9:26 p.m.)
Review request for atlas, Madhan Neethiraj and Ruchi Solani.
Changes
-------
Updates include: Scanning of struct type during scrub process.
Bugs: ATLAS-2717
https://issues.apache.org/jira/browse/ATLAS-2717
Repository: atlas
Description
-------
**Approach**
- Attributes of classification types are pre-processed (before incoming types
are processed).
- New _STRUCT_ types are created with the classification name prefixed with
_legacy_. So, classification of the name PII is used to create a _STRUCT_ type
named _legacy_PII_.
- Entity definitions are updated to reference these newly created types.
- During edge processing, the type name of the in vertex is replaced with this
new type.
- Rest of the processing continues as is.
**Refacoring**
- Moved migration-specific functions to separate class _GraphDBMigrator_.
Diffs (updated)
-----
graphdb/api/src/main/java/org/apache/atlas/repository/graphdb/AtlasGraph.java
e5316d80a
graphdb/api/src/main/java/org/apache/atlas/repository/graphdb/GraphDBMigrator.java
PRE-CREATION
graphdb/janus/src/main/java/org/apache/atlas/repository/graphdb/janus/AtlasJanusGraph.java
b4d6b33ea
graphdb/janus/src/main/java/org/apache/atlas/repository/graphdb/janus/AtlasJanusGraphDatabase.java
c9d6067db
graphdb/janus/src/main/java/org/apache/atlas/repository/graphdb/janus/migration/ElementProcessors.java
f51080a61
graphdb/janus/src/main/java/org/apache/atlas/repository/graphdb/janus/migration/GraphDBGraphSONMigrator.java
PRE-CREATION
graphdb/janus/src/main/java/org/apache/atlas/repository/graphdb/janus/migration/GraphSONUtility.java
f1bbfcfcd
graphdb/janus/src/main/java/org/apache/atlas/repository/graphdb/janus/migration/TypesDefScrubber.java
PRE-CREATION
graphdb/janus/src/test/java/org/apache/atlas/repository/graphdb/janus/migration/BaseUtils.java
aee1b698c
graphdb/janus/src/test/java/org/apache/atlas/repository/graphdb/janus/migration/GraphSONUtilityTest.java
049fd6418
graphdb/janus/src/test/resources/entity-with-trait-type.json PRE-CREATION
repository/src/main/java/org/apache/atlas/repository/impexp/MigrationProgressService.java
9620c1357
repository/src/main/java/org/apache/atlas/repository/migration/DataMigrationService.java
081376ff5
repository/src/test/java/org/apache/atlas/TestModules.java 23d128740
repository/src/test/java/org/apache/atlas/repository/migration/ComplexAttributesTest.java
800638d43
repository/src/test/java/org/apache/atlas/repository/migration/HiveParititionTest.java
cae751215
repository/src/test/java/org/apache/atlas/repository/migration/HiveStocksTest.java
9f255189d
repository/src/test/java/org/apache/atlas/repository/migration/MigrationBaseAsserts.java
4b02b7fb7
repository/src/test/java/org/apache/atlas/repository/migration/MigrationProgressServiceTest.java
8fc524e81
repository/src/test/java/org/apache/atlas/repository/migration/PathTest.java
468b5268c
repository/src/test/java/org/apache/atlas/repository/migration/TypesDefScrubberTest.java
PRE-CREATION
repository/src/test/java/org/apache/atlas/repository/migration/TypesWithClassificationTest.java
PRE-CREATION
repository/src/test/java/org/apache/atlas/repository/migration/TypesWithCollectionsFinderTest.java
4b236c1fe
repository/src/test/resources/classification_defs/atlas-migration-data.json
PRE-CREATION
repository/src/test/resources/classification_defs/atlas-migration-typesdef.json
PRE-CREATION
repository/src/test/resources/legacy-typesdef.json PRE-CREATION
Diff: https://reviews.apache.org/r/67302/diff/5/
Changes: https://reviews.apache.org/r/67302/diff/4-5/
Testing
-------
**Unit tests**
- Added tests to verify the new flow.
- Executed existing tests.
**Test data**
- In versions 2.5 onwards (post 2/13/2017) it is not possible to create
entities of such types. Entity creation API throws a validation error.
- The test data was thus hand-created.
- This data verifies the cases where attribute type is:
- Classification type
- Array of classification type.
- Map of classification type.
**Build**
[Pre-commit](https://builds.apache.org/view/A/view/Atlas/job/PreCommit-ATLAS-Build-Test/451/)
File Attachments
----------------
atlas-migration-typesdef.json
https://reviews.apache.org/media/uploaded/files/2018/05/24/c5cc2171-27f5-4d26-acd9-08aac9e1e1ef__atlas-migration-typesdef.json
atlas-migration-data.json
https://reviews.apache.org/media/uploaded/files/2018/05/24/8230c44b-9f2e-4ae2-bc1a-04424dafc08c__atlas-migration-data.json
Thanks,
Ashutosh Mestry
