----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/67302/#review203927 -----------------------------------------------------------
Ship it! Ship It! - Madhan Neethiraj On May 25, 2018, 9:26 p.m., Ashutosh Mestry wrote: > > ----------------------------------------------------------- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/67302/ > ----------------------------------------------------------- > > (Updated May 25, 2018, 9:26 p.m.) > > > Review request for atlas, Madhan Neethiraj and Ruchi Solani. > > > Bugs: ATLAS-2717 > https://issues.apache.org/jira/browse/ATLAS-2717 > > > Repository: atlas > > > Description > ------- > > **Approach** > - Attributes of classification types are pre-processed (before incoming types > are processed). > - New _STRUCT_ types are created with the classification name prefixed with > _legacy_. So, classification of the name PII is used to create a _STRUCT_ > type named _legacy_PII_. > - Entity definitions are updated to reference these newly created types. > - During edge processing, the type name of the in vertex is replaced with > this new type. > - Rest of the processing continues as is. > > **Refacoring** > - Moved migration-specific functions to separate class _GraphDBMigrator_. > > > Diffs > ----- > > > graphdb/api/src/main/java/org/apache/atlas/repository/graphdb/AtlasGraph.java > e5316d80a > > graphdb/api/src/main/java/org/apache/atlas/repository/graphdb/GraphDBMigrator.java > PRE-CREATION > > graphdb/janus/src/main/java/org/apache/atlas/repository/graphdb/janus/AtlasJanusGraph.java > b4d6b33ea > > graphdb/janus/src/main/java/org/apache/atlas/repository/graphdb/janus/AtlasJanusGraphDatabase.java > c9d6067db > > graphdb/janus/src/main/java/org/apache/atlas/repository/graphdb/janus/migration/ElementProcessors.java > f51080a61 > > graphdb/janus/src/main/java/org/apache/atlas/repository/graphdb/janus/migration/GraphDBGraphSONMigrator.java > PRE-CREATION > > graphdb/janus/src/main/java/org/apache/atlas/repository/graphdb/janus/migration/GraphSONUtility.java > f1bbfcfcd > > graphdb/janus/src/main/java/org/apache/atlas/repository/graphdb/janus/migration/TypesDefScrubber.java > PRE-CREATION > > graphdb/janus/src/test/java/org/apache/atlas/repository/graphdb/janus/migration/BaseUtils.java > aee1b698c > > graphdb/janus/src/test/java/org/apache/atlas/repository/graphdb/janus/migration/GraphSONUtilityTest.java > 049fd6418 > graphdb/janus/src/test/resources/entity-with-trait-type.json PRE-CREATION > > repository/src/main/java/org/apache/atlas/repository/impexp/MigrationProgressService.java > 9620c1357 > > repository/src/main/java/org/apache/atlas/repository/migration/DataMigrationService.java > 081376ff5 > repository/src/test/java/org/apache/atlas/TestModules.java 23d128740 > > repository/src/test/java/org/apache/atlas/repository/migration/ComplexAttributesTest.java > 800638d43 > > repository/src/test/java/org/apache/atlas/repository/migration/HiveParititionTest.java > cae751215 > > repository/src/test/java/org/apache/atlas/repository/migration/HiveStocksTest.java > 9f255189d > > repository/src/test/java/org/apache/atlas/repository/migration/MigrationBaseAsserts.java > 4b02b7fb7 > > repository/src/test/java/org/apache/atlas/repository/migration/MigrationProgressServiceTest.java > 8fc524e81 > > repository/src/test/java/org/apache/atlas/repository/migration/PathTest.java > 468b5268c > > repository/src/test/java/org/apache/atlas/repository/migration/TypesDefScrubberTest.java > PRE-CREATION > > repository/src/test/java/org/apache/atlas/repository/migration/TypesWithClassificationTest.java > PRE-CREATION > > repository/src/test/java/org/apache/atlas/repository/migration/TypesWithCollectionsFinderTest.java > 4b236c1fe > repository/src/test/resources/classification_defs/atlas-migration-data.json > PRE-CREATION > > repository/src/test/resources/classification_defs/atlas-migration-typesdef.json > PRE-CREATION > repository/src/test/resources/legacy-typesdef.json PRE-CREATION > > > Diff: https://reviews.apache.org/r/67302/diff/5/ > > > Testing > ------- > > **Unit tests** > - Added tests to verify the new flow. > - Executed existing tests. > > **Test data** > - In versions 2.5 onwards (post 2/13/2017) it is not possible to create > entities of such types. Entity creation API throws a validation error. > - The test data was thus hand-created. > - This data verifies the cases where attribute type is: > - Classification type > - Array of classification type. > - Map of classification type. > > **Build** > [Pre-commit](https://builds.apache.org/view/A/view/Atlas/job/PreCommit-ATLAS-Build-Test/451/) > > > File Attachments > ---------------- > > atlas-migration-typesdef.json > > https://reviews.apache.org/media/uploaded/files/2018/05/24/c5cc2171-27f5-4d26-acd9-08aac9e1e1ef__atlas-migration-typesdef.json > atlas-migration-data.json > > https://reviews.apache.org/media/uploaded/files/2018/05/24/8230c44b-9f2e-4ae2-bc1a-04424dafc08c__atlas-migration-data.json > > > Thanks, > > Ashutosh Mestry > >
