----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/72330/#review220242 -----------------------------------------------------------
repository/src/main/java/org/apache/atlas/repository/store/graph/v2/EntityGraphMapper.java Lines 623 (patched) <https://reviews.apache.org/r/72330/#comment308572> Instead of adding 4 blocks (#623, #639, #677, #693) consider handling this special case (for PRIMITIVE and ENUM) inside mapAttribute() - at #713. repository/src/main/java/org/apache/atlas/repository/store/graph/v2/EntityGraphMapper.java Lines 1068 (patched) <https://reviews.apache.org/r/72330/#comment308573> Not populating uniqPropName will cause UniqAttrBasedEntityResolver to not find the entity - as it uses AtlasGraphUtilsV2.findByUniqueAttributes(), which in turn looks for entity using uniqPropName. This could result in multiple entity instances be created with the same uniqAttribute value during migration-import. To improve throughput during migration, consider enabling bulk-load option; refer to https://docs.janusgraph.org/advanced-topics/bulk-loading/. - Madhan Neethiraj On April 7, 2020, 4:30 p.m., Ashutosh Mestry wrote: > > ----------------------------------------------------------- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/72330/ > ----------------------------------------------------------- > > (Updated April 7, 2020, 4:30 p.m.) > > > Review request for atlas, Madhan Neethiraj, Nikhil Bonte, and Nixon Rodrigues. > > > Bugs: ATLAS-3715 > https://issues.apache.org/jira/browse/ATLAS-3715 > > > Repository: atlas > > > Description > ------- > > **Approach** > - Create a new flag within _RequestContext_ that indicates if migration > import is in progress. > - Refactored _mapPrimitiveAttribute_ to not create > _AtlasAttributeMutationContext_ for primitive attributes. The new method is > used in entity and relationship mapping. > - Within _mapPrimitiveAttribute_ skip setting the uniqueProperty name. > > **Reason** > - The change does not set the _uniquePropertyKey_. The act of doing this > causes throughput to be improved by ~75%. > - This attribute would be handled post-upgrade of this attribute via the > _UniqueAttributePatch_. > > > Diffs > ----- > > > repository/src/main/java/org/apache/atlas/repository/store/graph/v2/EntityGraphMapper.java > 0d5b6314d > > repository/src/main/java/org/apache/atlas/repository/store/graph/v2/bulkimport/pc/EntityConsumer.java > e0a457334 > server-api/src/main/java/org/apache/atlas/RequestContext.java 282a66f1d > > > Diff: https://reviews.apache.org/r/72330/diff/1/ > > > Testing > ------- > > **Volume test** > Enable metric log. > Baseline: 25 secs. > New: 2 secs. > ``` > atlas.migration.mode.batch.size=700 > atlas.migration.mode.workers=24 > ```` > > > Thanks, > > Ashutosh Mestry > >
