----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/70196/#review213630 -----------------------------------------------------------
Ship it! Ship It! - Sarath Subramanian On March 12, 2019, 10:48 a.m., madhan wrote: > > ----------------------------------------------------------- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/70196/ > ----------------------------------------------------------- > > (Updated March 12, 2019, 10:48 a.m.) > > > Review request for atlas, Ashutosh Mestry, keval bhatt, Mehul Parikh, Nikhil > Bonte, Nixon Rodrigues, and Sarath Subramanian. > > > Bugs: ATLAS-3054 > https://issues.apache.org/jira/browse/ATLAS-3054 > > > Repository: atlas > > > Description > ------- > > - while batch-processing of entities in notification handler, an entity could > refer to another entity processed in earlier batch. This would cause Atlas to > process the already processed referenced entity to be processed again - since > the reference would use internal-guid. This can be expensive for large > referenced entities - such as a hive_table with 1000s of columns. This can be > avoided by updating the reference to replace internal-guid with assigned-guid > of the entity > - few other fixes found while processing of hive_table message > > > Diffs > ----- > > addons/models/1000-Hadoop/1030-hive_model.json 324d71665 > intg/src/main/java/org/apache/atlas/type/AtlasBuiltInTypes.java 6bedf6d43 > intg/src/main/java/org/apache/atlas/type/AtlasEntityType.java 2557bb31b > intg/src/main/java/org/apache/atlas/type/AtlasRelationshipType.java > 3ea8d805d > intg/src/main/java/org/apache/atlas/type/AtlasStructType.java 84c76d7e0 > intg/src/main/java/org/apache/atlas/utils/AtlasEntityUtil.java 30022173e > > repository/src/main/java/org/apache/atlas/discovery/EntityDiscoveryService.java > 9df360c89 > > repository/src/main/java/org/apache/atlas/repository/converters/AtlasStructFormatConverter.java > 51a642614 > > repository/src/main/java/org/apache/atlas/repository/store/graph/v2/AtlasEntityStoreV2.java > a62f335e7 > > repository/src/main/java/org/apache/atlas/repository/store/graph/v2/EntityGraphMapper.java > a6f1250fb > > webapp/src/main/java/org/apache/atlas/notification/NotificationHookConsumer.java > d16d544d9 > > webapp/src/main/java/org/apache/atlas/notification/preprocessor/HivePreprocessor.java > ff9c9cb44 > > webapp/src/main/java/org/apache/atlas/notification/preprocessor/PreprocessorContext.java > 94e099377 > > > Diff: https://reviews.apache.org/r/70196/diff/2/ > > > Testing > ------- > > - verified that notification for a large hive_table entity, with 1000s of > columns & few partition-keys, was processed correctly > - pre-commit tests run: > https://builds.apache.org/view/A/view/Atlas/job/PreCommit-ATLAS-Build-Test/971/ > > > Thanks, > > madhan > >
