-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/70196/
-----------------------------------------------------------
Review request for atlas, Ashutosh Mestry, keval bhatt, Mehul Parikh, Nikhil
Bonte, Nixon Rodrigues, and Sarath Subramanian.
Bugs: ATLAS-3054
https://issues.apache.org/jira/browse/ATLAS-3054
Repository: atlas
Description
-------
- while batch-processing of entities in notification handler, an entity could
refer to another entity processed in earlier batch. This would cause Atlas to
process the already processed referenced entity to be processed again - since
the reference would use internal-guid. This can be expensive for large
referenced entities - such as a hive_table with 1000s of columns. This can be
avoided by updating the reference to replace internal-guid with assigned-guid
of the entity
- few other fixes found while processing of hive_table message
Diffs
-----
addons/models/1000-Hadoop/1030-hive_model.json 324d71665
intg/src/main/java/org/apache/atlas/type/AtlasBuiltInTypes.java 6bedf6d43
intg/src/main/java/org/apache/atlas/type/AtlasEntityType.java 2557bb31b
intg/src/main/java/org/apache/atlas/type/AtlasRelationshipType.java 3ea8d805d
intg/src/main/java/org/apache/atlas/type/AtlasStructType.java 84c76d7e0
intg/src/main/java/org/apache/atlas/utils/AtlasEntityUtil.java 30022173e
repository/src/main/java/org/apache/atlas/discovery/EntityDiscoveryService.java
9df360c89
repository/src/main/java/org/apache/atlas/repository/converters/AtlasStructFormatConverter.java
51a642614
repository/src/main/java/org/apache/atlas/repository/store/graph/v2/AtlasEntityStoreV2.java
a62f335e7
repository/src/main/java/org/apache/atlas/repository/store/graph/v2/EntityGraphMapper.java
a6f1250fb
webapp/src/main/java/org/apache/atlas/notification/NotificationHookConsumer.java
d16d544d9
webapp/src/main/java/org/apache/atlas/notification/preprocessor/HivePreprocessor.java
ff9c9cb44
webapp/src/main/java/org/apache/atlas/notification/preprocessor/PreprocessorContext.java
94e099377
Diff: https://reviews.apache.org/r/70196/diff/1/
Testing
-------
- verified that notification for a large hive_table entity, with 1000s of
columns & few partition-keys, was processed correctly
- pre-commit tests run:
https://builds.apache.org/view/A/view/Atlas/job/PreCommit-ATLAS-Build-Test/971/
Thanks,
madhan
