Re: Review Request 60578: [ATLA-1907]: Create RelationshipDefs for all base models and use this definitions when creating edges for entities

2017-07-11 Thread Madhan Neethiraj 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/60578/#review180183
---


Ship it!




Ship It!

- Madhan Neethiraj


On July 11, 2017, 7:40 a.m., Sarath Subramanian wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/60578/
> ---
> 
> (Updated July 11, 2017, 7:40 a.m.)
> 
> 
> Review request for atlas, David Radley and Madhan Neethiraj.
> 
> 
> Bugs: ATLAS-1907
> https://issues.apache.org/jira/browse/ATLAS-1907
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> Create relationshipDefs for all out of the box models. Use relationshipDef 
> information when creating edges for composite attributes in entities.
> 
> 
> Diffs
> -
> 
>   addons/models/0010-base_model.json 7f64d85e 
>   addons/models/0030-hive_model.json b359ea82 
>   addons/models/0050-falcon_model.json b7398ef4 
>   addons/models/0060-hbase_model.json 42fe00c4 
>   addons/models/0080-storm_model.json 095936e0 
>   intg/src/main/java/org/apache/atlas/AtlasErrorCode.java e8afed1d 
>   intg/src/main/java/org/apache/atlas/model/instance/AtlasEntity.java 
> 0e277b19 
>   intg/src/main/java/org/apache/atlas/model/instance/AtlasRelationship.java 
> 8d2e7ec5 
>   intg/src/main/java/org/apache/atlas/model/instance/AtlasStruct.java 
> 7c8ae2d7 
>   intg/src/main/java/org/apache/atlas/type/AtlasEntityType.java 70e3067a 
>   intg/src/main/java/org/apache/atlas/type/AtlasRelationshipType.java 
> 49a94931 
>   intg/src/main/java/org/apache/atlas/type/AtlasStructType.java c2e0be50 
>   intg/src/main/java/org/apache/atlas/type/AtlasType.java 86072fe8 
>   intg/src/main/java/org/apache/atlas/type/AtlasTypeRegistry.java 29fae1c8 
>   repository/src/main/java/org/apache/atlas/repository/graph/GraphHelper.java 
> 329dd7a4 
>   
> repository/src/main/java/org/apache/atlas/repository/store/bootstrap/AtlasTypeDefStoreInitializer.java
>  4530f51b 
>   
> repository/src/main/java/org/apache/atlas/repository/store/graph/v1/AtlasGraphUtilsV1.java
>  7b3f1e6e 
>   
> repository/src/main/java/org/apache/atlas/repository/store/graph/v1/AtlasRelationshipStoreV1.java
>  86212338 
>   
> repository/src/main/java/org/apache/atlas/repository/store/graph/v1/EntityGraphMapper.java
>  ebf6a201 
>   
> repository/src/main/java/org/apache/atlas/repository/store/graph/v1/EntityGraphRetriever.java
>  a5b5730f 
>   
> repository/src/test/java/org/apache/atlas/repository/store/graph/v1/AtlasRelationshipStoreV1Test.java
>  PRE-CREATION 
> 
> 
> Diff: https://reviews.apache.org/r/60578/diff/6/
> 
> 
> Testing
> ---
> 
> mvn clean package succeeded. Validated hive_table creation and lineage 
> rendering.
> 
> UTs/ITs in progress.
> 
> 
> Thanks,
> 
> Sarath Subramanian
> 
>

Re: Review Request 60578: [ATLA-1907]: Create RelationshipDefs for all base models and use this definitions when creating edges for entities

2017-07-10 Thread Sarath Subramanian 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/60578/
---

(Updated July 10, 2017, 5:09 p.m.)


Review request for atlas, David Radley and Madhan Neethiraj.


Changes
---

fixed UT failures.


Bugs: ATLAS-1907
https://issues.apache.org/jira/browse/ATLAS-1907


Repository: atlas


Description
---

Create relationshipDefs for all out of the box models. Use relationshipDef 
information when creating edges for composite attributes in entities.


Diffs (updated)
-

  addons/models/0010-base_model.json 7f64d85e 
  addons/models/0030-hive_model.json b359ea82 
  addons/models/0050-falcon_model.json b7398ef4 
  addons/models/0060-hbase_model.json 42fe00c4 
  addons/models/0080-storm_model.json 095936e0 
  intg/src/main/java/org/apache/atlas/AtlasErrorCode.java e8afed1d 
  intg/src/main/java/org/apache/atlas/model/instance/AtlasEntity.java 0e277b19 
  intg/src/main/java/org/apache/atlas/model/instance/AtlasRelationship.java 
8d2e7ec5 
  intg/src/main/java/org/apache/atlas/model/instance/AtlasStruct.java 7c8ae2d7 
  intg/src/main/java/org/apache/atlas/model/typedef/AtlasRelationshipDef.java 
fc820d49 
  intg/src/main/java/org/apache/atlas/type/AtlasEntityType.java 70e3067a 
  intg/src/main/java/org/apache/atlas/type/AtlasRelationshipType.java 49a94931 
  intg/src/main/java/org/apache/atlas/type/AtlasStructType.java c2e0be50 
  intg/src/main/java/org/apache/atlas/type/AtlasType.java 86072fe8 
  intg/src/main/java/org/apache/atlas/type/AtlasTypeRegistry.java 29fae1c8 
  repository/src/main/java/org/apache/atlas/repository/graph/GraphHelper.java 
329dd7a4 
  
repository/src/main/java/org/apache/atlas/repository/store/bootstrap/AtlasTypeDefStoreInitializer.java
 4530f51b 
  
repository/src/main/java/org/apache/atlas/repository/store/graph/v1/AtlasGraphUtilsV1.java
 7b3f1e6e 
  
repository/src/main/java/org/apache/atlas/repository/store/graph/v1/AtlasRelationshipStoreV1.java
 86212338 
  
repository/src/main/java/org/apache/atlas/repository/store/graph/v1/EntityGraphMapper.java
 ebf6a201 
  
repository/src/main/java/org/apache/atlas/repository/store/graph/v1/EntityGraphRetriever.java
 a5b5730f 
  
repository/src/test/java/org/apache/atlas/repository/store/graph/v1/AtlasEntityStoreV1Test.java
 2ac0fc66 
  
repository/src/test/java/org/apache/atlas/repository/store/graph/v1/AtlasRelationshipStoreV1Test.java
 PRE-CREATION 


Diff: https://reviews.apache.org/r/60578/diff/4/

Changes: https://reviews.apache.org/r/60578/diff/3-4/


Testing (updated)
---

mvn clean package succeeded. Validated hive_table creation and lineage is 
rendering.

UTs/ITs in progress.


Thanks,

Sarath Subramanian

Re: Review Request 60578: [ATLA-1907]: Create RelationshipDefs for all base models and use this definitions when creating edges for entities

2017-07-06 Thread Sarath Subramanian 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/60578/
---

(Updated July 6, 2017, 5:12 p.m.)


Review request for atlas, David Radley and Madhan Neethiraj.


Changes
---

addressed review comments


Bugs: ATLAS-1907
https://issues.apache.org/jira/browse/ATLAS-1907


Repository: atlas


Description
---

Create relationshipDefs for all out of the box models. Use relationshipDef 
information when creating edges for composite attributes in entities.


Diffs (updated)
-

  addons/models/0010-base_model.json 7f64d85e 
  addons/models/0030-hive_model.json b359ea82 
  addons/models/0050-falcon_model.json b7398ef4 
  addons/models/0060-hbase_model.json 42fe00c4 
  addons/models/0080-storm_model.json 095936e0 
  intg/src/main/java/org/apache/atlas/AtlasErrorCode.java 73dd33aa 
  intg/src/main/java/org/apache/atlas/model/instance/AtlasRelationship.java 
8d2e7ec5 
  intg/src/main/java/org/apache/atlas/type/AtlasEntityType.java 70e3067a 
  intg/src/main/java/org/apache/atlas/type/AtlasRelationshipType.java 49a94931 
  intg/src/main/java/org/apache/atlas/type/AtlasStructType.java c2e0be50 
  intg/src/main/java/org/apache/atlas/type/AtlasType.java 86072fe8 
  intg/src/main/java/org/apache/atlas/type/AtlasTypeRegistry.java 29fae1c8 
  
repository/src/main/java/org/apache/atlas/repository/store/bootstrap/AtlasTypeDefStoreInitializer.java
 4530f51b 
  
repository/src/main/java/org/apache/atlas/repository/store/graph/v1/AtlasEntityStoreV1.java
 5ea4ff20 
  
repository/src/main/java/org/apache/atlas/repository/store/graph/v1/AtlasGraphUtilsV1.java
 7b3f1e6e 
  
repository/src/main/java/org/apache/atlas/repository/store/graph/v1/AtlasRelationshipStoreV1.java
 86212338 
  
repository/src/main/java/org/apache/atlas/repository/store/graph/v1/EntityGraphMapper.java
 ebf6a201 
  
repository/src/test/java/org/apache/atlas/repository/store/graph/v1/AtlasRelationshipStoreV1Test.java
 PRE-CREATION 


Diff: https://reviews.apache.org/r/60578/diff/2/

Changes: https://reviews.apache.org/r/60578/diff/1-2/


Testing
---

Tested using Postman REST client.
Validated all hive_table creation works fine and lineage is rendered right.

UTs/ITs in progress.


Thanks,

Sarath Subramanian

Re: Review Request 60578: [ATLA-1907]: Create RelationshipDefs for all base models and use this definitions when creating edges for entities

2017-07-02 Thread David Radley 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/60578/#review179478
---




addons/models/0010-base_model.json
Lines 107 (patched)


why do we need to expose the legacy label value in the relationshipDef - 
can we not calculate it in the same way that the legacy code does. Is there any 
use case to have this label value different from the legacy values?



intg/src/main/java/org/apache/atlas/type/AtlasEntityType.java
Lines 126 (patched)


What is the thinking behind issuing this warning? What is the user expected 
to change?



intg/src/main/java/org/apache/atlas/type/AtlasEntityType.java
Lines 196 (patched)


remember to do the TODO



intg/src/main/java/org/apache/atlas/type/AtlasTypeRegistry.java
Lines 369 (patched)


remember the TODO



repository/src/main/java/org/apache/atlas/repository/store/graph/v1/AtlasRelationshipStoreV1.java
Lines 247 (patched)


check =>is



repository/src/main/java/org/apache/atlas/repository/store/graph/v1/AtlasRelationshipStoreV1.java
Lines 253 (patched)


I am not sure what this is doing. Why does each endDef having a legacy 
relation mean that there is a bidirectional relationship. Do you mean that the 
relaitonships is implmneted by 2 directional edges? This is not a bidirecitonal 
relationship (which we were thinking of as a single edge with direction BOTH).



repository/src/main/java/org/apache/atlas/repository/store/graph/v1/EntityGraphMapper.java
Lines 409 (patched)


I see in the models we have columns defined in the old way and the new way 
- but this code seems to imply that without a relationshipDef we use the legacy 
way to create edges. I am unsure on the proposed design in this area.



repository/src/main/java/org/apache/atlas/repository/store/graph/v1/EntityGraphMapper.java
Lines 568 (patched)


reltionDef => relationshipDef



repository/src/main/java/org/apache/atlas/repository/store/graph/v1/EntityGraphMapper.java
Lines 1157 (patched)


What are we calling the relationship attribute. Is this an attribute of the 
relationship or the name of the relationship or the name associated with the 
end. It would be good if we could agree on terms that we could use consistently 
through the code.



repository/src/main/java/org/apache/atlas/repository/store/graph/v1/EntityGraphMapper.java
Lines 1189 (patched)


atleast => at least
entry => Def


- David Radley


On June 30, 2017, 8:25 p.m., Sarath Subramanian wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/60578/
> ---
> 
> (Updated June 30, 2017, 8:25 p.m.)
> 
> 
> Review request for atlas, David Radley and Madhan Neethiraj.
> 
> 
> Bugs: ATLAS-1907
> https://issues.apache.org/jira/browse/ATLAS-1907
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> Create relationshipDefs for all out of the box models. Use relationshipDef 
> information when creating edges for composite attributes in entities.
> 
> 
> Diffs
> -
> 
>   addons/models/0010-base_model.json 7f64d85e 
>   addons/models/0030-hive_model.json b359ea82 
>   addons/models/0050-falcon_model.json b7398ef4 
>   addons/models/0060-hbase_model.json 42fe00c4 
>   addons/models/0080-storm_model.json 095936e0 
>   intg/src/main/java/org/apache/atlas/AtlasErrorCode.java 49ead6e3 
>   intg/src/main/java/org/apache/atlas/model/instance/AtlasRelationship.java 
> 8d2e7ec5 
>   intg/src/main/java/org/apache/atlas/type/AtlasEntityType.java 70e3067a 
>   intg/src/main/java/org/apache/atlas/type/AtlasRelationshipType.java 
> f85cf35c 
>   intg/src/main/java/org/apache/atlas/type/AtlasStructType.java c2e0be50 
>   intg/src/main/java/org/apache/atlas/type/AtlasTypeRegistry.java 29fae1c8 
>   
> repository/src/main/java/org/apache/atlas/repository/store/bootstrap/AtlasTypeDefStoreInitializer.java
>  4530f51b 
>   
> repository/src/main/java/org/apache/atlas/repository/store/graph/v1/AtlasEntityStoreV1.java
>  5ea4ff20 
>   
> repository/src/main/java/org/apache/atlas/repository/store/graph/v1/AtlasGraphUtilsV1.java
>  7b3f1e6e 
>   
> repository/src/main/java/org/apache/atlas/repository/store/graph/v1/AtlasRelationshipStoreV1.java
>  86212338 
>   
>

Review Request 60578: [ATLA-1907]: Create RelationshipDefs for all base models and use this definitions when creating edges for entities

2017-06-30 Thread Sarath Subramanian 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/60578/
---

Review request for atlas, David Radley and Madhan Neethiraj.


Bugs: ATLAS-1907
https://issues.apache.org/jira/browse/ATLAS-1907


Repository: atlas


Description
---

Create relationshipDefs for all out of the box models. Use relationshipDef 
information when creating edges for composite attributes in entities.


Diffs
-

  addons/models/0010-base_model.json 7f64d85e 
  addons/models/0030-hive_model.json b359ea82 
  addons/models/0050-falcon_model.json b7398ef4 
  addons/models/0060-hbase_model.json 42fe00c4 
  addons/models/0080-storm_model.json 095936e0 
  intg/src/main/java/org/apache/atlas/AtlasErrorCode.java 49ead6e3 
  intg/src/main/java/org/apache/atlas/model/instance/AtlasRelationship.java 
8d2e7ec5 
  intg/src/main/java/org/apache/atlas/type/AtlasEntityType.java 70e3067a 
  intg/src/main/java/org/apache/atlas/type/AtlasRelationshipType.java f85cf35c 
  intg/src/main/java/org/apache/atlas/type/AtlasStructType.java c2e0be50 
  intg/src/main/java/org/apache/atlas/type/AtlasTypeRegistry.java 29fae1c8 
  
repository/src/main/java/org/apache/atlas/repository/store/bootstrap/AtlasTypeDefStoreInitializer.java
 4530f51b 
  
repository/src/main/java/org/apache/atlas/repository/store/graph/v1/AtlasEntityStoreV1.java
 5ea4ff20 
  
repository/src/main/java/org/apache/atlas/repository/store/graph/v1/AtlasGraphUtilsV1.java
 7b3f1e6e 
  
repository/src/main/java/org/apache/atlas/repository/store/graph/v1/AtlasRelationshipStoreV1.java
 86212338 
  
repository/src/main/java/org/apache/atlas/repository/store/graph/v1/EntityGraphMapper.java
 ebf6a201 
  
repository/src/test/java/org/apache/atlas/repository/store/graph/v1/AtlasRelationshipStoreV1Test.java
 PRE-CREATION 


Diff: https://reviews.apache.org/r/60578/diff/1/


Testing
---

Tested using Postman REST client.
Validated all hive_table creation works fine and lineage is rendered right.

UTs/ITs in progress.


Thanks,

Sarath Subramanian