Re: Review Request 72619: ATLAS-3863: Avoid re-evaluating tag propagation when deleting relationship (edge)

2020-06-25 Thread Madhan Neethiraj 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/72619/#review221072
---


Ship it!




Ship It!

- Madhan Neethiraj


On June 25, 2020, 12:33 a.m., Sarath Subramanian wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/72619/
> ---
> 
> (Updated June 25, 2020, 12:33 a.m.)
> 
> 
> Review request for atlas, Ashutosh Mestry, chaitali, Jayendra Parab, Madhan 
> Neethiraj, Mandar Ambawane, mayank jain, Nikhil Bonte, Nixon Rodrigues, Pinal 
> Shah, and Sidharth Mishra.
> 
> 
> Bugs: ATLAS-3863
> https://issues.apache.org/jira/browse/ATLAS-3863
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> Tag propagation is re-evaluated every time when a relationship (edge) is 
> deleted.
> 
> Removal of propagated classifications is controlled at the assignment time to 
> an entity:
> 
> 
> Remove propagation on entity delete - TRUE : 
> Propagated classifications (to all downstream entities) will be removed when 
> the source entity where the classification is originally assigned is deleted.
>  
> 
> Remove propagation on entity delete - FALSE : 
> Propagated classifications (to all downstream entities) will not be removed 
> when the  source entity where the classification is originally 
> assigned is deleted.
>  
> 
> There is no need to revaluate tag propagation during relationship (edge) 
> deletion since removal of tag propagation depends on the above config.
> 
> 
> Diffs
> -
> 
>   repository/src/main/java/org/apache/atlas/repository/graph/GraphHelper.java 
> 7b7ec6567 
>   
> repository/src/main/java/org/apache/atlas/repository/store/graph/v1/DeleteHandlerV1.java
>  3f8503ac3 
> 
> 
> Diff: https://reviews.apache.org/r/72619/diff/2/
> 
> 
> Testing
> ---
> 
> Manually validated:
> 
> Deleted relationship, Deleted downstream entities and verified that 
> propagated tags are not affected and revaluation of tag propagation is not 
> computed.
> 
> Precommit - 
> https://builds.apache.org/view/A/view/Atlas/job/PreCommit-ATLAS-Build-Test/1978/console
> 
> 
> `Significant performance improvement observed:`
> 
> Deleting a hive_table entity (with 44 columns and 16 propagated 
> classifications)
> 
> Before patch : 279 secs (4 mins, 39 secs)
> After patch  : 46.92 secs
> 
> 83% performance improvement seen.
> 
> 
> Thanks,
> 
> Sarath Subramanian
> 
>

Re: Review Request 72619: ATLAS-3863: Avoid re-evaluating tag propagation when deleting relationship (edge)

2020-06-24 Thread Sarath Subramanian 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/72619/
---

(Updated June 24, 2020, 5:33 p.m.)


Review request for atlas, Ashutosh Mestry, chaitali, Jayendra Parab, Madhan 
Neethiraj, Mandar Ambawane, mayank jain, Nikhil Bonte, Nixon Rodrigues, Pinal 
Shah, and Sidharth Mishra.


Bugs: ATLAS-3863
https://issues.apache.org/jira/browse/ATLAS-3863


Repository: atlas


Description
---

Tag propagation is re-evaluated every time when a relationship (edge) is 
deleted.

Removal of propagated classifications is controlled at the assignment time to 
an entity:


Remove propagation on entity delete - TRUE : 
Propagated classifications (to all downstream entities) will be removed when 
the source entity where the classification is originally assigned is deleted.
 

Remove propagation on entity delete - FALSE : 
Propagated classifications (to all downstream entities) will not be removed 
when the  source entity where the classification is originally assigned 
is deleted.
 

There is no need to revaluate tag propagation during relationship (edge) 
deletion since removal of tag propagation depends on the above config.


Diffs (updated)
-

  repository/src/main/java/org/apache/atlas/repository/graph/GraphHelper.java 
7b7ec6567 
  
repository/src/main/java/org/apache/atlas/repository/store/graph/v1/DeleteHandlerV1.java
 3f8503ac3 


Diff: https://reviews.apache.org/r/72619/diff/2/

Changes: https://reviews.apache.org/r/72619/diff/1-2/


Testing
---

Manually validated:

Deleted relationship, Deleted downstream entities and verified that propagated 
tags are not affected and revaluation of tag propagation is not computed.

Precommit - 
https://builds.apache.org/view/A/view/Atlas/job/PreCommit-ATLAS-Build-Test/1978/console


`Significant performance improvement observed:`

Deleting a hive_table entity (with 44 columns and 16 propagated classifications)

Before patch : 279 secs (4 mins, 39 secs)
After patch  : 46.92 secs

83% performance improvement seen.


Thanks,

Sarath Subramanian

Re: Review Request 72619: ATLAS-3863: Avoid re-evaluating tag propagation when deleting relationship (edge)

2020-06-24 Thread Ashutosh Mestry via Review Board 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/72619/#review221066
---




repository/src/main/java/org/apache/atlas/repository/graph/GraphHelper.java
Line 411 (original), 411 (patched)


Consider using hasEdges.



repository/src/main/java/org/apache/atlas/repository/graph/GraphHelper.java
Line 411 (original), 411 (patched)


Consider using hasEdges.


- Ashutosh Mestry


On June 24, 2020, 8:03 p.m., Sarath Subramanian wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/72619/
> ---
> 
> (Updated June 24, 2020, 8:03 p.m.)
> 
> 
> Review request for atlas, Ashutosh Mestry, chaitali, Jayendra Parab, Madhan 
> Neethiraj, Mandar Ambawane, mayank jain, Nikhil Bonte, Nixon Rodrigues, Pinal 
> Shah, and Sidharth Mishra.
> 
> 
> Bugs: ATLAS-3863
> https://issues.apache.org/jira/browse/ATLAS-3863
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> Tag propagation is re-evaluated every time when a relationship (edge) is 
> deleted.
> 
> Removal of propagated classifications is controlled at the assignment time to 
> an entity:
> 
> 
> Remove propagation on entity delete - TRUE : 
> Propagated classifications (to all downstream entities) will be removed when 
> the source entity where the classification is originally assigned is deleted.
>  
> 
> Remove propagation on entity delete - FALSE : 
> Propagated classifications (to all downstream entities) will not be removed 
> when the  source entity where the classification is originally 
> assigned is deleted.
>  
> 
> There is no need to revaluate tag propagation during relationship (edge) 
> deletion since removal of tag propagation depends on the above config.
> 
> 
> Diffs
> -
> 
>   repository/src/main/java/org/apache/atlas/repository/graph/GraphHelper.java 
> 7b7ec6567 
>   
> repository/src/main/java/org/apache/atlas/repository/store/graph/v1/DeleteHandlerV1.java
>  3f8503ac3 
> 
> 
> Diff: https://reviews.apache.org/r/72619/diff/1/
> 
> 
> Testing
> ---
> 
> Manually validated:
> 
> Deleted relationship, Deleted downstream entities and verified that 
> propagated tags are not affected and revaluation of tag propagation is not 
> computed.
> 
> Precommit - 
> https://builds.apache.org/view/A/view/Atlas/job/PreCommit-ATLAS-Build-Test/1978/console
> 
> 
> `Significant performance improvement observed:`
> 
> Deleting a hive_table entity (with 44 columns and 16 propagated 
> classifications)
> 
> Before patch : 279 secs (4 mins, 39 secs)
> After patch  : 46.92 secs
> 
> 83% performance improvement seen.
> 
> 
> Thanks,
> 
> Sarath Subramanian
> 
>