-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/71902/#review219011
-----------------------------------------------------------




intg/src/main/java/org/apache/atlas/AtlasConfiguration.java
Lines 66 (patched)
<https://reviews.apache.org/r/71902/#comment307029>

    LINEAGE_GRAPH_QUERY("atlas.use.graph.query.for.lineage", false) => 
LINEAGE_USING_GREMLIN("atlas.lineage.query.use.gremlin", false)



repository/src/main/java/org/apache/atlas/discovery/EntityLineageService.java
Lines 84 (patched)
<https://reviews.apache.org/r/71902/#comment307030>

    useGraphQueryForLineage => LINEAGE_USING_GREMLIN



repository/src/main/java/org/apache/atlas/discovery/EntityLineageService.java
Lines 205 (patched)
<https://reviews.apache.org/r/71902/#comment307023>

    To be consistent, consider replacing "UsingGraphQuery" with "V1" as suffix.



repository/src/main/java/org/apache/atlas/discovery/EntityLineageService.java
Lines 264 (patched)
<https://reviews.apache.org/r/71902/#comment307034>

    Consider returning "Collection<AtlasEdge>" from this method, as the caller 
uses only the value of the returned map.



repository/src/main/java/org/apache/atlas/discovery/EntityLineageService.java
Lines 280 (patched)
<https://reviews.apache.org/r/71902/#comment307025>

    How is 'direction == BOTH' handled here? Looks like only 
PROCESS_OUTPUTS_EDGE edges will be traversed. Please review.
    
      AtlasVertex processVertex = AtlasGraphUtilsV2.findByGuid(guid);
    
      if (direction == INPUT || direction == BOTH) {
        Iterable<AtlasEdge> processEdges = 
processVertex.getEdges(AtlasEdgeDirection.OUT, PROCESS_INPUTS_EDGE);
    
        for (AtlasEdge processEdge : processEdges) {
          AtlasVertex datasetVertex = processEdge.getInVertex();
    
          traverseEdges(datasetVertex, INPUT, depth - 1, ret);
        }
      }
    
      if (direction == OUTPUT || direction == BOTH) {
        Iterable<AtlasEdge> processEdges = 
processVertex.getEdges(AtlasEdgeDirection.OUT, PROCESS_OUTPUTS_EDGE);
    
        for (AtlasEdge processEdge : processEdges) {
          AtlasVertex datasetVertex = processEdge.getInVertex();
    
          traverseEdges(datasetVertex, OUTPUT, depth - 1, ret);
        }
      }



repository/src/main/java/org/apache/atlas/discovery/EntityLineageService.java
Lines 303 (patched)
<https://reviews.apache.org/r/71902/#comment307031>

    visitedIds => visitedVertices



repository/src/main/java/org/apache/atlas/discovery/EntityLineageService.java
Lines 308 (patched)
<https://reviews.apache.org/r/71902/#comment307028>

    Given 'BOTH' is not expected here, perhaps the parameter can be:
      "LineageDirection direction" => "boolean isInput"
    
    Same in #299 as well.



repository/src/main/java/org/apache/atlas/discovery/EntityLineageService.java
Lines 312 (patched)
<https://reviews.apache.org/r/71902/#comment307027>

    - can 'incomingEdge' be already present in 'ret'? If yes, rest of this loop 
body can be skipped with a 'continue' at #312. Please review.
    - can 'processVertex' be already visited? If yes, rest of this loop body 
can be skipped with a 'continue' at #313. For this to work, processVertex 
should be added to 'visitedIds'. Please review.



repository/src/main/java/org/apache/atlas/discovery/EntityLineageService.java
Lines 320 (patched)
<https://reviews.apache.org/r/71902/#comment307033>

    Looks like #320 can be moved before entering this 'for' loop, at #315. 
Please review.



repository/src/main/java/org/apache/atlas/discovery/EntityLineageService.java
Lines 340 (patched)
<https://reviews.apache.org/r/71902/#comment307024>

    startVertex => vertex


- Madhan Neethiraj


On Dec. 11, 2019, 11:59 p.m., Sarath Subramanian wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/71902/
> -----------------------------------------------------------
> 
> (Updated Dec. 11, 2019, 11:59 p.m.)
> 
> 
> Review request for atlas, Ashutosh Mestry, Aadarsh Jajodia, keval bhatt, 
> Sridhar K, Le Ma, Mandar Ambawane, mayank jain, Nixon Rodrigues, Sameer 
> Shaikh, and Sarath Subramanian.
> 
> 
> Bugs: ATLAS-3558
>     https://issues.apache.org/jira/browse/ATLAS-3558
> 
> 
> Repository: atlas
> 
> 
> Description
> -------
> 
> Lineage in atlas uses graph query to compute lineage across entities (inputs, 
> outputs or both). Lineage rendering performance have degraded after using 
> janusgraph version 0.4.0
> 
> On investigation, lineage graph query initialization and execution using 
> gremlin script engine has been found to be the bottleneck.
> 
> Alternate in-memory computation of lineage has improved performance by many 
> folds (~90% improvement). This Jira is about adding alternate in-memory 
> computation of lineage.
> 
> "atlas.use.graph.query.for.lineage" property can be used to toggle between 
> graph query and in-memory computation of lineage. The default option will be 
> in-memory.
> 
> 
> Diffs
> -----
> 
>   intg/src/main/java/org/apache/atlas/AtlasConfiguration.java 979bd0ae3 
>   
> repository/src/main/java/org/apache/atlas/discovery/EntityLineageService.java 
> 9a020468d 
> 
> 
> Diff: https://reviews.apache.org/r/71902/diff/1/
> 
> 
> Testing
> -------
> 
> Precommit: 
> https://builds.apache.org/view/A/view/Atlas/job/PreCommit-ATLAS-Build-Test/1581/console
> 
> Manually validated lineage rendering works fine for simple, complex and 
> circular lineages.
> 
> 
> Thanks,
> 
> Sarath Subramanian
> 
>

Reply via email to