[ 
https://issues.apache.org/jira/browse/ATLAS-3558?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sarath Subramanian updated ATLAS-3558:
--------------------------------------
    Description: 
Lineage in atlas uses graph query to compute lineage across entities (inputs, 
outputs or both). Lineage rendering performance have degraded after using 
janusgraph version 0.4.0

On investigation, lineage graph query initialization and execution using 
gremlin script engine has been found to be the bottleneck.

Alternate in-memory computation of lineage has improved performance by many 
folds (~90% improvement). This Jira is about adding alternate in-memory 
computation of lineage.

*"atlas.lineage.query.use.gremlin"* property can be used to toggle between 
graph query and in-memory computation of lineage. The default option will be 
in-memory.

  was:
Lineage in atlas uses graph query to compute lineage across entities (inputs, 
outputs or both). Lineage rendering performance have degraded after using 
janusgraph version 0.4.0

On investigation, lineage graph query initialization and execution using 
gremlin script engine has been found to be the bottleneck.

Alternate in-memory computation of lineage has improved performance by many 
folds (~90% improvement). This Jira is about adding alternate in-memory 
computation of lineage.

*"atlas.use.graph.query.for.lineage"* property can be used to toggle between 
graph query and in-memory computation of lineage. The default option will be 
in-memory.


> Improve lineage performance using in-memory traversal
> -----------------------------------------------------
>
>                 Key: ATLAS-3558
>                 URL: https://issues.apache.org/jira/browse/ATLAS-3558
>             Project: Atlas
>          Issue Type: Improvement
>          Components:  atlas-core
>    Affects Versions: 2.0.0
>            Reporter: Sarath Subramanian
>            Assignee: Sarath Subramanian
>            Priority: Major
>             Fix For: 2.1.0
>
>         Attachments: image-2019-12-11-15-54-30-483.png
>
>
> Lineage in atlas uses graph query to compute lineage across entities (inputs, 
> outputs or both). Lineage rendering performance have degraded after using 
> janusgraph version 0.4.0
> On investigation, lineage graph query initialization and execution using 
> gremlin script engine has been found to be the bottleneck.
> Alternate in-memory computation of lineage has improved performance by many 
> folds (~90% improvement). This Jira is about adding alternate in-memory 
> computation of lineage.
> *"atlas.lineage.query.use.gremlin"* property can be used to toggle between 
> graph query and in-memory computation of lineage. The default option will be 
> in-memory.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

Reply via email to