[
https://issues.apache.org/jira/browse/ATLAS-3519?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16970362#comment-16970362
]
Madhan Neethiraj commented on ATLAS-3519:
-----------------------------------------
[~leslizhang] - thanks for details of the issue and the patch that improves the
lineage query performance from 3/sec to 200/sec!
GremlinGroovyScriptEngine returned by AtlasJanusGraph.getGremlinScriptEngine()
may not safe for use in multiple threads simultaneously i.e. side-effects of an
execution in a thread might be visible in other threads. I suggest to use a
ThreadLocal<GremlinGroovyScriptEngine> in AtlasJanusGraph, to instantiate and
use one script engine per thread.
> pool performance in hive lineage query
> ---------------------------------------
>
> Key: ATLAS-3519
> URL: https://issues.apache.org/jira/browse/ATLAS-3519
> Project: Atlas
> Issue Type: Improvement
> Components: atlas-core
> Affects Versions: 1.1.0, 2.0.0
> Reporter: zhangxiong
> Priority: Minor
> Labels: lineage, performance
> Time Spent: 20m
> Remaining Estimate: 0h
>
> when query lineage info for hive, the throughput reachs only 3 query per
> second.
> we use jstack and jfr+flame graph tools and found that the
> GremlinGroovyScriptEngine object loading class logic takes much of the time.
> so we make the GremlinGroovyScriptEngine a global singleton object, which
> would be shared accross all query threads. finally the query performance
> improved much and got 200 qps.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
