[
https://issues.apache.org/jira/browse/ATLAS-1995?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Madhan Neethiraj updated ATLAS-1995:
------------------------------------
Affects Version/s: (was: trunk)
Fix Version/s: 0.8.1
1.0.0
Committed to following branches:
- master: http://git-wip-us.apache.org/repos/asf/atlas/commit/6fb2a038
- branch-0.8: http://git-wip-us.apache.org/repos/asf/atlas/commit/cf07f774
> Performance of Entity Creation Can Be Improved By Using Index Query to Fetch
> Entity Using Unique Attributes
> ------------------------------------------------------------------------------------------------------------
>
> Key: ATLAS-1995
> URL: https://issues.apache.org/jira/browse/ATLAS-1995
> Project: Atlas
> Issue Type: Improvement
> Components: atlas-core
> Affects Versions: 0.8-incubating
> Reporter: Ashutosh Mestry
> Assignee: Ashutosh Mestry
> Priority: Major
> Fix For: 0.8.1, 1.0.0
>
> Attachments: ATLAS-1995.patch
>
>
> *Background*
> On profiling entity creation flow, it was observed that several calls are
> made to _AtlasGraphUtilsV1.getVertexByUniqueAttributes_.
> These calls result in querying database using graph query. There is a
> potential for improving this if index query was used.
> *Analysis*
> Upon experimentation, it was found that there is a 50% improvement in
> performance of entity creation if this method was replaced with equivalent
> that uses _indexQuery_.
> Also, when large number of entities are created (typically using
> _import_hive.sh_), the CPU usage on Atlas was reduced, as the Solr was being
> used for doing some of the work.
> *Implementation Guidance*
> * Add new method to _AtlasGraphUtilsV1.getAtlasVertexFromIndexQuery_ that
> will use _AtlasGraphProvider.indexQuery_ to fetch vertices.
> * Ensure that query created is 'escaped' appropriately.
> * Include logic to fallback to graph query if the property being queried for
> is not indexed.
> Since this is a high-impact change, it will be worth while to verify other
> dependent modules.
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
