Jeffrey Hagelberg created ATLAS-1114:
----------------------------------------
Summary: Performance improvements for create/update entities
Key: ATLAS-1114
URL: https://issues.apache.org/jira/browse/ATLAS-1114
Project: Atlas
Issue Type: Bug
Reporter: Jeffrey Hagelberg
Assignee: Jeffrey Hagelberg
General Atlas Perforance Improvements
Apply performance fixes for create/update entities from IBM fork to Atlas.
During our performance profiling, we found a number of performance hotspots in
JProfiler. Note that our profiling was done with the IBM Graph implementation,
but the performance optimizations are very general in nature and help with
performance on Titan as well. Our main findings were
- multiple queries were being executed for each instance being
created/updated to find matches by unique attribute.
- one query was being executed for each instance being created/updated to
find the corresponding vertex if there is one
- Calculating the value of the full text property was taking a significant
portion of the time to create/update entities, mainly due to its calls to
getVertexForGUID
The changes we put in do the following:
- batch lookups by guid when create/update entities. Execute one
AtlasGraphQuery to find them all.
- batch lookups by unique attribute when create/update entities. Execute one
AtlasGraphQuery per class to find unique attribute matches.
- find all existing vertices up front during create/update entity. Use those
vertices during the graph mapping process to avoid running unnecessary graph
queries
- reuse reference vertices from instance to graph mapping when computing full
text property
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
