Ashutosh Mestry created ATLAS-3797:
--------------------------------------
Summary: Entity Creation: Refactoring: Improve Edge Creation
Key: ATLAS-3797
URL: https://issues.apache.org/jira/browse/ATLAS-3797
Project: Atlas
Issue Type: Improvement
Components: atlas-core
Affects Versions: trunk
Reporter: Ashutosh Mestry
Assignee: Ashutosh Mestry
Fix For: trunk
*Background*
[ATLAS-3762|https://issues.apache.org/jira/browse/ATLAS-3762] introduced edge
creation improvement by improving on edge existence check.
The same change could be applied to another area within the implementation.
*Investigation*
This was discovered as part of volume testing done for high-volume import. This
included importing over 100M entities. Various parts of the entity creation
process had probes added to check for performance.
During this investigation the edge creation was doing the same existence check
that was addressed in earlier fixes. Re-using the same implementation showed
marked improvement in entity creation.
These were the parameters used for during import.
Node configuration: 128 GB RAM, 40 cores.
{code:java}
atlas.migration.mode.batch.size=1500
atlas.migration.mode.workers=40{code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
