Ashutosh Mestry created ATLAS-3797: -------------------------------------- Summary: Entity Creation: Refactoring: Improve Edge Creation Key: ATLAS-3797 URL: https://issues.apache.org/jira/browse/ATLAS-3797 Project: Atlas Issue Type: Improvement Components: atlas-core Affects Versions: trunk Reporter: Ashutosh Mestry Assignee: Ashutosh Mestry Fix For: trunk
*Background* [ATLAS-3762|https://issues.apache.org/jira/browse/ATLAS-3762] introduced edge creation improvement by improving on edge existence check. The same change could be applied to another area within the implementation. *Investigation* This was discovered as part of volume testing done for high-volume import. This included importing over 100M entities. Various parts of the entity creation process had probes added to check for performance. During this investigation the edge creation was doing the same existence check that was addressed in earlier fixes. Re-using the same implementation showed marked improvement in entity creation. These were the parameters used for during import. Node configuration: 128 GB RAM, 40 cores. {code:java} atlas.migration.mode.batch.size=1500 atlas.migration.mode.workers=40{code} -- This message was sent by Atlassian Jira (v8.3.4#803005)