[ https://issues.apache.org/jira/browse/ATLAS-3797?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ashutosh Mestry updated ATLAS-3797: ----------------------------------- Attachment: (was: ATLAS-3797-Refactoring-Improve-Edge-Creation.patch) > Entity Creation: Refactoring: Improve Edge Creation > --------------------------------------------------- > > Key: ATLAS-3797 > URL: https://issues.apache.org/jira/browse/ATLAS-3797 > Project: Atlas > Issue Type: Improvement > Components: atlas-core > Affects Versions: trunk > Reporter: Ashutosh Mestry > Assignee: Ashutosh Mestry > Priority: Major > Fix For: trunk > > Attachments: ATLAS-3797-Refactoring-Improve-Edge-Creation.patch > > > *Background* > [ATLAS-3762|https://issues.apache.org/jira/browse/ATLAS-3762] introduced edge > creation improvement by improving on edge existence check. > The same change could be applied to another area within the implementation. > *Investigation* > This was discovered as part of volume testing done for high-volume import. > This included importing over 100M entities. Various parts of the entity > creation process had probes added to check for performance. > During this investigation the edge creation was doing the same existence > check that was addressed in earlier fixes. Re-using the same implementation > showed marked improvement in entity creation. > These were the parameters used for during import. > Node configuration: 128 GB RAM, 40 cores. > {code:java} > atlas.migration.mode.batch.size=1500 > atlas.migration.mode.workers=40{code} -- This message was sent by Atlassian Jira (v8.3.4#803005)