[
https://issues.apache.org/jira/browse/ATLAS-3796?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17106704#comment-17106704
]
Sarath Subramanian commented on ATLAS-3796:
-------------------------------------------
+1 for the patch
[^ATLAS-3796-Status-Reporter-processed-set-cleanup.patch]Thanks [~amestry].
> Migration Import Display: StatusReporter Does not Remove Processed Items
> ------------------------------------------------------------------------
>
> Key: ATLAS-3796
> URL: https://issues.apache.org/jira/browse/ATLAS-3796
> Project: Atlas
> Issue Type: Bug
> Components: atlas-intg
> Affects Versions: trunk
> Reporter: Ashutosh Mestry
> Assignee: Ashutosh Mestry
> Priority: Major
> Fix For: trunk
>
> Attachments: ATLAS-3796-Status-Reporter-processed-set-cleanup.patch
>
>
> *Background*
> _Migration Import_ implementation introduced _StatusReporter_. The class
> maintains GUIDs of entities that are to be processed and entities that are
> processed. When the entity is marked as processed, it needs to be removed
> from the processed set.
> *Steps to Duplicate*
> _Pre-requisites_
> Large migration import dataset (over 50 M entities).
> _Steps to Duplicate_
> * Start importing the dataset.
> * After over 60%, the Atlas server will crash with out of memory exception.
> _Expected results_: Migration Import should continue without problems.
> _Actual results_: Atlas server crashes with out of memory exception, hprof
> dump indicates large number of items allocated by
> _StatusReporter.processedSet_.
> _Additional information_: This bug does not affect status reporting.
> _Work around_: Resume migration import. It will continue without problems.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
