[
https://issues.apache.org/jira/browse/HDDS-8580?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Zita Dombi resolved HDDS-8580.
------------------------------
Resolution: Fixed
> Reduce memory usage in ContainerKeyMapperTask#reprocess
> -------------------------------------------------------
>
> Key: HDDS-8580
> URL: https://issues.apache.org/jira/browse/HDDS-8580
> Project: Apache Ozone
> Issue Type: Improvement
> Components: Ozone Recon
> Affects Versions: 1.3.0
> Reporter: Ivan Andika
> Assignee: Ivan Andika
> Priority: Major
> Labels: pull-request-available
> Fix For: 1.4.0
>
> Attachments: image-2023-05-11-11-46-01-929.png
>
>
> As OM performance test during Recon OM full snapshot, I removed the Recon DB
> directory before restarting Recon to trigger full snapshot (essentially
> bootstrapping a new Recon).
> However, it is found after OM DB is successfully downloaded, during the
> {{reprocess}} of {{{}ContainerKeyMapperTask{}}}, the Recon heap usage
> increased significantly for large keys table (our cluster has around 350
> million keys).
> It is found that the issue was caused due to in-memory maps that store all
> the OM keys during the reprocess. This is a regression introduced in
> [HDDS-6783|https://jira.shopee.io/browse/HDDS-6783]. In essence, the patch is
> to revert the implementation of
> [HDDS-6783|https://jira.shopee.io/browse/HDDS-6783] only for
> {{{}ContainerKeyMapperTask#reprocess{}}}.
> * {{ContainerKeyMapperTask#process}} should not increase the heap memory
> significantly since the number of delta updates are already limited by the
> Recon configurations
> * [HDDS-6783|https://jira.shopee.io/browse/HDDS-6783] aims for atomicity
> during the Recon OM task updates. However since
> {{ContainerKeyMapperTask#reprocess}} truncate all the Recon Container DB
> before it starts and rebuilt the Recon Container DB, I think it's acceptable.
> After the patch is applied, the Recon heap size stays stable during the full
> snapshot.
> Attached is Recon heap memory before and after the patch.
> !image-2023-05-11-11-46-01-929.png|width=716,height=546!
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
