[
https://issues.apache.org/jira/browse/HDDS-9641?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
GuoHao updated HDDS-9641:
-------------------------
Description:
Currently the rocksdb that loads schema v3 uses sequential loading and is slow
to load, the purpose of this PR is to speed up the loading of the rocksdb using
parallel loading.
This is my test data:
1. datanode 35 disks * 8TB Wrote about 50% of the capacity
2. after restarting the datanode, it took 5 minutes to load the checksdb.
!image-2023-11-07-17-13-29-406.png!
After using this feature:
{code:java}
2023-11-07 16:25:06,699 [main] INFO
org.apache.hadoop.ozone.container.ozoneimpl.OzoneContainer: Load 35 volumes
DbStore cost: 1532ms {code}
was:
Currently the rocksdb that loads schema v3 uses sequential loading and is slow
to load, the purpose of this PR is to speed up the loading of the rocksdb using
parallel loading.
This is my test data:
1. datanode 35 disks * 14TB Wrote about 50% of the capacity
2. after restarting the datanode, it took 5 minutes to load the checksdb.
!image-2023-11-07-17-13-29-406.png!
After using this feature:
{code:java}
2023-11-07 16:25:06,699 [main] INFO
org.apache.hadoop.ozone.container.ozoneimpl.OzoneContainer: Load 35 volumes
DbStore cost: 1532ms {code}
> Parallel loading datanode volume db store
> -----------------------------------------
>
> Key: HDDS-9641
> URL: https://issues.apache.org/jira/browse/HDDS-9641
> Project: Apache Ozone
> Issue Type: Improvement
> Components: Ozone Datanode
> Reporter: GuoHao
> Assignee: GuoHao
> Priority: Major
> Labels: pull-request-available
> Attachments: image-2023-11-07-17-13-29-406.png
>
>
> Currently the rocksdb that loads schema v3 uses sequential loading and is
> slow to load, the purpose of this PR is to speed up the loading of the
> rocksdb using parallel loading.
> This is my test data:
> 1. datanode 35 disks * 8TB Wrote about 50% of the capacity
> 2. after restarting the datanode, it took 5 minutes to load the checksdb.
> !image-2023-11-07-17-13-29-406.png!
> After using this feature:
> {code:java}
> 2023-11-07 16:25:06,699 [main] INFO
> org.apache.hadoop.ozone.container.ozoneimpl.OzoneContainer: Load 35 volumes
> DbStore cost: 1532ms {code}
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
