aierate opened a new pull request, #5430:
URL: https://github.com/apache/ozone/pull/5430
## Background
There are two methods for retrieving om roskDB snapshots from om:
incremental update and full update. Incremental update may result in inaccurate
update data due to cluster operations and other reasons, resulting in
inaccurate record statistics. Therefore, it is necessary to add a timed full
update function to update the om roskDB snapshot in a timely and full manner,
ensuring the accuracy of recon statistical information. The following is the
test report.
## Configuration
Enable zone recon to support the XML configuration items recon. om.
snapshot. full. update. interval. enable and recon. om. snapshot. full. update.
interval. delay. Set the time interval for recon to schedule and fully obtain
om roskDB snapshots. The configuration is as follows:
```
<property>
<name>recon.om.snapshot.full.update.interval.delay</name>
<value>5m</value>
<tag>OZONE, RECON</tag>
<description>
Recon regularly fetches ozone manager rocks DB from ozone manager and
updates recon rocks DB information
</description>
</property>
<property>
<name>recon.om.snapshot.full.update.interval.enable</name>
<value>true</value>
<tag>OZONE, RECON</tag>
<description>
Whether to enable recon to fetch ozone manager rocks DB from ozone
manager regularly
</description>
</property>
```
## Test result
**2023-10-12 12:01:29,922**
```
org.apache.hadoop.ozone.recon.spi.impl.OzoneManagerServiceProviderImpl:
Syncing data from Ozone Manager.
2023-10-12 12:01:29,922 [pool-27-thread-1] INFO
org.apache.hadoop.ozone.recon.spi.impl.OzoneManagerServiceProviderImpl:
Obtaining full snapshot from Ozone Manager
2023-10-12 12:01:33,729 [pool-27-thread-1] INFO
org.apache.hadoop.ozone.recon.spi.impl.OzoneManagerServiceProviderImpl: Got new
checkpoint from OM :
/home/hadoop-addfullUpdateXML-03/ozonedata/omdbdir/om.snapshot.db_1697083289922
2023-10-12 12:01:33,734 [pool-27-thread-1] INFO
org.apache.hadoop.ozone.om.helpers.OmKeyInfo: OmKeyInfo.getCodec ignorePipeline
= true
2023-10-12 12:01:34,149 [pool-27-thread-1] INFO
org.apache.hadoop.ozone.recon.recovery.ReconOmMetadataManagerImpl: Created OM
DB handle from snapshot at
/home/hadoop-addfullUpdateXML-03/ozonedata/omdbdir/om.snapshot.db_1697083289922.
```
**2023-10-12 12:06:35,387**
```
2023-10-12 12:06:35,386 [pool-27-thread-1] INFO
org.apache.hadoop.ozone.recon.spi.impl.OzoneManagerServiceProviderImpl: Syncing
data from Ozone Manager.
2023-10-12 12:06:35,387 [pool-27-thread-1] INFO
org.apache.hadoop.ozone.recon.spi.impl.OzoneManagerServiceProviderImpl:
Obtaining full snapshot from Ozone Manager
2023-10-12 12:06:35,449 [pool-27-thread-1] INFO
org.apache.hadoop.ozone.recon.spi.impl.OzoneManagerServiceProviderImpl: Got new
checkpoint from OM :
/home/hadoop-addfullUpdateXML-03/ozonedata/omdbdir/om.snapshot.db_1697083595387
2023-10-12 12:06:35,450 [pool-27-thread-1] INFO
org.apache.hadoop.ozone.recon.recovery.ReconOmMetadataManagerImpl: Cleaning up
old OM snapshot db at
/home/hadoop-addfullUpdateXML-03/ozonedata/omdbdir/om.snapshot.db_1697083289922.
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
