[
https://issues.apache.org/jira/browse/HDDS-12316?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Hemant Kumar updated HDDS-12316:
--------------------------------
Description:
As part of the changes
([PR#7045|https://github.com/apache/ozone/pull/7045/files#diff-a424f5d3db1b2b8c0ffede0b757478c8ab646ea7f7990fd13f36f0346e6a73e0R63]),
a local map was added to OMSnapshotPurgeRequest to reduce the number of calls
to the DB. The same map was later passed to OMSnapshotPurgeResponse to update
the DB. The problem is that Snapshot to be purged is added to this map as well.
So while updating the DB as part of the doubleBuffer flush, we first update the
snapshot and then purge it. It is not wrong in correctness but unnecessary
update. Also, we log the snapshotToBePurged and updatedMap in
[OMSnapshotPurgeResponse|https://github.com/apache/ozone/pull/7045/files#diff-a424f5d3db1b2b8c0ffede0b757478c8ab646ea7f7990fd13f36f0346e6a73e0R125]
which creates confusion if the entry is deleted, updated, or both.
Here is an example log form a live cluster
{code:java}
ozone-om.log.4:2025-02-10 16:54:53,521 INFO [OM StateMachine ApplyTransaction
Thread - 0]-org.apache.hadoop.ozone.om.request.snapshot.OMSnapshotPurgeRequest:
Successfully executed snapshotPurgeRequest: {snapshotDBKeys:
"/vol/bucket/cm-tmp-5a462285-ca48-4c7a-b2b8-63a590f1d3ef" ozone-om.log.4:}
along with updating
snapshots:{/vol/bucket/cm-tmp-5a462285-ca48-4c7a-b2b8-63a590f1d3ef=SnapshotInfo{snapshotId:
'51a5d74f-526f-42f6-bce7-e54603326626', name:
'cm-tmp-5a462285-ca48-4c7a-b2b8-63a590f1d3ef', volumeName: 'vol', bucketName:
'bucket', snapshotStatus: 'SNAPSHOT_DELETED', creationTime: '1739228072015',
deletionTime: '1739228084229', pathPreviousSnapshotId:
'87c4c52e-ea4e-4e47-8dfa-707655eaf295', globalPreviousSnapshotId:
'a05e17e6-91ec-44b8-918c-5d412fd4dd33', snapshotPath: 'vol/bucket',
checkpointDir: '-51a5d74f-526f-42f6-bce7-e54603326626', dbTxSequenceNumber:
'6249154121', deepClean: 'true', sstFiltered: 'false'}} {code}
This task is to remove the purgedSnapshot from updatedMap before logging and
passing it to the OMSnapshotPurgeResponse to keep it clean.
was:
As part of the changes
([PR#7045|https://github.com/apache/ozone/pull/7045/files#diff-a424f5d3db1b2b8c0ffede0b757478c8ab646ea7f7990fd13f36f0346e6a73e0R63]),
a local map was added to OMSnapshotPurgeRequest to reduce the number of calls
to the DB. The same map was later passed to OMSnapshotPurgeResponse to update
the DB. The problem is that Snapshot to be purged is added to this map as well.
So while updating the DB as part of the doubleBuffer flush, we first update the
snapshot and then purge it. It is not wrong in correctness but unnecessary
update. Also, we log the snapshotToBePurged and updatedMap in
[OMSnapshotPurgeResponse|https://github.com/apache/ozone/pull/7045/files#diff-a424f5d3db1b2b8c0ffede0b757478c8ab646ea7f7990fd13f36f0346e6a73e0R125]
which creates confusion if the entry is deleted, updated, or both.
Here is an example log form a live cluster
{code:java}
ozone-om.log.4:2025-02-10 16:54:53,521 INFO [OM StateMachine ApplyTransaction
Thread - 0]-org.apache.hadoop.ozone.om.request.snapshot.OMSnapshotPurgeRequest:
Successfully executed snapshotPurgeRequest: {snapshotDBKeys: "/vol/bucket/cm-tm
p-5a462285-ca48-4c7a-b2b8-63a590f1d3ef" ozone-om.log.4:} along with updating
snapshots:{/vol/bucket/cm-tmp-5a462285-ca48-4c7a-b2b8-63a590f1d3ef=SnapshotInfo{snapshotId:
'51a5d74f-526f-42f6-bce7-e54603326626', name:
'cm-tmp-5a462285-ca48-4c7a-b2b8-63a590f1d3ef', volumeName: 'vol', bucketName:
'bucket', snapshotStatus: 'SNAPSHOT_DELETED', creationTime: '1739228072015',
deletionTime: '1739228084229', pathPreviousSnapshotId:
'87c4c52e-ea4e-4e47-8dfa-707655eaf295', globalPreviousSnapshotId:
'a05e17e6-91ec-44b8-918c-5d412fd4dd33', snapshotPath: 'vol/bucket',
checkpointDir: '-51a5d74f-526f-42f6-bce7-e54603326626', dbTxSequenceNumber:
'6249154121', deepClean: 'true', sstFiltered: 'false'}} {code}
This task is to remove the purgedSnapshot from updatedMap before logging and
passing it to the OMSnapshotPurgeResponse to keep it clean.
> Snapshot purge first updates deleted snapshot and then purge it.
> ----------------------------------------------------------------
>
> Key: HDDS-12316
> URL: https://issues.apache.org/jira/browse/HDDS-12316
> Project: Apache Ozone
> Issue Type: Sub-task
> Reporter: Hemant Kumar
> Priority: Major
>
> As part of the changes
> ([PR#7045|https://github.com/apache/ozone/pull/7045/files#diff-a424f5d3db1b2b8c0ffede0b757478c8ab646ea7f7990fd13f36f0346e6a73e0R63]),
> a local map was added to OMSnapshotPurgeRequest to reduce the number of
> calls to the DB. The same map was later passed to OMSnapshotPurgeResponse to
> update the DB. The problem is that Snapshot to be purged is added to this map
> as well. So while updating the DB as part of the doubleBuffer flush, we first
> update the snapshot and then purge it. It is not wrong in correctness but
> unnecessary update. Also, we log the snapshotToBePurged and updatedMap in
> [OMSnapshotPurgeResponse|https://github.com/apache/ozone/pull/7045/files#diff-a424f5d3db1b2b8c0ffede0b757478c8ab646ea7f7990fd13f36f0346e6a73e0R125]
> which creates confusion if the entry is deleted, updated, or both.
> Here is an example log form a live cluster
> {code:java}
> ozone-om.log.4:2025-02-10 16:54:53,521 INFO [OM StateMachine ApplyTransaction
> Thread -
> 0]-org.apache.hadoop.ozone.om.request.snapshot.OMSnapshotPurgeRequest:
> Successfully executed snapshotPurgeRequest: {snapshotDBKeys:
> "/vol/bucket/cm-tmp-5a462285-ca48-4c7a-b2b8-63a590f1d3ef" ozone-om.log.4:}
> along with updating
> snapshots:{/vol/bucket/cm-tmp-5a462285-ca48-4c7a-b2b8-63a590f1d3ef=SnapshotInfo{snapshotId:
> '51a5d74f-526f-42f6-bce7-e54603326626', name:
> 'cm-tmp-5a462285-ca48-4c7a-b2b8-63a590f1d3ef', volumeName: 'vol', bucketName:
> 'bucket', snapshotStatus: 'SNAPSHOT_DELETED', creationTime: '1739228072015',
> deletionTime: '1739228084229', pathPreviousSnapshotId:
> '87c4c52e-ea4e-4e47-8dfa-707655eaf295', globalPreviousSnapshotId:
> 'a05e17e6-91ec-44b8-918c-5d412fd4dd33', snapshotPath: 'vol/bucket',
> checkpointDir: '-51a5d74f-526f-42f6-bce7-e54603326626', dbTxSequenceNumber:
> '6249154121', deepClean: 'true', sstFiltered: 'false'}} {code}
> This task is to remove the purgedSnapshot from updatedMap before logging and
> passing it to the OMSnapshotPurgeResponse to keep it clean.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
