Priyesh Karatha created HDDS-17152:
--------------------------------------
Summary: Reconcile delete progress for container replicas that
register after their delete transactions are purged
Key: HDDS-17152
URL: https://issues.apache.org/jira/browse/HDDS-17152
Project: Apache Ozone
Issue Type: Improvement
Components: SCM
Reporter: Priyesh Karatha
SCMDeletedBlockTransactionStatusManager#commitTransactions purges a
delete-block transaction once every known replica of the container has ACKed
it. HDDS-16144 holds back the purge while the container has a pending replica
ADD. Two cases are still not covered, and in both a replica can end up keeping
blocks that were already deleted, with no way for SCM to resend the transaction:
1. Replication ADD expires before the copy registers.
- On the datanode, export and delete recording are serialized under the
container lock, so a copy only misses a deletion if it was exported before the
source recorded it.
- SCM defers the purge while that copy's ADD is pending in
ContainerReplicaPendingOps.
- The datanode checks the command deadline only before it starts the
replication task. A slow import can therefore finish after SCM's pending ADD
expires (hdds.scm.replication.event.timeout, default 12m, against a datanode
deadline 6m earlier).
- If SCM purges in that gap, the copy registers later with the blocks and no
pending-delete record.
2. EC reconstruction rebuilds blocks that are pending deletion.
- With schema V2/V3, marking blocks for deletion only adds a row to the
delete transaction table. The blocks stay in the block data table, so listBlock
still returns them.
- ECReconstructionCoordinator rebuilds those blocks on the target, which has
no delete record. This happens whenever reconstruction runs after a purge but
before the sources physically delete the blocks, so it is not limited to the
race above.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
