F64116045 commented on code in PR #12016:
URL: https://github.com/apache/ozone/pull/12016#discussion_r4540147573
##########
hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/block/SCMDeletedBlockTransactionStatusManager.java:
##########
@@ -465,11 +481,41 @@ public void onLeaderReady() {
}
public void cleanAllTimeoutSCMCommand(long timeoutMs) {
- scmDeleteBlocksCommandStatusManager.cleanAllTimeoutSCMCommand(timeoutMs);
+ removeUncommittedTransactions(scmDeleteBlocksCommandStatusManager
+ .cleanAllTimeoutSCMCommand(timeoutMs));
}
void onDatanodeDead(DatanodeID dnId) {
- scmDeleteBlocksCommandStatusManager.onDatanodeDead(dnId);
+ removeUncommittedTransactions(
+ scmDeleteBlocksCommandStatusManager.onDatanodeDead(dnId));
+ }
+
+ /**
+ * Drops the commit map entries of transactions no datanode has committed
+ * yet. Their commands timed out or went to a dead datanode, so the entries
+ * would otherwise wait for ACKs that never come. Nothing is lost: an empty
+ * entry is the same as no entry, and it is recreated when the transaction
+ * is sent again. Entries some datanodes committed are kept, so a resend
+ * only goes to the datanodes still missing.
+ */
+ private void removeUncommittedTransactions(Set<Long> txIds) {
+ int removed = 0;
+ for (Long txId : txIds) {
+ Set<DatanodeID> dns = transactionToDNsCommitMap.get(txId);
+ if (dns != null && dns.isEmpty()
Review Comment:
minor: If a DN dies before any replica has ACKed a tx, this removes the
entry while the other replicas may still be executing it.
Their ACKs then get ignored in commitTransactions since the entry is gone,
and the tx is sent to them again later.
Should we keep the entry if another DN still has a pending command for that
tx?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]