This is an automated email from the ASF dual-hosted git repository.
gavinchou pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/doris.git
The following commit(s) were added to refs/heads/master by this push:
new 53e1f62096e [Improve](cloud) Only register watershed txn id once when
BE enters decommissioning state (#67322)
53e1f62096e is described below
commit 53e1f62096e85fda10d24dd7ef65547ecdc694d0
Author: shee <[email protected]>
AuthorDate: Tue Sep 8 10:34:48 2026 +0800
[Improve](cloud) Only register watershed txn id once when BE enters
decommissioning state (#67322)
Related PR: #30243
Problem Summary:
**This caused issues:**
Wrong guard flag. The block checked isDecommissioned() (the final
decommissioned state) instead of isDecommissioning() (the in-progress
state). As long as the BE had not yet reached the terminal
DECOMMISSIONED state, every scheduling round of
[CloudClusterChecker](copilot://navigate?keyword=CloudClusterChecker)
would re-enter this branch and call registerWaterShedTxnId(be.getId())
again. The watershed txn id is only meaningful to be registered once per
decommissioning session; repeatedly registering it on every check
interval produces redundant RPCs/log entries and pollutes the upgrade
manager's internal state.
**Fix**
Change the guard from `!be.isDecommissioned()` to
`!be.isDecommissioning()`, so the branch is only entered on the first
transition into the decommissioning state.
Move `be.setDecommissioning(true)` inside the try block, right after
registerWaterShedTxnId succeeds. This guarantees the in-memory
decommissioning flag is only flipped when the watershed txn id has been
registered successfully; if the RPC fails, the BE stays in its previous
state and will be retried on the next check round.
**Impact**
Eliminates repeated registerWaterShedTxnId calls for a BE that is stuck
in DECOMMISSIONING state.
Makes the decommissioning state transition atomic with respect to the
watershed txn id registration: either both succeed, or neither takes
effect (and the operation is retried next round).
No behavior change for BEs that have already reached
NODE_STATUS_DECOMMISSIONED.
---
.../java/org/apache/doris/cloud/catalog/CloudClusterChecker.java | 6 +++---
1 file changed, 3 insertions(+), 3 deletions(-)
diff --git
a/fe/fe-core/src/main/java/org/apache/doris/cloud/catalog/CloudClusterChecker.java
b/fe/fe-core/src/main/java/org/apache/doris/cloud/catalog/CloudClusterChecker.java
index 0445d48545c..c8eee36f505 100644
---
a/fe/fe-core/src/main/java/org/apache/doris/cloud/catalog/CloudClusterChecker.java
+++
b/fe/fe-core/src/main/java/org/apache/doris/cloud/catalog/CloudClusterChecker.java
@@ -185,14 +185,14 @@ public class CloudClusterChecker extends MasterDaemon {
}
if (status == Cloud.NodeStatusPB.NODE_STATUS_DECOMMISSIONING) {
- if (!be.isDecommissioned()) {
- LOG.info("decommissioned backend: {} status: {}", be,
status);
+ if (!be.isDecommissioning()) {
+ LOG.info("decommissioning backend: {} status: {}", be,
status);
try {
((CloudEnv)
Env.getCurrentEnv()).getCloudUpgradeMgr().registerWaterShedTxnId(be.getId());
+ be.setDecommissioning(true);
} catch (UserException e) {
LOG.warn("failed to register water shed txn id,
decommission be {}", be.getId(), e);
}
- be.setDecommissioning(true);
}
}
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
