[
https://issues.apache.org/jira/browse/HDDS-16029?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ivan Andika updated HDDS-16029:
-------------------------------
Description:
An incident is triggered when SCM is stuck when leader transfer is triggered
while ContainerBalancer is running.
{code:java}
changeToFollower
-> role.shutdownLeaderState(false);
-> stateMachine.leaderEvent().notifyNotLeader(...)
-> ContainerBalancer.stop() < -- blocked here
-> state.setLeader(null);
-> state.updateCurrentTerm(newTerm);
-> role.startFollowerState(...); {code}
ContainerBalancer is blocking due to the Thread.join call and will block
StateMachineUpdater.
The rule of thumb is notification API should NOT be blocking.
!image-2026-07-30-16-42-14-712.png!
was:
An incident is triggered when SCM is stuck when leader transfer is triggered
while ContainerBalancer is running.
{code:java}
changeToFollower
-> role.shutdownLeaderState(false);
-> stateMachine.leaderEvent().notifyNotLeader(...)
-> ContainerBalancer.stop() < -- blocked here
-> state.setLeader(null);
-> state.updateCurrentTerm(newTerm);
-> role.startFollowerState(...); {code}
ContainerBalancer is blocking due to the Thread.join call and will block
StateMachineUpdater.
The rule of thumb is notification API should NOT be blocking.
> ContainerBalancer can cause SCM to be stuck
> -------------------------------------------
>
> Key: HDDS-16029
> URL: https://issues.apache.org/jira/browse/HDDS-16029
> Project: Apache Ozone
> Issue Type: Bug
> Reporter: Ivan Andika
> Assignee: ChenXi
> Priority: Major
> Attachments: image-2026-07-30-16-42-14-712.png
>
>
> An incident is triggered when SCM is stuck when leader transfer is triggered
> while ContainerBalancer is running.
> {code:java}
> changeToFollower
> -> role.shutdownLeaderState(false);
> -> stateMachine.leaderEvent().notifyNotLeader(...)
> -> ContainerBalancer.stop() < -- blocked here
> -> state.setLeader(null);
> -> state.updateCurrentTerm(newTerm);
> -> role.startFollowerState(...); {code}
> ContainerBalancer is blocking due to the Thread.join call and will block
> StateMachineUpdater.
> The rule of thumb is notification API should NOT be blocking.
> !image-2026-07-30-16-42-14-712.png!
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
