[
https://issues.apache.org/jira/browse/HDDS-5330?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sammi Chen reassigned HDDS-5330:
--------------------------------
Assignee: Siddhant Sangwan
> Datanode commands are not always invalidated when the SCM leader switches
> --------------------------------------------------------------------------
>
> Key: HDDS-5330
> URL: https://issues.apache.org/jira/browse/HDDS-5330
> Project: Apache Ozone
> Issue Type: Bug
> Reporter: Stephen O'Donnell
> Assignee: Siddhant Sangwan
> Priority: Major
>
> A datanode should only process commands from the SCM which is the leader.
> In StateContext.getNextCommand(), there is logic to update the current leader
> SCM term for each command seen on the DN. It picks the command, and the
> updates the term based on the term stored in the command:
> {code}
> updateTermOfLeaderSCM(command);
> if (command.getTerm() == termOfLeaderSCM.get()) {
> return command;
> }
> {code}
> There are a few problems here:
> 1) If there are commands in the queue with a newer term, then the term stored
> in the DN will not be updated until all the pending commands have been
> processed. Therefore the SCM switch can have happened and the stale commands
> continue to be processed.
> 2) While there is a single command queue, there are further sub-queues. For
> example DeleteContainerCommandHandler places the commands into a executor
> queue. Similar for ReplicateContainerCommandHandler. These queues could be
> quite large and hence stale commands could be processed.
> For (1), I believe the term should be updated when the commands are enqueued,
> not dequeued. That would ensure new commands update the term and invalidate
> the old commands immediately.
> For (2), we should check the DN term prior to executing the command and drop
> the command if it is no longer valid.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
