sodonnel opened a new pull request #1857:
URL: https://github.com/apache/ozone/pull/1857


   ## What changes were proposed in this pull request?
   
   When a datanode is decommission or put to maintenance, its new state is 
persisted into the datanode.yaml file. When running on a cluster with Recon 
enabled, we can see conflicting commands are received repeatedly on the 
Datanode, eg:
   
   ```
   datanode_3  | 2021-01-29 16:26:20,009 [EndpointStateMachine task thread for 
scm/172.24.0.6:9861 - 0 ] INFO endpoint.HeartbeatEndpointTask: Received SCM set 
operational state command. State: DECOMMISSIONED Expiry: 0 id 3645344
   datanode_3  | 2021-01-29 16:26:50,012 [EndpointStateMachine task thread for 
recon/172.24.0.3:9891 - 0 ] INFO commands.SetNodeOperationalStateCommand: 
Create a new command to set op state IN_SERVICE 0 id is 3675347
   ```
   
   This is happening because Recon delegates processing the DN heartbeats 
received by ReconNodeManager to an instance of SCMNodeManager running inside 
Recon. SCMNodeManager checks the reported state of the datanode matches the SCM 
memory state, and if they don't match, it issues a command to the DN to update 
its state.
   
   In this case, Recon always tries to set the DN state back to IN_SERVICE.
   
   Recon sub-classes the SCMNodeManager where this event is produced. Recon 
filters events it is allowed to send via the onMessage interface on 
SCMNodeManager, but the newly added event for decommission did not use that 
interface and hence bypassed the filter.
   
   This change pushes the even over the onMessage interface to avoid this 
problem.
   
   ## What is the link to the Apache JIRA
   
   https://issues.apache.org/jira/browse/HDDS-4766
   
   ## How was this patch tested?
   
   Existing tests and manually verified in a Docker cluster.
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
[email protected]



---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Reply via email to