snmvaughan opened a new pull request, #4692:
URL: https://github.com/apache/hadoop/pull/4692

   Avoid reconnecting to the old address after detecting that the address has 
been updated.
   
   ### Description of PR
   
   When the IPC Client recognizes that an IP address has changed, it updates 
the server field and logs a message:
   
   `Address change detected. Old: 
journalnode-1.journalnode.hdfs.svc.cluster.local/10.1.0.178:8485 New: 
journalnode-1.journalnode.hdfs.svc.cluster.local/10.1.0.182:8485`
   
   Although the change is detected, the client will continue to connect to the 
old IP address, resulting in repeated log messages.  This is seen in managed 
environments when JournalNode syncing is enabled and a JournalNode is 
restarted, with the remaining nodes in the set repeatedly logging this message 
when syncing to the restarted JournalNode.
   
   The source of the problem is that the remoteId.address is not updated.
   
   ### How was this patch tested?
   
   HA configuration deployed on a Kubernetes cluster, using Java 11.
   
   1. Deleted an individual JournalNode pod
   2. Looked for "Address change detected" messages from each other JournalNode
   3. Looked for "Address change detected" messages from each NameNode (Active 
and Standby)
   4. Waiting long enough for the `JounalNodeSyncer` to  wrap back around to 
the Restarted JournalNode to ensure that the message wasn't repeated and 
communications went directly to the new instance
   5. Waited long enough for the NameNodes to push to the restarted 
JournalNode, restarting other JournalNodes if necessary as a forcing function, 
to ensure that the message wasn't repeated and communications went directly to 
the new instance
   
   ### For code changes:
   
   - [X] Does the title or this PR starts with the corresponding JIRA issue id 
(e.g. 'HADOOP-17799. Your PR title ...')?
   - [ ] Object storage: have the integration tests been executed and the 
endpoint declared according to the connector-specific documentation?
   - [ ] If adding new dependencies to the code, are these dependencies 
licensed in a way that is compatible for inclusion under [ASF 
2.0](http://www.apache.org/legal/resolved.html#category-a)?
   - [ ] If applicable, have you updated the `LICENSE`, `LICENSE-binary`, 
`NOTICE-binary` files?
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]


---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Reply via email to