[ 
https://issues.apache.org/jira/browse/HADOOP-9545?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13653178#comment-13653178
 ] 

Chris Nauroth commented on HADOOP-9545:
---------------------------------------

Hi, Todd.  This is slightly off-topic for this logging jira, but you may want 
to look at ZOOKEEPER-1702/HADOOP-9555 to see if they could somehow explain the 
problem that you saw with your running ZKFC.

This is a bug I found where the ZK client can experience an accidental TCP RST 
instead of a graceful shutdown of its socket.  The symptoms I've seen are that 
the ZK client fails to deliver a session expiration event, and instead the 
{{ActiveStandbyElector}} receives repeated disconnected events.  We don't 
attempt to recover from the disconnected events, because we expect the ZK 
client library to reestablish the connection to the ZK cluster for us 
internally in that case.  So far, I can only get a consistent repro on Windows, 
but it's not really a Windows-specific bug.

I definitely agree on adding more logging though.  This was very challenging to 
troubleshoot.  :-)
                
> Improve logging in ActiveStandbyElector
> ---------------------------------------
>
>                 Key: HADOOP-9545
>                 URL: https://issues.apache.org/jira/browse/HADOOP-9545
>             Project: Hadoop Common
>          Issue Type: Improvement
>          Components: auto-failover, ha
>    Affects Versions: 2.0.5-beta
>            Reporter: Todd Lipcon
>            Assignee: Todd Lipcon
>            Priority: Minor
>
> The ActiveStandbyElector currently logs a lot of stuff at DEBUG level which 
> would be useful for troubleshooting. We've seen one instance in the wild of a 
> ZKFC thinking it should be in standby state when in fact it won the election, 
> but the logging is insufficient to understand why. I'd like to bump most of 
> the existing DEBUG logs to INFO and add some additional logs as well.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Reply via email to