[ 
https://issues.apache.org/jira/browse/ZOOKEEPER-2707?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15940770#comment-15940770
 ] 

Michael Han edited comment on ZOOKEEPER-2707 at 3/24/17 5:35 PM:
-----------------------------------------------------------------

This should (hopefully) be fixed with ZOOKEEPER-2737. The "unexpected beans" 
were leaked connection beans caused by ZOOKEEPER-2737 and the connection only 
leaks when we write to a channel but for some reasons, the exception happens 
while we are writing. Because of ZOOKEEPER-2737 we did not close the 
connection, the connection bean associated with the netty connection will not 
be deregistered which causes the the assertion fails here.

On a side note, this is a good example that our unit test catches an important 
issue that left unnoticed for years (probably also because not many people 
using Netty). It's also good that in ZOOKEEPER-1858 that [~fpj] pointed out to 
stick with the original semantic of the bean checks to ensure no bean leak 
otherwise we would not notice this issue while debugging watcher test here.

For reference here is how the leak is identified:
* I noticed that no matter how hard I tried, these watcher test can't be 
reproduced stand alone. It is only reproducible with the rest of test suite. 
* I set up Jenkins jobs to stress test the entire suite. When I am lucky, I get 
the failures of watchers.
* Identify the leaked bean, which contains the session id.
* Mine entire log, find where the session id is created.
* Continuing mine the log, find all references where this session id is 
referenced.
* I find a consistent pattern that around this session id, there is an 
exception thrown inside channel.write.
* Check the code that references channel.write and spotted the connection leak.


was (Author: hanm):
This should (hopefully) be fixed with ZOOKEEPER-2737.

> Fix "Unexpected bean exists!" issue in WatcherTests
> ---------------------------------------------------
>
>                 Key: ZOOKEEPER-2707
>                 URL: https://issues.apache.org/jira/browse/ZOOKEEPER-2707
>             Project: ZooKeeper
>          Issue Type: Sub-task
>          Components: tests
>    Affects Versions: 3.4.10, 3.5.3
>            Reporter: Abraham Fine
>            Assignee: Michael Han
>
> All the WatcherTests occasionally fail with: 
> {code}
> Error Message:
> Unexpected bean exists! expected:<0> but was:<1>
> Stack Trace:
> junit.framework.AssertionFailedError: Unexpected bean exists! expected:<0> 
> but was:<1>
>       at 
> org.apache.zookeeper.test.ClientBase.verifyUnexpectedBeans(ClientBase.java:498)
>       at org.apache.zookeeper.test.ClientBase.startServer(ClientBase.java:477)
>       at org.apache.zookeeper.test.ClientBase.setUp(ClientBase.java:460)
>       at org.apache.zookeeper.test.WatcherTest.setUp(WatcherTest.java:76)
> {code}
> Here is an example: 
> https://builds.apache.org/job/ZooKeeper_branch35_openjdk7/422/



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

Reply via email to