Many recent issues that I saw internally is due to incorrect handling or no sufficient testing on ZooKeeper failure scenario in the custom wrapper API or in the applications.
I am thinking that we might be able to expose a few more API calls that allow user write unit tests that cover various failure scenarios (similar to the TestableZookeer in zookeeper test) . This should also minimize the effort on setting the test framework. Ideally, if we have a mock client that don't need a running the server that would be ideal, but I think it is too much effort to write and maintain for all the languages. Our internal test facility is that we have a dedicated ensemble used by all unit tests. This ensure application logic correctness but it is hard to test various failure scenarios. So my current thought is to expose the following functionalities. 1. zookeeper_close() that don't actually send close request to the server: This can be used to simulate a client crash without actually crashing the test program. 2. Allow client to force triggering CONNECTION_LOSS or SESSSION_EXPIRE event: This will allow the user to test their watchers and callback (and possible race condition) Let me know if you have additional suggestions. -- Thawan Kooburat
