"A" (/10.30.84.207 a zookeeper client) is attempting to establish more then 10 sessions to the ZooKeeper server where you got the log. This can be caused by a bug in user code (we've seen bugs where incorrectly implemented ZK clients attempt to create an infinite number of sessions, which essentially DOS the service, so we added the maxClientCnxn default limit of 10).
Often users see this problem when they are trying to simulate a real environment - they run a simulated set of clients sessions (>10) from a single host (ip) hitting the servers. However in your case I'm guessing that it has something to do with this "The observers are running in a different data center from where the ZK non-observers are running." Could you have a NAT or some other networking configuration that makes all the observers seem to be coming from the same IP address? Patrick On Tue, Oct 5, 2010 at 9:33 AM, Avinash Lakshman <avinash.laksh...@gmail.com > wrote: > Thanks Patrick. But what does this mean? I see the log on server A telling > me "Too many connections from A - default is 10". Too many connection from > A > to whom? I do not see who the other end of the connection is. > > Cheers > Avinash > > On Tue, Oct 5, 2010 at 9:27 AM, Patrick Hunt <ph...@apache.org> wrote: > > > See this configuration param in the docs "maxClientCnxns": > > > > > http://hadoop.apache.org/zookeeper/docs/current/zookeeperAdmin.html#sc_advancedConfiguration > > > > < > > > http://hadoop.apache.org/zookeeper/docs/current/zookeeperAdmin.html#sc_advancedConfiguration > > > > > Patrick > > > > On Tue, Oct 5, 2010 at 8:10 AM, Avinash Lakshman < > > avinash.laksh...@gmail.com > > > wrote: > > > > > I find this happening in my observers node in the logs. The observers > are > > > running in a different data center from where the ZK non-observers are > > > running. The only way to fix this seems to be restarting. How can I > start > > > addressing this? Here is the stack trace. > > > > > > Too many connections from /10.30.84.207 - max is 10 > > > WARN - Session 0x0 for server mybox.mydomain.com/10.30.84.207:5001, > > > unexpected error, closing socket connection and attempting reconnect > > > java.io.IOException: Connection reset by peer > > > at sun.nio.ch.FileDispatcher.read0(Native Method) > > > at sun.nio.ch.SocketDispatcher.read(SocketDispatcher.java:21) > > > at sun.nio.ch.IOUtil.readIntoNativeBuffer(IOUtil.java:233) > > > at sun.nio.ch.IOUtil.read(IOUtil.java:200) > > > at sun.nio.ch.SocketChannelImpl.read(SocketChannelImpl.java:236) > > > at > > > org.apache.zookeeper.ClientCnxn$SendThread.doIO(ClientCnxn.java:817) > > > at > > > org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1089) > > > > > > Please advice. > > > > > > Cheers > > > Avinash > > > > > >
