Manos, I'm not 100% clear on what you're asking. A way to scale the
total number of connections in a ZooKeeper deployment is by adding
more servers. It is true, though, that adding more servers will
provide more connections, but still a bounded number of connections
(changing the kernel limit or not). I think that you're suggesting
that we should be discarding connections at some point because you
might be concerned with an attack in which a client collapses a server
by reaching its maximum number of connections. If this is what you're
concerned about, then this doesn't fit into our current model. As our
clients are not malicious by assumption, by closing a connection to a
client we would be denying service to a legitimate client. However, I
think it might not be too hard to enforce a total number of
connections per IP by extending our quota mechanism.
-Flavio
On Feb 17, 2009, at 11:11 PM, Manos Kapritsos wrote:
Hi all,
in my experiments with Zookeeper I have tried to have 1024 clients
connected simultaneously to a single server. When that happens i get
a "Too many open files" exception on the server side. It is my
understanding that this is the NR_OPEN limit imposed by the kernel.
Although it is possible to increase this limit, this is only a
static solution, that could fail under a spike in the load.
Is there a dynamic workaround for this problem? Given that sessions
are statically linked to the open sockets, it seems to me that there
is no way we can manipulate the existing sockets (e.g. by closing
them in a LRU way).
Thank you,
Manos
