To answer your last question first, no you don't have to do anything explicit to keep the ZK connection alive. It is maintained by a dedicated thread. You do have to keep your java program responsive and ZK problems like this almost always indicate that you have a problem with your program checking out for extended periods of time.
My strong guess is that you have something evil happening with your java process that is actually causing this delay. Since you have tiny memory, it probably isn't GC. Since you have a bunch of processes, swap and process wakeup delays seem plausible. What is the load average on your box? On the topic of your application, why you are using processes instead of threads? With threads, you can get your memory overhead down to 10's of kilobytes as opposed to 10's of megabytes. Also, why not use something like Bixo so you don't have to prototype a threaded crawler? On Tue, Sep 21, 2010 at 8:24 AM, Tim Robertson <timrobertson...@gmail.com>wrote: > Hi all, > > I am seeing a lot of my clients being kicked out after the 10 minute > negotiated timeout is exceeded. > My clients are each a JVM (around 100 running on a machine) which are > doing web crawling of specific endpoints and handling the response XML > - so they do wait around for 3-4 minutes on HTTP timeouts, but > certainly not 10 mins. > I am just prototyping right now on a 2xquad core mac pro with 12GB > memory, and the 100 child processes only get -Xmx64m and I don't see > my machine exhausted. > > Do my clients need to do anything in order to initiate keep alive > heart beats or should this be automatic (I thought the ticktime would > dictate this)? > > # my conf is: > tickTime=2000 > dataDir=/Volumes/Data/zookeeper > clientPort=2181 > maxClientCnxns=10000 > minSessionTimeout=4000 > maxSessionTimeout=800000 > > Thanks for any pointers to this newbie, > Tim >
