Re: java heap size

2010-03-16 Thread Ted Dunning
Of course, with only 32 - 200 MB of heap, almost anything will collect pretty quickly! If you are only storing a little bit of data (coordination, basically) then a small heap might work fine. Turn on gc logging and read what you get. If that is ok even with high load, then you should be fine.

Re: java heap size

2010-03-16 Thread Patrick Hunt
We have some ZK clusters that service 1 clients, using ZK for group membership, leader election, sharding, etc... Additionally we have clusters that service multi-tennant instances (more than one service using a ZK cluster). Add on top of that the poor performance of the GC in 1.6 jvms and

Re: java heap size

2010-03-16 Thread Lei Zhang
Thanks for your reply Mahadev - I wonder if you can elaborate on how you get to your observation of most applications that use ZooKeeper needing JVM heap size > 64M. Our platform uses ZooKeeper to manager cluster of hundreds of nodes with -Xmx32m. The only time I observed OutOfMemory issue is when

Re: java heap size

2010-03-16 Thread Mahadev Konar
I am not sure I understand it. Lei, the default JVM size actually would not fit the JVM heap size of most of the applications that use ZooKeeper. Its always good to understand the requirements that are needed to run a system. I am not sure how its confusing. Is it the language or how to set it? T

Re: java heap size

2010-03-15 Thread Lei Zhang
Sorry I was too terse - I meant to say the default JVM heap size setting probably fits the need of most applications. Singling out this setting in Zookeeper Admin Guide is confusing rather than helping.

Re: java heap size

2010-03-15 Thread Lei Zhang
Well JVM's default maximum heap size is 64M according to this . I think this is a doc bug.

Re: java heap size

2010-03-15 Thread Ted Dunning
Your understanding is correct. But if you set a heap size nearly as big as your physical memory (or larger) then java may allocate that heap which will cause swapping. So swapping is definitely done by the OS, but it is the applications like Java that can cause the OS to do it. On Mon, Mar 15, 2

java heap size

2010-03-15 Thread Lei Zhang
Hi: I am puzzled by the Zookeeper Admin Guide's advice on choosing Java heap size: "Set the Java heap size. This is very important to avoid swapping, which will seriously degrade ZooKeeper performance. To determine the correct value, use load tests, and make sure you are well below