Great. You should have the gc logs then. Mind creating a jira and uploading to it?
mahadev 2012/2/10 César Álvarez Núñez <[email protected]>: > This is the java.env file content. > > now=`date +%d%m%Y_%H%M%S` > > gcLogFile="/srv/zk/GC/`hostname`-${now}.log" > gcOpts="${gcOpts} -Xloggc:$gcLogFile" > gcOpts="${gcOpts} -XX:+PrintGC" > gcOpts="${gcOpts} -XX:+PrintGCTimeStamps" > gcOpts="${gcOpts} -XX:+PrintGCDetails" > gcOpts="${gcOpts} -XX:+PrintTenuringDistribution" > gcOpts="${gcOpts} -XX:+PrintHeapAtGC" > > gcOpts="${gcOpts} -XX:+AggressiveHeap" > #https://cwiki.apache.org/confluence/display/ZOOKEEPER/Troubleshooting > #gcOpts="${gcOpts} -XX:+UseConcMarkSweepGC" > #gcOpts="${gcOpts} -XX:ParallelGCThreads=8" > #gcOpts="-Xms128M -Xmx1G" > > jvmOpts="${jvmOpts} -d64" > jvmOpts="${jvmOpts} -server" > jvmOpts="${jvmOpts} -XX:+UseCompressedOops" > > JVMFLAGS="${gcOpts} ${jvmOpts}" > > /César. > > On Thu, Feb 9, 2012 at 7:17 PM, Mahadev Konar <[email protected]>wrote: > >> This is interesting and important. >> >> Cesar, what jvm options are you running with? Can you the options in: >> >> https://cwiki.apache.org/confluence/display/ZOOKEEPER/Troubleshooting >> >> Atleast get the GC logs that we can look at? >> >> This will be very interesting. >> >> mahadev >> >> >> 2012/2/9 César Álvarez Núñez <[email protected]>: >> > In my case, our stress test show up a linear increase of "tenured memory" >> > from 0 to > 3GiB with ZK 3.4.0 whereas the same stress-test with 3.3.3 >> > keeps "tenured memory" stable and < 10MiB. >> > >> > The stress test performs many zNodes creation and delete but the overall >> zk >> > usage at any moment in time was relative small. >> > >> > BR, >> > /César. >> > >> > On Thu, Feb 9, 2012 at 3:14 PM, Camille Fournier <[email protected]> >> wrote: >> > >> >> This is really a question about how the jvm grows its heaps and resizes >> >> them. If the jvm cannot allocate enough memory for the process because >> you >> >> didn't set the max memory high enough, it will fall over. Zookeeper >> keeps >> >> its entire state in memory for performance reasons, if it were to swap >> that >> >> would be quite bad for performance. >> >> >> >> C >> >> On Feb 8, 2012 8:23 PM, "Mike Schilli" <[email protected]> wrote: >> >> >> >> > We've got a ZooKeeper instance that's using about 5 GB of resident >> >> > memory. Every time we restart it, it starts at 200MB, and then grows >> >> > slowly until it is back at 5 GB. >> >> > >> >> > The large footprint is related to how much data we've got in there. >> >> > What's interesting, though, is that the process size doesn't shrink if >> >> > we purge some of the data. >> >> > >> >> > Now, this isn't a big problem, I'm just curious if the process will >> fall >> >> > over at some point if it can't get more memory or if it'll just make >> due >> >> > by caching less data. >> >> > >> >> > Also, if I remember correctly, there's a configuration variable to set >> >> > the maximum size, what happens if ZK reaches that? >> >> > >> >> > -- -- Mike >> >> > >> >> > Mike Schilli >> >> > [email protected] >> >> > >> >> >> >> >> >> -- >> Mahadev Konar >> Hortonworks Inc. >> http://hortonworks.com/ >> -- Mahadev Konar Hortonworks Inc. http://hortonworks.com/
