Interesting tests. Would like to get more details (like detailed timings for those concurrent users or at least summary stats) once you are done with it if you don't mind sharing.
Now to explain your results: - the DB connections are normal. JR maintains one connection per workspace no matter what the load, so the amount of connections will not change. What might change is how busy those connections are (i.e. how many requests they are executing against the DB) - the connections being idle and CPU usage being 0% suggests waiting for some timeout ... it might be a deadlock, but if that is somewhere in Magnolia itself I would expect it to be rather busy re-trying to either save or read the content rather then simply sleeping. I might be able to tell more if I knew the details of the tests - was it on public or on the author box? Was it just reading the content? Was the content cached by Magnolia? Were you hitting just single piece of content or distributing the requests all over the instance (i.e. randomly retrieving/editing different pieces of content). - Just because you have 800 threads, doesn't mean you are able to serve 800 concurrent users. If the processing of the request takes more time then the average time between requests ... Might be that you simply filled the queue to the limit and Tomcat stopped queuing further requests until the queue was emptied. If you have Tomcat management console installed it would help to see the info from there as well. Also you can try to have some minimal webapp deployed next to Magnolia and see if that one will be still responding even when Magnolia is not ... that should rule out the tomcat involvement. JR uses internal in-memory cache for the items, but it is somewhat limited to about 30MB of content. It would help to know what kind of content were you retrieving in your test - binary or text? Another thing related to threads ... for some ops Magnolia spawn additional threads and so does the JackRabbit. Not sure if those counts into the 800 threads limit or not. If so and if you were updating the content as part of your test, it might be that JR has simply stopped at some point as it was not able to create indexer threads for re-indexing the updated content. It would also help to see what happened just after the silence period was over ... what was written in FS, what was the first few DB requests sent ... Also during the test, do you see memory consumption growing or do you see response times getting longer before the silence? Have you (or can you) try with different GC collectors (e.g. the default one)? Cheers & good luck with more testing. Jan On Mon, 2009-10-05 at 09:15 -0400, Denis Demichev wrote: > Hello All, > > I've been playing with Magnolia for a while trying to asses its > stability. Eventually I put it under the stress on previous week - 800 > concurrent users on a single box. > > Server: Windows Server 2007(?) SP1 64 bit, Intel Xeon 2 Cores, 4Gb > RAM > JDK 1.6.0_16 64-bit version > Magnolia 4.1 Community Edition (STK Installed, Debug log is disabled) > + MySQL 5.1.39 > Tomcat: maxThreads="800" minSpareThreads="25" maxSpareThreads="75" > Tomcat startup params: > JAVA_OPTS=%JAVA_OPTS% -Xms2G -Xmx2G -XX:MaxPermSize=1024M -XX: > +UseConcMarkSweepGC -XX:+CMSIncrementalMode > -Djava.net.preferIPv4Stack=true -XX:+DoEscapeAnalysis > Client - J-Meter, 800 Concurrent threads, 5 static links + 1 search > link, Gaussian timer set for 1000ms+-1500ms > Response time 954ms, Errors 17% > Avg. CPU utilization ~85%, RAM Utilization ~2Gb > > Now the strange behavior I noticed is following: Magnolia is > responding to incoming requests normally for about 2-5 minutes (more > or less) and then just freezes. CPU usage = 0%, RAM is the same (GC is > obviously not running). No responses at all. > After removing the stress from Magnolia it starts responding in 30-40 > seconds. > 500 Concurrent users will produce the same effect with the only > exception it might take 50-100 minutes (maybe more) to reproduce. > Tried to look at MySQL connection stack using remote admin - 8 > connections all of them are idle. Seems like all the content was > cached by JCR previously. > > Question: What exactly might cause this strange freezing? It looks > like tomcat has enough maximum threads, connection pool to MySQL is > not that busy... Could it be some internal cache of jackrabbit? Not > sure but it looks like JVM is not the rootcause in this case. > > Would appreciate your thoughts on this point. > Thank you in advance! > > Regards, > Denis ---------------------------------------------------------------- For list details see http://www.magnolia-cms.com/home/community/mailing-lists.html To unsubscribe, E-mail to: <[email protected]> ----------------------------------------------------------------
