Morning Vadim,
I have just tested cocoon with your patch (by the way, can
you explain me how to diff between 2 files?).
It's a little bit late, and I don't know if I did every-
thing right:
I shoot only 5 threads against /cocoon/hello.html to test
your changes. The load is still high, but it seems to be a
other reason now: java.io.Win32FileSystem.getLastModifiedTime()
now eats 50% CPU time.
org.apache.cocoon.components.source.URLSource.getInfos()
org.apache.cocoon.components.source.URLSource.getLastModified()
org.apache.cocoon.sitemap.Handler.hasChanged()
org.apache.cocoon.sitemap.Manager.getHandler()
org.apache.cocoon.sitemap.Manager.invoke()
org.apache.cocoon.Cocoon.process()
are invokers of this method.
I attached the trace file to this mail. I will test it
again tomorrow. I have a 14 days trail software, that
should be sufficient :).
Will catch some sleep now...
Cheers
Gerhard
>-----Ursprungliche Nachricht-----
>Von: Vadim Gritsenko [mailto:[EMAIL PROTECTED]]
>Gesendet: Thursday, August 23, 2001 1:03 AM
>An: [EMAIL PROTECTED]
>Betreff: RE: LoadTest
>
>
>Marcus,
>Do not use cocoon20b2 - this is really old beta release!
>Switch to cocoon_20_branch - that's the RC Berin is talking about.
>
>Vadim
>
>> -----Original Message-----
>> From: Marcus Crafter [mailto:[EMAIL PROTECTED]]
>> Sent: Wednesday, August 22, 2001 6:58 PM
>> To: [EMAIL PROTECTED]
>> Subject: Re: LoadTest
>>
>>
>> Hi Berin,
>>
>> Hope all is well mate. Thanks again for your advice.
>Please, don't be
>> afraid to give out more! :-)
>>
>> On Wed, 22 Aug 2001, Berin Loritsch wrote:
>>
>> > Marcus Crafter wrote:
>> > >
>> > > We had a range of errors:
>> > >
>> > > o components not found: fixed recently in avalon's
>DefaultPool
>> > > (also seems to have improved what was before a memory leak)
>> >
>> > I'm glad to see this is no longer a problem. Let me know if it rears
>> > its ugly head again.
>>
>> Sure will. :-)
>>
>> > > o decreased performance over time (100 users lasts
>about 20 minutes,
>> > > 50 users lasts about 40 minutes with our
>application, before
>> > > performance drops)
>> >
>> > What is the load (beyond number of users: time between responses, type
>> > of resource).
>>
>> The load actually changed with today's source code. Thanks
>to all that
>> checked in changes today. Yesterday the load was less on our system
>> (averaging between 3 and 6, for 50 and 100 users).
>>
>> With the changes from today (which have helped dramatically)
>> the load is much higher, around 7-14 with the same user counts.
>>
>> Performance in general is actually much better than before,
>even though
>> the machine seems to be under more load.
>>
>> > Probably should use the cocoon_2_0 branch (check the site for
>the proper
>> > name).
>> > That is what is being prepared for production readiness.
>>
>> Ok, we've switched over to the cocoon20b2 branch.
>>
>> > Cocoon.xconf provides attributes to allow you to manipulate
>> > the size of Component pools for Poolable Components:
>> >
>> > pool-min // Minimum number of Components in the pool
>> > pool-max // Maximum number of Components maintained in the pool
>> > pool-grow // Number of Components to add to the pool each
>time we run
>> > out.
>> >
>> > The Pool will basically turn into a Factory when the number of
>> > Components in
>> > the pool exceed pool-max. This is a performance drain.
>>
>> Ok. Excellent advice. :-)
>>
>> We pumped up our pool settings and saw an immediate improvement in
>> general application performance, especially when testing
>directly with
>> Tomcat (vs via a ajp13 connector).
>>
>> I've increased pooling settings on as much components as I
>can in our
>> cocoon.xconf and sitemap.xmap files (min 32 to max 64 for 50
>> simultaneous users).
>>
>> There are still several components which are regularly being
>> decommissioned in the log files during our tests, that I cannot find
>> locations where I can set pooling parameters for. These are:
>>
>> org.apache.cocoon.components.language.markup.xsp.XSPMarkupLanguage
>> org.apache.avalon.excalibur.component.ExcaliburComponentSelector
>>
>> and all generated xsp classes, eg:
>>
>> org.apache.cocoon.www.client.common.mask.mask_xsp_xml
>> (custom generated xsp class).
>>
>> Is it possible to set pooling sizes on these components too ?
>>
>> > Increase your pool sizes and cache sizes. If your JVM is not
>set to use
>> > all 2 gig of RAM, then it will perform garbage collection more often.
>>
>> Yep. Have done already, we're running with mx1526 ms1024m,
>although I
>> missed the MRU settings. I've increased the MRUMemory sizes
>today from
>> 60m to 200m (for each event cache, stream cache, and the store).
>>
>> > > Runaway threads ? Could cause such a problem, but
>they wouldn't allow
>> > > the system to regain itself - or should I not assume this ?
>> >
>> > I don't think this is a problem. You would see your CPU maxed out.
>>
>> I agree.
>>
>> > > Threading issues ? We've also seen cases in the log
>files under load
>> > > where a single (same) request is handled by 4
>separate threads, all
>> > > starting in the same second - quite vexing.
>> >
>> > That _might_ be your Servlet Engine. The last time I ran my tests I
>> > used Caucho's Resin. It is pretty decent, although it has some
>> > synchronization issues of its own regarding archived files
>(i.e. jar files).
>> > They are so severe that the entire JVM crashes.
>>
>> Ouch. Ok, I'll try everything with Resin as well, and also with
>> Tomcat 4. BTW - Have you (or anyone else) experienced
>performance/load
>> problems when using the apache connectors at all ?
>>
>> > > Our environment is a on a Sun Enterprise 250
>(2x450mhz UltraII, 2gig
>> > > ram), with Tomcat 3.2.3, CVS Cocoon2, and our
>reporting application
>> > > (generates various html tables and pdf reports).
>> >
>> > That sounds very impressive. Can I have one? ;P
>>
>> Sure, we have 5 of them here :-)
>>
>> That's it so far. Thanks very much for your advice Berin, we'll
>> continue to test here with the ideas you and others have
>laid out with
>> the current codebase. As we do more testing I'll send in more
>> information.
>>
>> Cheers,
>>
>> Marcus
>>
>>
>> --
>> .....
>> ,,$$$$$$$$$, Marcus Crafter
>> ;$' '$$$$: Computer Systems Engineer
>> $: $$$$: Open Software Associates GmbH
>> $ o_)$$$: 82-84 Mainzer Landstrasse
>> ;$, _/\ &&:' 60327 Frankfurt Germany
>> ' /( &&&
>> \_&&&&' Email : [EMAIL PROTECTED]
>> &&&&. Business Hours : +49 69 9757 200
>> &&&&&&&:
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: [EMAIL PROTECTED]
>> For additional commands, email: [EMAIL PROTECTED]
>>
>
>_________________________________________________________
>Do You Yahoo!?
>Get your free @yahoo.com address at http://mail.yahoo.com
>
>
>---------------------------------------------------------------------
>To unsubscribe, e-mail: [EMAIL PROTECTED]
>For additional commands, email: [EMAIL PROTECTED]
>
Profiler output for thread Thread-14 . application (CPU profiler output - Sampler /
Methods)
---------------------------------------------------------------------------------------------
Description of CPU usage for thread Thread-14
50.67% - 5053 ms - java.io.Win32FileSystem.getLastModifiedTime()
50.67% - 5053 ms - java.io.File.lastModified()
49.15% - 4902 ms - org.apache.tomcat.loader.AdaptiveClassLoader.shouldReload()
49.15% - 4902 ms -
org.apache.tomcat.loader.AdaptiveServletLoader.shouldReload()
49.15% - 4902 ms - org.apache.tomcat.core.ServletWrapper.handleReload()
49.15% - 4902 ms - org.apache.tomcat.core.ServletWrapper.service()
49.15% - 4902 ms -
org.apache.tomcat.core.ContextManager.internalService()
49.15% - 4902 ms -
org.apache.tomcat.core.ContextManager.service()
49.15% - 4902 ms -
org.apache.tomcat.service.http.HttpConnectionHandler.processConnection()
49.15% - 4902 ms -
org.apache.tomcat.service.TcpWorkerThread.runIt()
49.15% - 4902 ms -
org.apache.tomcat.util.ThreadPool$ControlRunnable.run()
49.15% - 4902 ms - java.lang.Thread.run()
1.51% - 151 ms - org.apache.cocoon.components.source.URLSource.getInfos()
1.51% - 151 ms -
org.apache.cocoon.components.source.URLSource.getLastModified()
0.7% - 70 ms - org.apache.cocoon.sitemap.Handler.hasChanged()
0.7% - 70 ms - org.apache.cocoon.sitemap.Manager.getHandler()
0.7% - 70 ms - org.apache.cocoon.sitemap.Manager.invoke()
0.7% - 70 ms - org.apache.cocoon.Cocoon.process()
0.7% - 70 ms -
org.apache.cocoon.servlet.CocoonServlet.service()
0.7% - 70 ms - javax.servlet.http.HttpServlet.service()
0.7% - 70 ms -
org.apache.tomcat.core.ServletWrapper.doService()
0.7% - 70 ms -
org.apache.tomcat.core.Handler.service()
0.7% - 70 ms -
org.apache.tomcat.core.ServletWrapper.service()
0.7% - 70 ms -
org.apache.tomcat.core.ContextManager.internalService()
0.7% - 70 ms -
org.apache.tomcat.core.ContextManager.service()
0.7% - 70 ms -
org.apache.tomcat.service.http.HttpConnectionHandler.processConnection()
0.7% - 70 ms -
org.apache.tomcat.service.TcpWorkerThread.runIt()
0.7% - 70 ms -
org.apache.tomcat.util.ThreadPool$ControlRunnable.run()
0.7% - 70 ms -
java.lang.Thread.run()
0.61% - 61 ms - org.apache.cocoon.generation.FileGenerator.generateKey()
0.2% - 20 ms - org.apache.cocoon.transformation.TraxTransformer.setup()
6.99% - 698 ms - java.lang.StringBuffer.<init>()
5.81% - 580 ms - java.net.SocketInputStream.socketRead()
5.64% - 563 ms - java.net.PlainSocketImpl.socketAccept()
4.54% - 453 ms - java.lang.StringBuffer.toString()
4.18% - 417 ms - java.lang.String.substring()
3.44% - 344 ms - java.lang.StringBuffer.expandCapacity()
2.18% - 218 ms - java.io.Win32FileSystem.canonicalize()
1.2% - 120 ms - java.net.URLStreamHandler.toExternalForm()
1.08% - 108 ms - java.lang.String.replace()
1.04% - 104 ms - java.io.Win32FileSystem.getLength()
0.93% - 93 ms - java.util.Hashtable.put()
0.91% - 91 ms - java.io.File.lastModified()
0.86% - 86 ms - java.lang.String.toUpperCase()
0.79% - 79 ms - java.util.HashMap.put()
0.74% - 74 ms - java.lang.Integer.toString()
0.68% - 68 ms - java.util.HashMap.<init>()
0.65% - 65 ms - java.net.SocketOutputStream.socketWrite()
0.6% - 60 ms - java.lang.Long.toString()
0.51% - 51 ms - sun.io.Converters.getConverterClass()
0.48% - 48 ms - java.io.Win32FileSystem.normalize()
0.45% - 45 ms - java.lang.Object.clone()
0.34% - 34 ms - java.lang.String.<init>()
0.34% - 34 ms - java.lang.StringBuffer.append()
0.32% - 32 ms - java.io.Win32FileSystem.resolve()
0.31% - 31 ms - java.util.Hashtable$Enumerator.nextElement()
0.29% - 29 ms - java.io.File.toURL()
0.28% - 28 ms - java.util.Hashtable.<init>()
0.28% - 28 ms - java.lang.String.<init>()
0.28% - 28 ms - java.lang.String.charAt()
0.27% - 27 ms - java.lang.Throwable.fillInStackTrace()
0.25% - 25 ms - java.lang.String.toCharArray()
0.18% - 18 ms - java.net.URL.toString()
0.17% - 17 ms - java.lang.StringBuffer.append()
0.17% - 17 ms - java.io.Win32FileSystem.getBooleanAttributes()
0.16% - 16 ms - java.util.AbstractList.iterator()
0.16% - 16 ms - java.util.ArrayList.<init>()
0.12% - 12 ms - java.lang.Class.newInstance0()
0.11% - 11 ms - java.util.HashMap.keySet()
0.11% - 11 ms - java.lang.Object.toString()
0.1% - 10 ms - java.util.Hashtable.get()
0.1% - 10 ms - java.util.HashMap.getHashIterator()
0.09% - 9 ms - java.io.File.<init>()
0.07% - 7 ms - java.net.Socket.getInputStream()
0.07% - 7 ms - java.net.Socket.getOutputStream()
0.06% - 6 ms - java.lang.StringBuffer.<init>()
0.06% - 6 ms - java.util.LinkedList.addBefore()
0.05% - 5 ms - java.lang.String.indexOf()
0.05% - 5 ms - java.io.Win32FileSystem.isLetter()
0.05% - 5 ms - java.lang.String.indexOf()
0.05% - 5 ms - java.net.ServerSocket.implAccept()
0.05% - 5 ms - java.lang.String.getChars()
0.05% - 5 ms - java.lang.String.<init>()
0.05% - 5 ms - java.io.Win32FileSystem.resolve()
0.05% - 5 ms - java.lang.System.arraycopy()
0.05% - 5 ms - java.lang.ref.Finalizer.register()
0.05% - 5 ms - java.util.HashMap.get()
0.05% - 5 ms - sun.io.Converters.newConverter()
0.05% - 5 ms - java.lang.ThreadLocal.set()
0.05% - 5 ms - java.lang.String.startsWith()
0.05% - 5 ms - java.net.SocketInputStream.available()
0.05% - 5 ms - java.util.Collections$SynchronizedMap.get()
0.05% - 5 ms - java.lang.Integer.toUnsignedString()
0.05% - 5 ms - java.net.PlainSocketImpl.getOutputStream()
---------------------------------------------------------------------
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, email: [EMAIL PROTECTED]
