On 1/3/07, stephan.lux <[EMAIL PROTECTED]> wrote:
Hi Guillaume, now I have used JProfiler and found the main problem why my loadtest fails with out of memory messages... The reason is simple, due to that the timer fires again while the jobs were retrieved, a lot of more messages are send to the bus. That was to much load for the eip - xpath-splitter, so that I am running out of memory :-) But it seams that there is still an other memory leak, because after each run some megabyte more are left in memory. Even injecting the garbage collector does not free them. I can see in JProfiler, that some thousend java.net.Inet4Address objects were created but not destroyed. What do you think? Maybe a memory leak in the http provider?
Can you track where these objects are allocated from ?
Attached you will find the jprofiler report before and after my test run. http://www.nabble.com/file/5271/Deadlock_Detection_Graph.zip Deadlock_Detection_Graph.zip One more question: The JProfiler deadlock detector reports sometimes while testing that some threads are blocked while accessing a common resource or object. In this case it is the CondVar class of the edu.emory.mathcs.backport.java.util.concurrent.locks package. Is this a kind of in-memory database/store?
These are thread from thread pools, waiting for jobs to process. So it's fine.
Cheers, Stephan gnodet wrote: > > If you run OOM after a certain time, this means you have a memory leak > somewhere ... It may be because one of the components never answers > to exchanges, or one never receives them ... (do you have hand-written > components ?) > Another reason could be that there is lots of messages injected in the > container, > but they are processed slowly, hence after a while, they can't all be kept > in > memory: this can happen if you use a jms consumer which receives lots > of messages. > > You should try configuring a profile and see which objects are kept in > memory. > Also, to make sure that this is a memory leak somewhere, make sure you > inject messages into the bus at a slow rate, so that they can be processed > in a timely manner (to avoid keeping lots of waiting messages in memory). > > On 1/2/07, stephan.lux <[EMAIL PROTECTED]> wrote: >> >> >> stephan.lux wrote: >> > >> > Sure, the performance is suffering if you do a lot of xml conversions, >> but >> > in this cas I only use a small 300 byte xml file and the xslt >> > transformation only copies the content into a new cover. >> > >> > The blocking ressource is the system memory, 512 MB is not enough on >> > windows. Has anyone run some loadtests on unix/linux with more than >> 5000 >> > jobs? How many memory was used? >> > >> > Cheers >> > Stephan >> > >> > >> > gnodet wrote: >> >> >> >> ServiceMix performance mainly depends on the processing performed, >> >> either directly or indirectly (scripting may be slower than java, lots >> >> of xml conversions ...). >> >> Make sure the log level is set to INFO or more. >> >> And also make sure your CPU is used at 100%, else it means >> >> there is a blocking resource (number of threads, IO ...) >> >> >> >> On 12/29/06, stephan.lux <[EMAIL PROTECTED]> wrote: >> >>> >> >>> >> >>> stephan.lux wrote: >> >>> > >> >>> > Hi Guillaume, >> >>> > >> >>> > I have now all of your tipps applied and rerun the test with 5.000 >> >>> jobs. >> >>> > It failes again this out of memory messages and the processing >> stops. >> >>> So I >> >>> > have seperated the service assemblies on two boxes with each 512MB >> of >> >>> RAM >> >>> > for Service Mix. On one maschine once again out of memory messages >> are >> >>> > thrown, but the most of the jobs could be processed. So it seams, >> that >> >>> the >> >>> > script component needs a lot of memory... >> >>> > >> >>> > But for the processing servicemix needs round about 8 hours! That >> is >> >>> not >> >>> > realy acceptable for us, so do you have any more tuning tipps for >> me? >> >>> > >> >>> > >> >>> >> >>> >> >>> >> >>> >> >>> >> >>> >> >>> >> >>> >> >>> >> >>> >> >>> -- >> >>> View this message in context: >> >>> >> http://www.nabble.com/Severe-issues-while-load-testing-tf2845236s12049.html#a8088779 >> >>> Sent from the ServiceMix - User mailing list archive at Nabble.com. >> >>> >> >>> >> >> >> >> >> >> -- >> >> Cheers, >> >> Guillaume Nodet >> >> >> >> >> > >> >> >> -- >> View this message in context: >> http://www.nabble.com/Severe-issues-while-load-testing-tf2845236s12049.html#a8122637 >> Sent from the ServiceMix - User mailing list archive at Nabble.com. >> >> > > > -- > Cheers, > Guillaume Nodet > > </qoute> -- View this message in context: http://www.nabble.com/Severe-issues-while-load-testing-tf2845236s12049.html#a8144674 Sent from the ServiceMix - User mailing list archive at Nabble.com.
-- Cheers, Guillaume Nodet
