Hello I attached all infos on ISSUE: - Test case - Log file in DEBUG - Thread Dump
Note that I reproduced it every time with what I described in issue. I also attached a patch that works on my tests. I let you check. Regards Philippe Mouawad On Sun, Sep 25, 2011 at 1:46 PM, sebb <[email protected]> wrote: > On 25 September 2011 12:22, sebb <[email protected]> wrote: > > On 25 September 2011 01:45, Milamber <[email protected]> wrote: > >> > >> > >> Le 25/09/2011 00:08, sebb a ecrit : > >>> On 25 September 2011 01:03, Milamber <[email protected]> wrote: > >>> > >>>> > >>>> Le 23/09/2011 23:38, sebb a ecrit : > >>>> > >>>>> On 23 September 2011 23:17, sebb <[email protected]> wrote: > >>>>> > >>>>> > >>>>>> On 23 September 2011 18:14, sebb <[email protected]> wrote: > >>>>>> > >>>>>> > >>>>>>> On 23 September 2011 17:15, Milamber <[email protected]> wrote: > >>>>>>> > >>>>>>> > >>>>>>>> Hello, > >>>>>>>> > >>>>>>>> It's seems all open bugs with 2.5.1RC1 are closed. > >>>>>>>> I will start the RC2 process tomorrow. > >>>>>>>> > >>>>>>>> > >>>>>>> OK. > >>>>>>> > >>>>>>> > >>>>>> Just found a problem with the HC4 retries - they prevent StopTest > from working. > >>>>>> > >>>>>> There's a deadlock in the code that prevents the interrupt from > >>>>>> working when there is a retry. > >>>>>> [At the back of my mind, I thought there was another reason why I > set > >>>>>> the retry count to 0!] > >>>>>> > >>>>>> Shutdown works fine, because it does not try to interrupt the > sample. > >>>>>> > >>>>>> I think there's a work-round I can use in the JMeter code, but I > don't > >>>>>> think we should release the code as is. > >>>>>> > >>>>>> Sorry. > >>>>>> > >>>>>> The Java and HC3.1 samplers work fine; it's only HC4 that has the > problem. > >>>>>> > >>>>>> I'll let you know if there's a solution ASAP. > >>>>>> > >>>>>> > >>>>> URL: http://svn.apache.org/viewvc?rev=1175069&view=rev > >>>>> Log: > >>>>> Temporary hack to work round > >>>>> > >>>>> > >>>> This temporary hack don't always work for me. When I call Stop command > >>>> at the beginning of a test (before end of ramp up), I have the same > >>>> deadlock. > >>>> (but sometimes the stop works fine.) > >>>> > >>> Can you do a thread dump when this happens? > >>> > >> > >> > >> Found one Java-level deadlock: > >> ============================= > >> "Thread-205": > >> waiting to lock monitor 0x0000000000d0bf78 (object 0x00000000e2c89ba8, > >> a org.apache.http.impl.conn.SingleClientConnManager), > >> which is held by "Thread Group 1-1" > >> "Thread Group 1-1": > >> waiting to lock monitor 0x000000000205e638 (object 0x00000000e3425510, > >> a org.apache.http.impl.conn.SingleClientConnManager$ConnAdapter), > >> which is held by "Thread-205" > >> > >> Java stack information for the threads listed above: > >> =================================================== > >> "Thread-205": > >> at > >> > org.apache.http.impl.conn.SingleClientConnManager.releaseConnection(SingleClientConnManager.java:258) > >> - waiting to lock <0x00000000e2c89ba8> (a > >> org.apache.http.impl.conn.SingleClientConnManager) > >> at > >> > org.apache.http.impl.conn.AbstractClientConnAdapter.abortConnection(AbstractClientConnAdapter.java:323) > >> - locked <0x00000000e3425510> (a > >> org.apache.http.impl.conn.SingleClientConnManager$ConnAdapter) > >> at > >> > org.apache.http.client.methods.HttpRequestBase.abort(HttpRequestBase.java:161) > >> at > >> > org.apache.jmeter.protocol.http.sampler.HTTPHC4Impl.interrupt(HTTPHC4Impl.java:1087) > >> at > >> > org.apache.jmeter.protocol.http.sampler.HTTPSamplerProxy.interrupt(HTTPSamplerProxy.java:77) > >> at > >> org.apache.jmeter.threads.JMeterThread.interrupt(JMeterThread.java:580) > >> at > >> > org.apache.jmeter.engine.StandardJMeterEngine.tellThreadsToStop(StandardJMeterEngine.java:552) > >> at > >> > org.apache.jmeter.engine.StandardJMeterEngine.access$300(StandardJMeterEngine.java:58) > >> at > >> > org.apache.jmeter.engine.StandardJMeterEngine$StopTest.run(StandardJMeterEngine.java:284) > >> at java.lang.Thread.run(Thread.java:722) > >> "Thread Group 1-1": > >> at > >> > org.apache.http.impl.conn.AbstractPooledConnAdapter.detach(AbstractPooledConnAdapter.java:106) > >> - waiting to lock <0x00000000e3425510> (a > >> org.apache.http.impl.conn.SingleClientConnManager$ConnAdapter) > >> at > >> > org.apache.http.impl.conn.SingleClientConnManager.shutdown(SingleClientConnManager.java:342) > >> - locked <0x00000000e2c89ba8> (a > >> org.apache.http.impl.conn.SingleClientConnManager) > >> at > >> > org.apache.jmeter.protocol.http.sampler.HTTPHC4Impl.closeThreadLocalConnections(HTTPHC4Impl.java:1072) > >> at > >> > org.apache.jmeter.protocol.http.sampler.HTTPHC4Impl.threadFinished(HTTPHC4Impl.java:1061) > >> at > >> > org.apache.jmeter.protocol.http.sampler.HTTPSamplerProxy.threadFinished(HTTPSamplerProxy.java:71) > >> at > >> > org.apache.jmeter.threads.JMeterThread$ThreadListenerTraverser.addNode(JMeterThread.java:553) > >> at > >> org.apache.jorphan.collections.HashTree.traverseInto(HashTree.java:986) > >> at > org.apache.jorphan.collections.HashTree.traverse(HashTree.java:969) > >> at > >> > org.apache.jmeter.threads.JMeterThread.threadFinished(JMeterThread.java:528) > >> at org.apache.jmeter.threads.JMeterThread.run(JMeterThread.java:308) > >> at java.lang.Thread.run(Thread.java:722) > >> > >> Found 1 deadlock. > >> > >> > >> > >> Full thread dump in attachment. > > > > Thanks! > > > >> Test case is the SimpleTest.jmx, stop after 6-7 VU. > >> For this thread dump, I wait the end of ramp up for 100 VU, but only 99 > >> up. (the 'last' is the TG 1-1) > >> > >> > >>> > >>> > >>>> I thinks we must release the version 2.5.1 to correct the others bugs > >>>> already fixed, and add a known problem in documentation for this > deadlook? > >>>> > >>> Yes, that sounds reasonable. It's not clear yet whether this is a > >>> JMeter or HC4 problem; nor is it clear what to do to fix it. > >>> Anyway, it only occurs sometimes, and it only occurs when trying to > >>> stop the test - so the GUI can just be exitted. > >>> > >> > >> Ok, I will prepare the RC2 tomorrow. > >> (Perhaps open a jmeter bugzilla for this bug?) > > > > I've opened https://issues.apache.org/jira/browse/HTTPCLIENT-1127 in > > case the bug is in HC4. > > Looks like that may have a lock ordering problem. > > > > But it may be caused by a bug in the way JMeter processes shutdown; > > I'm a bit surprised that threadFinished occurs during interrupt > > processing. > > I'll raise a JMeter bug and take a further look later today. > > Raised https://issues.apache.org/bugzilla/show_bug.cgi?id=51888 > > I don't suppose you have the corresponding JMeter log file? > I think we need both dump and log to debug the problem. > > If not, never mind, I'll see if I can trigger the fault again. > > > > >> Milamber > >> > >>> > >>>> Milamber > >>>> > >>>> > >>>> > >>>> > >>>>> https://issues.apache.org/jira/browse/HTTPCLIENT-1120 > >>>>> Note: copying the code from the patch did not seem to work; it looks > >>>>> like isAborted() was not set. > >>>>> Hopefully that is fixed in 4.1.3 > >>>>> > >>>>> That seems to have fixed it for me, or at least improved matters. > >>>>> > >>>>> Still needs more testing to see if the deadlock I found - reported in > >>>>> https://issues.apache.org/jira/browse/HTTPCLIENT-1127 - can still > >>>>> occur. > >>>>> > >>>>> BTW, I found the deadlock testing against the mirror server. > >>>>> > >>>>> --------------------------------------------------------------------- > >>>>> To unsubscribe, e-mail: [email protected] > >>>>> For additional commands, e-mail: [email protected] > >>>>> > >>>>> > >>>>> > >>>>> > >>>> > >>>> --------------------------------------------------------------------- > >>>> To unsubscribe, e-mail: [email protected] > >>>> For additional commands, e-mail: [email protected] > >>>> > >>>> > >>>> > >>> --------------------------------------------------------------------- > >>> To unsubscribe, e-mail: [email protected] > >>> For additional commands, e-mail: [email protected] > >>> > >>> > >>> > >> > >> > > > > --------------------------------------------------------------------- > To unsubscribe, e-mail: [email protected] > For additional commands, e-mail: [email protected] > > -- Cordialement. Philippe Mouawad. Ubik-Ingénierie
