If you are running under jdk 6u1 and tapestry 5.0.5 (or greater) then there is a jvm bug that can cause a deadlock.
The bug report is at http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=6571733. It is supposed to be fixed in jdk 6u2 (which was release recently) but I haven't had a chance to test it yet. cheers Ben > -----Original Message----- > From: Martin Grotzke [mailto:[EMAIL PROTECTED] > Sent: Wednesday, 11 July 2007 9:06 AM > To: Tapestry users > Subject: Re: T5: NPE in Base64InputStream and > locked/waitingCheckForUpdatesFilter > > The NPE seems to be caused by a missing t:formdata request parameter. > > I just "reproduced" this (weird) situation by removing the hidden form > field "t:formdata" with firebug, so that I could produce the NPE. > > Although, following request went through well, and the number > of request > processing threads did not increase. > > So there seems to be no (direct) interrelationship between the NPE > and the locked threads. > > Cheers, > Martin > > > > On Tue, 2007-07-10 at 23:43 +0200, Martin Grotzke wrote: > > Hi, > > > > we had an issue with our deployed application that did not respond > > anymore. This happened two or three times in the last 4 days, but > > I was not able to reproduce it until now. > > > > The analysis of the logs showed, that there was a NPE in > > Base64InputStream, and afterwards the application did not respond > > anymore. > > > > When I triggered a thread dump, all 200 tomcat threads were > in status > > WAITING, like this one: > > > > "http-9090-1" daemon prio=10 tid=0x00002aaaf7e1fc00 > nid=0x3f05 waiting on condition > [0x000000004459e000..0x000000004459fbc0] > > java.lang.Thread.State: WAITING (parking) > > at sun.misc.Unsafe.park(Native Method) > > - parking to wait for <0x00002aaab8228360> (a > java.util.concurrent.locks.ReentrantReadWriteLock$NonfairSync) > > at > java.util.concurrent.locks.LockSupport.park(LockSupport.java:158) > > at > java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndC > heckInterrupt(AbstractQueuedSynchronizer.java:712) > > at > java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquir > eShared(AbstractQueuedSynchronizer.java:842) > > at > java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireS > hared(AbstractQueuedSynchronizer.java:1162) > > at > java.util.concurrent.locks.ReentrantReadWriteLock$ReadLock.loc k(ReentrantReadWriteLock.java:594) > > at > org.apache.tapestry.ioc.internal.util.ConcurrentBarrier.withRe > ad(ConcurrentBarrier.java:70) > > at > org.apache.tapestry.internal.services.CheckForUpdatesFilter.se > rvice(CheckForUpdatesFilter.java:110) > > at > $RequestHandler_1139c29ae4a.service($RequestHandler_1139c29ae4a.java) > > at > $RequestHandler_1139c29ae41.service($RequestHandler_1139c29ae41.java) > > at > org.apache.tapestry.services.TapestryModule$11.service(Tapestr yModule.java:1044) > > at > $HttpServletRequestHandler_1139c29ae40.service($HttpServletReq uestHandler_1139c29ae40.java) > > at > org.apache.tapestry.TapestryFilter.doFilter(TapestryFilter.java:135) > > at > org.apache.catalina.core.ApplicationFilterChain.internalDoFilt > er(ApplicationFilterChain.java:235) > > at > org.apache.catalina.core.ApplicationFilterChain.doFilter(Appli > cationFilterChain.java:206) > > at > org.apache.catalina.core.StandardWrapperValve.invoke(StandardW > rapperValve.java:230) > > at > org.apache.catalina.core.StandardContextValve.invoke(StandardC > ontextValve.java:175) > > at > org.apache.catalina.core.StandardHostValve.invoke(StandardHost > Valve.java:128) > > at > org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReport > Valve.java:104) > > at > org.apache.catalina.core.StandardEngineValve.invoke(StandardEn > gineValve.java:109) > > at > org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdap > ter.java:261) > > at > org.apache.coyote.http11.Http11Processor.process(Http11Process > or.java:844) > > at > org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandle r.process(Http11Protocol.java:581) > > at > org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint. java:447) > > at java.lang.Thread.run(Thread.java:619) > > > > I'm not sure if the NPE that happened before is the reason > for this, as I don't > > see the relationship between both issues (apart from the > correlation of events/time). > > > > Hopefully someone closer to the code can have a look at > this and tell what's the > > problem here? Then I'd like to submit a bug report / help > fixing(?)... > > > > More output of the logs you find at the end of this email. > > > > Thanx a lot in advance, > > cheers, > > Martin > > > > > > > > [INFO ] 2007-07-09 11:56:35,513 TP-Processor1 > org.comp.proj.search.presentation.pages.Search.onActivate: > > Got invoked args hose, cat:Herrenbekleidung, n, 100 > > > > [INFO ] 2007-07-09 11:56:35,518 TP-Processor1 > org.comp.proj.search.presentation.pages.Search.setupRender: > > Starting search... > > > > [DEBUG] 2007-07-09 11:56:35,520 TP-Processor1 > org.comp.proj.search.business.SearchServiceSolrImpl.search: > > Starting query: > > > q=hose;score+desc&fq=cat:Herrenbekleidung&start=0&rows=100&fac et=true&facet.field=cat&facet.field=brand&facet.field=type&facet> .field=color&facet > > > .field=price&facet.limit=5&facet.zeros=false&facet.missing=fal se&version=2.2 > > > > [DEBUG] 2007-07-09 11:56:35,610 TP-Processor1 > org.comp.proj.search.business.SearchServiceSolrImpl.logResponse: > > Got response: numFound: 207, queryTime: 43 > > > > [INFO ] 2007-07-09 11:56:35,803 TP-Processor1 > org.comp.proj.search.presentation.services.AppModule.TimingFil > ter.service: > > Request time: 292 ms > > > > [INFO ] 2007-07-09 11:57:05,982 TP-Processor5 > org.comp.proj.search.presentation.pages.Search.onActivate: > > Got invoked args hose, n, 100 > > > > [INFO ] 2007-07-09 11:57:05,988 TP-Processor5 > org.comp.proj.search.presentation.pages.Search.setupRender: > > Starting search... > > > > [DEBUG] 2007-07-09 11:57:05,989 TP-Processor5 > org.comp.proj.search.business.SearchServiceSolrImpl.search: > > Starting query: > > > q=hose;score+desc&start=0&rows=100&facet=true&facet.field=cat& facet.field=brand&facet.field=type&facet.field=color&facet.field> =price&facet.limit > > =5&facet.zeros=false&facet.missing=false&version=2.2 > > > > [DEBUG] 2007-07-09 11:57:06,078 TP-Processor5 > org.comp.proj.search.business.SearchServiceSolrImpl.logResponse: > > Got response: numFound: 2001, queryTime: 44 > > > > [INFO ] 2007-07-09 11:57:06,276 TP-Processor5 > org.comp.proj.search.presentation.services.AppModule.TimingFil > ter.service: > > Request time: 296 ms > > > > [INFO ] 2007-07-09 12:07:22,379 TP-Processor5 > org.comp.proj.search.presentation.pages.Search.onActivate: > > Got invoked args > > > > [INFO ] 2007-07-09 12:07:22,382 TP-Processor5 > org.comp.proj.search.presentation.services.AppModule.TimingFil > ter.service: > > Request time: 605041 ms > > ^^^^^^^^^^^^^^^^^^^^^^^ --> 10 minutes?! > > > > [ERROR] 2007-07-09 12:07:22,382 TP-Processor5 > org.apache.tapestry.internal.services.InternalModule.RequestEx > ceptionHandler.handleRequestExceptio > > n: > > Processing of request failed with uncaught exception: > org.apache.tapestry.ioc.internal.util.TapestryException > > > > org.apache.tapestry.ioc.internal.util.TapestryException > > at > org.apache.tapestry.corelib.components.Form.onAction(Form.java:375) > > at > org.apache.tapestry.corelib.components.Form.handleComponentEve > nt(Form.java) > > at > org.apache.tapestry.internal.structure.ComponentPageElementImp > l.handleEvent(ComponentPageElementImpl.java:893) > > at > org.apache.tapestry.internal.structure.ComponentPageElementImp > l.triggerEvent(ComponentPageElementImpl.java:998) > > at > org.apache.tapestry.internal.services.ComponentActionRequestHa > ndlerImpl.handle(ComponentActionRequestHandlerImpl.java:81) > > at > org.apache.tapestry.internal.services.InternalModule$11.handle (InternalModule.java:541) > > at > $ComponentActionRequestHandler_1139c29ae5a.handle($ComponentAc tionRequestHandler_1139c29ae5a.java) > > at > $ComponentActionRequestHandler_1139c29ae55.handle($ComponentAc tionRequestHandler_1139c29ae55.java) > > at > org.apache.tapestry.internal.services.ComponentActionDispatche > r.dispatch(ComponentActionDispatcher.java:116) > > at > $Dispatcher_1139c29ae57.dispatch($Dispatcher_1139c29ae57.java) > > at > $Dispatcher_1139c29ae49.dispatch($Dispatcher_1139c29ae49.java) > > at > org.apache.tapestry.services.TapestryModule$12.service(Tapestr yModule.java:1066) > > at > org.comp.proj.search.presentation.services.AppModule$2.service (AppModule.java:117) > > at > $RequestFilter_1139c29ae48.service($RequestFilter_1139c29ae48.java) > > at > $RequestHandler_1139c29ae4a.service($RequestHandler_1139c29ae4a.java) > > at > org.comp.proj.search.presentation.services.AppModule$1.service (AppModule.java:76) > > at > $RequestFilter_1139c29ae47.service($RequestFilter_1139c29ae47.java) > > at > $RequestHandler_1139c29ae4a.service($RequestHandler_1139c29ae4a.java) > > at > org.apache.tapestry.internal.services.LocalizationFilter.servi > ce(LocalizationFilter.java:43) > > at > $RequestHandler_1139c29ae4a.service($RequestHandler_1139c29ae4a.java) > > at > org.apache.tapestry.services.TapestryModule$2.service(Tapestry Module.java:657) > > at > $RequestHandler_1139c29ae4a.service($RequestHandler_1139c29ae4a.java) > > at > org.apache.tapestry.internal.services.StaticFilesFilter.servic > e(StaticFilesFilter.java:63) > > at > $RequestHandler_1139c29ae4a.service($RequestHandler_1139c29ae4a.java) > > at > org.apache.tapestry.internal.services.CheckForUpdatesFilter$2. invoke(CheckForUpdatesFilter.java:97) > > at > org.apache.tapestry.internal.services.CheckForUpdatesFilter$2. invoke(CheckForUpdatesFilter.java:88) > > at > org.apache.tapestry.ioc.internal.util.ConcurrentBarrier.withRe > ad(ConcurrentBarrier.java:77) > > at > org.apache.tapestry.internal.services.CheckForUpdatesFilter.se > rvice(CheckForUpdatesFilter.java:110) > > at > $RequestHandler_1139c29ae4a.service($RequestHandler_1139c29ae4a.java) > > at > $RequestHandler_1139c29ae41.service($RequestHandler_1139c29ae41.java) > > at > org.apache.tapestry.services.TapestryModule$11.service(Tapestr yModule.java:1044) > > at > $HttpServletRequestHandler_1139c29ae40.service($HttpServletReq uestHandler_1139c29ae40.java) > > at > org.apache.tapestry.TapestryFilter.doFilter(TapestryFilter.java:135) > > at > org.apache.catalina.core.ApplicationFilterChain.internalDoFilt > er(ApplicationFilterChain.java:235) > > at > org.apache.catalina.core.ApplicationFilterChain.doFilter(Appli > cationFilterChain.java:206) > > at > org.apache.catalina.core.StandardWrapperValve.invoke(StandardW > rapperValve.java:230) > > at > org.apache.catalina.core.StandardContextValve.invoke(StandardC > ontextValve.java:175) > > at > org.apache.catalina.core.StandardHostValve.invoke(StandardHost > Valve.java:128) > > at > org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReport > Valve.java:104) > > at > org.apache.catalina.core.StandardEngineValve.invoke(StandardEn > gineValve.java:109) > > at > org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdap > ter.java:261) > > at > org.apache.jk.server.JkCoyoteHandler.invoke(JkCoyoteHandler.java:190) > > at > org.apache.jk.common.HandlerRequest.invoke(HandlerRequest.java:283) > > at > org.apache.jk.common.ChannelSocket.invoke(ChannelSocket.java:767) > > at > org.apache.jk.common.ChannelSocket.processConnection(ChannelSo > cket.java:697) > > at > org.apache.jk.common.ChannelSocket$SocketConnection.runIt(Chan nelSocket.java:889) > > at > org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run( ThreadPool.java:686) > > at java.lang.Thread.run(Thread.java:619) > > Caused by: java.lang.NullPointerException > > at > org.apache.tapestry.internal.util.Base64InputStream.decode(Bas > e64InputStream.java:34) > > at > org.apache.tapestry.internal.util.Base64InputStream.<init>(Bas > e64InputStream.java:29) > > at > org.apache.tapestry.internal.util.Base64ObjectInputStream.<ini > t>(Base64ObjectInputStream.java:30) > > at > org.apache.tapestry.corelib.components.Form.onAction(Form.java:355) > > ... 47 more > > Jul 9, 2007 2:27:57 PM > org.apache.tomcat.util.threads.ThreadPool logFull > > SEVERE: All threads (200) are currently busy, waiting. > Increase maxThreads (200) or check the servlet status > > 2007-07-09 17:13:35 > > Full thread dump Java HotSpot(TM) 64-Bit Server VM > (1.6.0_01-b06 mixed mode): > > > > "http-9090-1" daemon prio=10 tid=0x00002aaaf7e1fc00 > nid=0x3f05 waiting on condition > [0x000000004459e000..0x000000004459fbc0] > > java.lang.Thread.State: WAITING (parking) > > at sun.misc.Unsafe.park(Native Method) > > - parking to wait for <0x00002aaab8228360> (a > java.util.concurrent.locks.ReentrantReadWriteLock$NonfairSync) > > at > java.util.concurrent.locks.LockSupport.park(LockSupport.java:158) > > at > java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndC > heckInterrupt(AbstractQueuedSynchronizer.java:712) > > at > java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquir > eShared(AbstractQueuedSynchronizer.java:842) > > at > java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireS > hared(AbstractQueuedSynchronizer.java:1162) > > at > java.util.concurrent.locks.ReentrantReadWriteLock$ReadLock.loc k(ReentrantReadWriteLock.java:594) > > at > org.apache.tapestry.ioc.internal.util.ConcurrentBarrier.withRe > ad(ConcurrentBarrier.java:70) > > at > org.apache.tapestry.internal.services.CheckForUpdatesFilter.se > rvice(CheckForUpdatesFilter.java:110) > > at > $RequestHandler_1139c29ae4a.service($RequestHandler_1139c29ae4a.java) > > at > $RequestHandler_1139c29ae41.service($RequestHandler_1139c29ae41.java) > > at > org.apache.tapestry.services.TapestryModule$11.service(Tapestr yModule.java:1044) > > at > $HttpServletRequestHandler_1139c29ae40.service($HttpServletReq uestHandler_1139c29ae40.java) > > at > org.apache.tapestry.TapestryFilter.doFilter(TapestryFilter.java:135) > > at > org.apache.catalina.core.ApplicationFilterChain.internalDoFilt > er(ApplicationFilterChain.java:235) > > at > org.apache.catalina.core.ApplicationFilterChain.doFilter(Appli > cationFilterChain.java:206) > > at > org.apache.catalina.core.StandardWrapperValve.invoke(StandardW > rapperValve.java:230) > > at > org.apache.catalina.core.StandardContextValve.invoke(StandardC > ontextValve.java:175) > > at > org.apache.catalina.core.StandardHostValve.invoke(StandardHost > Valve.java:128) > > at > org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReport > Valve.java:104) > > at > org.apache.catalina.core.StandardEngineValve.invoke(StandardEn > gineValve.java:109) > > at > org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdap > ter.java:261) > > at > org.apache.coyote.http11.Http11Processor.process(Http11Process > or.java:844) > > at > org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandle r.process(Http11Protocol.java:581) > > at > org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint. java:447) > > at java.lang.Thread.run(Thread.java:619) > > > > "TP-Processor200" daemon prio=10 tid=0x00002aaaf8896400 > nid=0x3ce9 waiting on condition > [0x000000004455d000..0x000000004455ec40] > > java.lang.Thread.State: WAITING (parking) > > at sun.misc.Unsafe.park(Native Method) > > - parking to wait for <0x00002aaab8228360> (a > java.util.concurrent.locks.ReentrantReadWriteLock$NonfairSync) > > at > java.util.concurrent.locks.LockSupport.park(LockSupport.java:158) > > at > java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndC > heckInterrupt(AbstractQueuedSynchronizer.java:712) > > at > java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquir > eShared(AbstractQueuedSynchronizer.java:842) > > at > java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireS > hared(AbstractQueuedSynchronizer.java:1162) > > at > java.util.concurrent.locks.ReentrantReadWriteLock$ReadLock.loc k(ReentrantReadWriteLock.java:594) > > at > org.apache.tapestry.ioc.internal.util.ConcurrentBarrier.withRe > ad(ConcurrentBarrier.java:70) > > at > org.apache.tapestry.internal.services.CheckForUpdatesFilter.se > rvice(CheckForUpdatesFilter.java:110) > > at > $RequestHandler_1139c29ae4a.service($RequestHandler_1139c29ae4a.java) > > at > $RequestHandler_1139c29ae41.service($RequestHandler_1139c29ae41.java) > > at > org.apache.tapestry.services.TapestryModule$11.service(Tapestr yModule.java:1044) > > at > $HttpServletRequestHandler_1139c29ae40.service($HttpServletReq uestHandler_1139c29ae40.java) > > at > org.apache.tapestry.TapestryFilter.doFilter(TapestryFilter.java:135) > > at > org.apache.catalina.core.ApplicationFilterChain.internalDoFilt > er(ApplicationFilterChain.java:235) > > at > org.apache.catalina.core.ApplicationFilterChain.doFilter(Appli > cationFilterChain.java:206) > > at > org.apache.catalina.core.StandardWrapperValve.invoke(StandardW > rapperValve.java:230) > > at > org.apache.catalina.core.StandardContextValve.invoke(StandardC > ontextValve.java:175) > > at > org.apache.catalina.core.StandardHostValve.invoke(StandardHost > Valve.java:128) > > at > org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReport > Valve.java:104) > > at > org.apache.catalina.core.StandardEngineValve.invoke(StandardEn > gineValve.java:109) > > at > org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdap > ter.java:261) > > at > org.apache.jk.server.JkCoyoteHandler.invoke(JkCoyoteHandler.java:190) > > at > org.apache.jk.common.HandlerRequest.invoke(HandlerRequest.java:283) > > at > org.apache.jk.common.ChannelSocket.invoke(ChannelSocket.java:767) > > at > org.apache.jk.common.ChannelSocket.processConnection(ChannelSo > cket.java:697) > > at > org.apache.jk.common.ChannelSocket$SocketConnection.runIt(Chan nelSocket.java:889) > > at > org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run( ThreadPool.java:686) > > at java.lang.Thread.run(Thread.java:619) > > > > ... and 199 more of them > > > > > -- > Martin Grotzke > http://www.javakaffee.de/blog/ > --------------------------------------------------------------------- To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
