Thanks Paul. I checked and did not find any "Client disconnected" entries in the event log so this may not be the cause of what I am seeing. I will bump the log level to ALL (which I think includes BLATHER) when I restart things tonight.
We use a remote ZEO but at the moment Zope and ZEO share the same machine. On Sun, 27 Nov 2005, Paul Winkler wrote: > On Sun, Nov 27, 2005 at 05:13:21PM -0800, Dennis Allison wrote: > > Examination of the raw trace log shows that Zope is continuing to accept > > requests, but nothing getting done. The raw log date-stamps four internal > > states for each transaction. The states are Begin (B), Input (I), > > action (A), and End (E). Inputs are gathered between B and I, outputs is > > made between A and E. The raw log shows B and I transactions, but > > apparently no processing is completing. I suspect that nothing is getting > > scheduled. > > I've seen the same symptoms a number of times recently with zope 2.7.x. > In our case, it seems to be related to ZEO. Zope seems to have lost > its connection to ZEO but doesn't realize it somehow. My theory is > that the symptom starts when all worker threads are waiting for objects > that aren't in the ZEO client cache, so they're all waiting on ZEO > requests. Meanwhile, requests keep piling up in the queue. > These are all the B and I lines you noticed. > > Eventually (10 minutes or so) one of these ZEO requests times out, and > Zope then realizes it's lost its ZEO connection and successfully reconnects. > This shows up in the Zope event log as a series of ZEO "Client > disconnected" errors, and immediately thereafter we see a successful > reconnect in the ZEO server's log. > > I have no idea what is causing the loss of connection in the first > place, or why it takes Zope so long to realize there's a problem. > Nothing in the logs so far gives me any clue. I've bumped up our log > levels to BLATHER and hopefully next time this happens I'll get some > more to work with and see if there's anythign I can do about it. > > It may be network issues between our Zope and Zeo boxes, which > are on different subnets with a firewall between them, over which > I have no control. > > -- _______________________________________________ Zope maillist - [email protected] http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
