Hi I'm wondering if any of the developers or someone who has seen this happen might want to comment.
Yesterday I saw SOAPMonitor bring a pretty high volumne site to a halt. Here's what I think happened. SOAPMonitor was used to check something on a number of the production machines. I believe it was left running. (not a good idea, I know) About 4 hours later our application servers stopped responding. A thread dump showed that all the threads were waiting on the SOAPMonitor. I'm guessing that the SOAPMonitor applet had been left running and at some point had some type of a problem like running out of memory or the vm just got hosed or something. From looking at the source it seems that the SOAPMonitorHandler will keep pumping messages via the SOAPMonitorService to the ConnectionThreads and try to send them out the wire. It looks like if something gets hung up with the socket from the applet that it might cause what we saw which is that threads get hung waiting for publishMessage to return. Anyone seen this or have any ideas how SOAPMonitor could be improved to cope with this situation? (perhaps a tuneable socket timeout? or some async decoupling of the Handler from the sending of the messages to the applet?) Thanks, Alex.
