On Tue, 29 Jul 2008 16:21:44 +0200 "Tom Wieczorek" <[EMAIL PROTECTED]> wrote:
> Hi all, > > I encountered an interesting problem today, in which MINA didn't > accept any incoming connections any longer, because we had too many > open files on the server machine. > > The application in question uses MINA version 2.0.0-M1 with 3 > different IoAcceptor services sharing the same Executor and > IoProcessor thread pool. The first two services are designed to have > typically 5-10 permanent IoSessions connected per service. These two > services don't have an ExecutionFilter in their filter chain and are > driven by the IoProcessor theads. The third service is a HTTP > Request/Response service based on AsyncWeb, using an ExecutorFilter > with a dedicated HTTP thread pool. > > After running for several weeks without any problems, the app stooped > responding to new connections and the log file was spammed with the > Exception Monitor log entries: > > WARN [NioSocketAcceptor-3] > org.apache.mina.common.DefaultExceptionMonitor: java.io.IOException: > Too many open files at > sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method) at > sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:145) > at > org.apache.mina.transport.socket.nio.NioSocketAcceptor.accept(NioSocketAcceptor.java:153) > at > org.apache.mina.transport.socket.nio.NioSocketAcceptor.accept(NioSocketAcceptor.java:47) > at > org.apache.mina.common.AbstractPollingIoAcceptor$Worker.processHandles(AbstractPollingIoAcceptor.java:305) > at > org.apache.mina.common.AbstractPollingIoAcceptor$Worker.run(AbstractPollingIoAcceptor.java:243) > at > org.apache.mina.util.NamePreservingRunnable.run(NamePreservingRunnable.java:51) > at > java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:650) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:675) > at java.lang.Thread.run(Thread.java:595) > > After further investigation using "lsof -p", I really found approx. > 1000 sockets in CLOSE_WAIT state that didn't get released. The > sockets were mainly bound to the HTTP service. Note that the already > open sessions on the ohter two services worked without any problems > (messages were processed normally). Also interesting: The app has a > Connection Limit by IP address enforced by an IoFilter. The filter > prints the managed session count for the IoService in its DEBUG log > messages when sessionClosed() is invoked. The last DEBUG entry of > this kind close before the max file limit was reached said that the > managed sessions count of the HTTP service was 0. I couldn't detect a > missing coseSession() event in the Logs either. > > What circumstances could lead to the effect that MINA doesn't > close/release sockets that have been colsed remotely? The processors > seemed to work (the other sessions were still active), the acceptors > spammed the exceptions. Any ideas? > > Some enviromental information: > Linux 2.6.16.21-0.25-smp #1 SMP Tue Sep 19 07:26:15 UTC 2006 x86_64 > x86_64 x86_64 GNU/Linux > > java version "1.5.0_11" > Java(TM) 2 Runtime Environment, Standard Edition (build 1.5.0_11-b03) > Java HotSpot(TM) 64-Bit Server VM (build 1.5.0_11-b03, mixed mode) > > > Thanks for your help. > > Cheers, > Tom. > hi, Sound very weird. do you tried to make a thread dump, for see if all the IoP are running smoothly ? Any other exception poped up ? Julien
signature.asc
Description: PGP signature
