It's possible that the server never received the TCP FIN packet and
therefor did not know that it should close the IoSession. There are
situations where a host has no way of knowing that the TCP socket has
closed unless it tries to write data to the socket and that write fails.
This is not a limitation of MINA. This is just the way socket I/O works.
If this is the cause of your problem, you have a couple of options. You
could use some type of watch dog mechanisms that sends data at a regular
interval. If the connection is no longer valid, the write will fail and
the socket will get closed thereby triggering a sessionClosed event. Or
you could use the MINA read idle notification. This way after a certain
amount of time has passed and your server hasn't received any data for a
particular IoSession, you can manually close the IoSession.
If your problem still persists after implementing one of these
solutions, please let us know.
-Mike
jhcha wrote:
> Dear Mina developers.
> I'm trying to use mina 1.1.5 to develop the very big stress server.
> I test my server with jmeter like some echo test now.
>
> test environment
>
> * jmeter :
> echo message size : 1024 bytes
> jmeter threads : 100
> loop count : 1000
> sampler : TCP Sampler
> reuse connection : false
>
> * server : HP-UX 2 cores
> java HotSpot(TM) Server VM (build 1.5.0.07 jinteg:03.20.07-11:05
> IA64, mixed mode)
> my server(mina)
>
> * client : pc (dual core).
> jmeter
>
> when I tested the stress, I found sessionCreated called counts are a little
> bigger than sessionClosed called counts.
> so, I tried to find out why that mismatch occured during several days.
> and I found the followings :
>
>
> org.apache.mina.common.support.IoServiceListenerSupport
> {
>
> public void fireSessionCreated(IoSession session) {
> SocketAddress serviceAddress = session.getServiceAddress();
>
> // Get the session set.
> Set<IoSession> s = new IdentityHashSet<IoSession>();
> Set<IoSession> sessions =
> managedSessions.putIfAbsent(serviceAddress,
> Collections
> .synchronizedSet(s));
> boolean firstSession;
>
> if (null == sessions) {
> sessions = s;
> firstSession = true;
> } else {
> firstSession = false;
> }
>
> // If already registered, ignore.
> if (!sessions.add(session)) {
> return;
> }
>
> // If the first connector session, fire a virtual service
> activation
> // event.
> if (session.getService() instanceof IoConnector &&
> firstSession) {
> fireServiceActivated(session.getService(),
> session.getServiceAddress(),
> session
> .getHandler(),
> session.getServiceConfig());
> }
>
> // Fire session events.
> session.getFilterChain().fireSessionCreated(session); <--
> called
> correctly
> session.getFilterChain().fireSessionOpened(session); <--
> called correctly
>
> // Fire listener events.
> for (IoServiceListener listener : listeners) {
> listener.sessionCreated(session); <-- called correctly
> }
> }
>
>
> public void fireSessionDestroyed(IoSession session) {
> boolean lastSession = false;
> SocketAddress serviceAddress = session.getServiceAddress();
>
> // Get the session set.
> Set<IoSession> sessions = managedSessions.get(serviceAddress);
> // Ignore if unknown.
> if (sessions == null) {
> // logger.error("fireSessionDestroyed() : sessions ==
> null < " +
> session);
> return; <-- occured sometimes !!!!!!!!!!!!!!!!!!
> }
>
> sessions.remove(session);
>
> // Try to remove the remaining empty session set after removal.
> if (sessions.isEmpty()) {
> lastSession = managedSessions.remove(serviceAddress,
> sessions);
> }
>
> // Fire session events.
> session.getFilterChain().fireSessionClosed(session); <--
> skipped sometimes
>
> // Fire listener events.
> try {
> for (IoServiceListener listener : listeners) {
> listener.sessionDestroyed(session); <-- skipped
> sometimes
> }
> } finally {
> // Fire a virtual service deactivation event for the
> last session of
> // the connector.
> // TODO double-check that this is *STILL* the last
> session. May not
> // be the case
> if (session.getService() instanceof IoConnector &&
> lastSession) {
> fireServiceDeactivated(session.getService(),
> session.getServiceAddress(), session
> .getHandler(),
> session.getServiceConfig()); <-- skipped sometimes
> }
> }
> }
>
> the above fireSessionDestroyed method called as the same number as
> fireSessionCreated method.
>
> but the mysterious event is that the sessions is not gotten from
> managedSessions in fireSessionDestroyed,
> i.e. SocketAddress key is not found in managedSessions.
> then the remaining logic was skipped.
> so fireSessionClosed, sessionDestroyed ... are not called sometimes.
>
> [IoServiceListenerSupport] - fireSessionDestroyed() : sessions == null <
> (SOCKET, R: /172.26.73.107:15059, L: /10.88.50.205:30000, S:
> 0.0.0.0/0.0.0.0:30000)
> [IoServiceListenerSupport] - fireSessionDestroyed() : sessions == null <
> (SOCKET, R: /172.26.73.107:15961, L: /10.88.50.205:30000, S:
> 0.0.0.0/0.0.0.0:30000)
> [IoServiceListenerSupport] - fireSessionDestroyed() : sessions == null <
> (SOCKET, R: /172.26.73.107:23883, L: /10.88.50.205:30000, S:
> 0.0.0.0/0.0.0.0:30000)
> [IoServiceListenerSupport] - fireSessionDestroyed() : sessions == null <
> (SOCKET, R: /172.26.73.107:24036, L: /10.88.50.205:30000, S:
> 0.0.0.0/0.0.0.0:30000)
> [IoServiceListenerSupport] - fireSessionDestroyed() : sessions == null <
> (SOCKET, R: /172.26.73.107:31341, L: /10.88.50.205:30000, S:
> 0.0.0.0/0.0.0.0:30000)
> [IoServiceListenerSupport] - fireSessionDestroyed() : sessions == null <
> (SOCKET, R: /172.26.73.107:31343, L: /10.88.50.205:30000, S:
> 0.0.0.0/0.0.0.0:30000)
> [IoServiceListenerSupport] - fireSessionDestroyed() : sessions == null <
> (SOCKET, R: /172.26.73.107:31345, L: /10.88.50.205:30000, S:
> 0.0.0.0/0.0.0.0:30000)
> [IoServiceListenerSupport] - fireSessionDestroyed() : sessions == null <
> (SOCKET, R: /172.26.73.107:45839, L: /10.88.50.205:30000, S:
> 0.0.0.0/0.0.0.0:30000)
> [IoServiceListenerSupport] - fireSessionDestroyed() : sessions == null <
> (SOCKET, R: /172.26.73.107:5317, L: /10.88.50.205:30000, S:
> 0.0.0.0/0.0.0.0:30000)
> [IoServiceListenerSupport] - fireSessionDestroyed() : sessions == null <
> (SOCKET, R: /172.26.73.107:8471, L: /10.88.50.205:30000, S:
> 0.0.0.0/0.0.0.0:30000)
>
> I think the reason that the 100 client threads tries to proceed "conect,
> send, receive, close",
> different sessions has the same client SocketAddress at times in server,
> and the origin of that problem is client's stress environment.
>
> It is possible the following senario :
>
> I/O processor thread I/O processor thread client thread client
> thread
> one two one two
> --------------------------------------------------------------------------------
> 1) connect(the same
> client SocketAddress)
> 2)fireSessionCreated
> 3)putIfAbsent
> 4)disconnect
>
> 5)connect(the same client SocketAddress)
> 6)fireSessionCreated
> 7)putIfAbsent
> 8)fireSessionDestroyed
> 9)remove
>
> 10)disconnect
> 11)fireSessionDestroyed
> 12)get fail
>
>
>
> I changed the mina source IoServiceListenerSupport.java
>
> org.apache.mina.common.support.IoServiceListenerSupport
> ...
>
> public void fireSessionDestroyed(IoSession session) {
> boolean lastSession = false;
> SocketAddress serviceAddress = session.getServiceAddress();
>
> // Get the session set.
> Set<IoSession> sessions = managedSessions.get(serviceAddress);
> // Ignore if unknown.
> if (sessions == null) {
> // return; <-- delete
> } else { // <-- add
>
> sessions.remove(session);
>
> // Try to remove the remaining empty session set after
> removal.
> if (sessions.isEmpty()) {
> lastSession =
> managedSessions.remove(serviceAddress, sessions);
> }
> } // <-- add
> ....
>
> then the Create, Destroy call mismatch was cleared.
> but I don't know this is correct solution.
>
> Would you tell me this is correct or not?
>
> I think this situation is very rare but...
> I hope the mina is the best network framework
> and this situation shoud be also considered.
>
>
> Thank you
>
> J. H. Cha
