Re: MINA: ExecutorFilter on NioSocketConnector

[Weact]

  
  

 Is this server ready for deployment for a chat application supporting 1mm 
users or is it more in the prototype phase?
  

  
We will be using group chat, roles, moderators and adding some unique chat 
threading models for distributed groups.
  

  
Thanks.   
  

  
  

  
  
>   
> On Apr 9, 2020 at 11:21 AM,  mailto:elecha...@gmail.com)> 
>  wrote:
>   
>   
>   
>  On 09/04/2020 16:01, Nitin Phuria wrote:  >  Dear Jonathan,  >   >   >   >  
> Anytime we write message to System-B session with session.write method the 
> messageSent event has to be generated with it actually gets written on TCP/IP 
> The messageSent event is only generated when the message has been fully 
> written into the socket. A message can be written piece by piece, depending 
> on the socket capacity. Also messages can be queued until the previous 
> messages have been fully written. What could happen is that if you have a 
> slow reader on the other side, then the socket will not be able to accept any 
> data, because it gets full.  >   >   >   >  
> org.apache.mina.core.session.IoSession.write(Object)  >   >   >   >  We see 
> that there is time difference of 10-14 seconds between timestamp of 
> session.write for particular message and messageSent event getting generated 
> for the same message. So you have a problem in the middle. Check that the 
> reader (ie the other server) is processing messages fast enough. Or maybe you 
> have network issues, as suggests Jonathan.  >   >   >   >  In non-peak load 
> this time difference is in 10-15 milliseconds and in peak load (60-70 
> messages per second) it goes till 10-14 seconds. 60/70 msg/s is not 
> exceptionally high, except if you are managing big messages, or do a lot of 
> processing on each message. 
> - To 
> unsubscribe, e-mail: users-unsubscr...@mina.apache.org For additional 
> commands, e-mail: users-h...@mina.apache.org  
>
>   
  
  
 

Re: MINA: ExecutorFilter on NioSocketConnector

[Jonathan Valliere]

Huh?

On Thu, Apr 9, 2020 at 1:09 PM Weact  wrote:

>
>
>
>  Is this server ready for deployment for a chat application supporting 1mm
> users or is it more in the prototype phase?
>
>
>
> We will be using group chat, roles, moderators and adding some unique chat
> threading models for distributed groups.
>
>
>
> Thanks.
>
>
>
>
>
>
>
> >
> > On Apr 9, 2020 at 11:21 AM,   elecha...@gmail.com)>  wrote:
> >
> >
> >
> >  On 09/04/2020 16:01, Nitin Phuria wrote:  >  Dear Jonathan,  >   >   >
>  >  Anytime we write message to System-B session with session.write method
> the messageSent event has to be generated with it actually gets written on
> TCP/IP The messageSent event is only generated when the message has been
> fully written into the socket. A message can be written piece by piece,
> depending on the socket capacity. Also messages can be queued until the
> previous messages have been fully written. What could happen is that if you
> have a slow reader on the other side, then the socket will not be able to
> accept any data, because it gets full.  >   >   >   >
> org.apache.mina.core.session.IoSession.write(Object)  >   >   >   >  We see
> that there is time difference of 10-14 seconds between timestamp of
> session.write for particular message and messageSent event getting
> generated for the same message. So you have a problem in the middle. Check
> that the reader (ie the other server) is processing messages fast enough.
> Or maybe you have network issues, as suggests Jonathan.  >   >   >   >  In
> non-peak load this time difference is in 10-15 milliseconds and in peak
> load (60-70 messages per second) it goes till 10-14 seconds. 60/70 msg/s is
> not exceptionally high, except if you are managing big messages, or do a
> lot of processing on each message.
> - To
> unsubscribe, e-mail: users-unsubscr...@mina.apache.org For additional
> commands, e-mail: users-h...@mina.apache.org
> >
> >
>
>
>

-- 

CONFIDENTIALITY NOTICE: The contents of this email message and any
attachments are intended solely for the addressee(s) and may contain
confidential and/or privileged information and may be legally protected
from disclosure.

MINA: ExecutorFilter on NioSocketConnector

[Nitin Phuria]

Dear All,

 

We have Server developed using MINA 2.0.16 where it connects
to two other systems say System-A and System-B.

 

The connectivity with both these system is persistence which means our
server connects and create only single session with each of these systems
and on the same session multiple message exchanges (Request/Response)
happens.

 

The current flow of message is as below:

Once we connect with System-B and have persistence connection (single
session), System-B will send us message which we have to process in our
server and then send processed message to System-A then get response from
System-A and then pass it back to System-B

 

We have provision that if load increases we can have multiple instances of
System-A and our server can establish persistence connection (single
session) with each of these System-A instances and manage the load.

 

We are facing problem with load coming from System-B to our server as it is
coming on single session and we could see that the read/write on the session
are slowed down and lot of queue build-up is happening for read/write.

 

Below is the code for establishing the Connectivity with System-B where we
have not added the ExecutorFilter

 

SocketAddress socketAddress = new InetSocketAddress(this.hostName,
this.port);

NioSocketConnector connector = new NioSocketConnector();

connector.setConnectTimeoutMillis(30 * 1000L);

connector.getSessionConfig().setTcpNoDelay(true);

connector.getSessionConfig().setIdleTime(IdleStatus.BOTH_IDLE,this.echoTime)
;

this.connector.setHandler(systemBHandler); // systemBHandler is our
IoHandler implementationfor System-B

IoFilter codecFilter = new
ProtocolCodecFilter((ProtocolCodecFactory)systemBProcFactory);//
systemBProcFactory is our Codec Implementation

DefaultIoFilterChainBuilder chain = this.connector.getFilterChain();

chain.clear();

chain.addLast("codec", codecFilter);

 

https://cwiki.apache.org/confluence/display/MINA/Configuring+Thread+Model   

>From this link we could understand as below 

 

If We didn't add any ExecutorFilter events are forwarded to an IoHandler via
method invocations. This means your business logic in your IoHandler
implementation will run in the I/O processor thread. We call this thread
model a 'single thread model'. The single thread model is known to be
adequate for low-latency network applications with CPU-bound business logic
(e.g. game servers).

 

Do we need to add the ExecutorFilter for above mentioned NioSocketConnector
to System-B so that I/O processor thread will only do the I/O job and all
other business logic will be given to worker threads in ExecutorFilter.

 

Is there any way to increase the number of threads for my I/O to do the
read/write as we have restriction to create only one session with System-B.

 

If we add ExecutorFilter  with IoEventType for read/write then will that
make the read/write faster with System-B on same single session.

 

Or we add ExecutorFilter  with default IoEventType (All Events) will that
make the read/write faster with System-B on same single session.

 

Pls help us understand the working on ExecutorFilter in above scenario where
NioSocketConnector has only single session to do all the read/write with
huge load of almost 60-70 messages per second with average message size of
3-4KiloBytes.

 

Currently we have server having 16 Core with 64 GB RAM.

 

Thanks And Regards,

Nitin Phuria

 

Confidentiality Disclaimer: "The information contained in this electronic
message (email) and any attachments to this email are intended for the
exclusive use of the addressee(s) and access to this email by anyone else is
unauthorized. The email may contain proprietary, confidential or privileged
information or information relating to Integra Group. If you are not the
intended recipient, please notify the sender by telephone, fax, or return
email and delete this communication and any attachments thereto, immediately
from your computer. Any dissemination, distribution, or copying of this
communication and the attachments thereto (in whole or part), in any manner,
is strictly prohibited and actionable at law. The recipient acknowledges
that emails are susceptible to alteration and their integrity cannot be
guaranteed and that Company does not guarantee that any e-mail is virus-free
and accept no liability for any damage caused by any virus transmitted by
this email."


-- 
*Confidentiality Disclaimer**: "The information contained in this 
electronic message
(email) and any attachments to this email are intended 
for the exclusive use of
the addressee(s) and access to this email by 
anyone else is unauthorized. The
email may contain proprietary, 
confidential or privileged information or
information relating to Integra 
Group. If you are not the intended recipient,
please notify the sender by 
telephone, fax, or return email and delete this
communication and any 
attachments thereto, immediately from your computer. Any

Re: MINA: ExecutorFilter on NioSocketConnector

[Jonathan Valliere]

Mina is a queue and flush framework. All writes are queued up and scheduled
later using the poll/kqueue system.

The ExecutorFilter uses threading to process incoming messages using a pool
of threads instead of the reactor/processor thread.  This frees the
processor thread to do IO.

Have you profiled the application to figure out which code is consuming the
CPU?

On Thu, Apr 9, 2020 at 3:39 AM Nitin Phuria 
wrote:

> Dear All,
>
>
>
> We have Server developed using MINA 2.0.16 where it
> connects
> to two other systems say System-A and System-B.
>
>
>
> The connectivity with both these system is persistence which means our
> server connects and create only single session with each of these systems
> and on the same session multiple message exchanges (Request/Response)
> happens.
>
>
>
> The current flow of message is as below:
>
> Once we connect with System-B and have persistence connection (single
> session), System-B will send us message which we have to process in our
> server and then send processed message to System-A then get response from
> System-A and then pass it back to System-B
>
>
>
> We have provision that if load increases we can have multiple instances of
> System-A and our server can establish persistence connection (single
> session) with each of these System-A instances and manage the load.
>
>
>
> We are facing problem with load coming from System-B to our server as it is
> coming on single session and we could see that the read/write on the
> session
> are slowed down and lot of queue build-up is happening for read/write.
>
>
>
> Below is the code for establishing the Connectivity with System-B where we
> have not added the ExecutorFilter
>
>
>
> SocketAddress socketAddress = new InetSocketAddress(this.hostName,
> this.port);
>
> NioSocketConnector connector = new NioSocketConnector();
>
> connector.setConnectTimeoutMillis(30 * 1000L);
>
> connector.getSessionConfig().setTcpNoDelay(true);
>
>
> connector.getSessionConfig().setIdleTime(IdleStatus.BOTH_IDLE,this.echoTime)
> ;
>
> this.connector.setHandler(systemBHandler); // systemBHandler is our
> IoHandler implementationfor System-B
>
> IoFilter codecFilter = new
> ProtocolCodecFilter((ProtocolCodecFactory)systemBProcFactory);//
> systemBProcFactory is our Codec Implementation
>
> DefaultIoFilterChainBuilder chain = this.connector.getFilterChain();
>
> chain.clear();
>
> chain.addLast("codec", codecFilter);
>
>
>
> https://cwiki.apache.org/confluence/display/MINA/Configuring+Thread+Model
>
>
> From this link we could understand as below
>
>
>
> If We didn't add any ExecutorFilter events are forwarded to an IoHandler
> via
> method invocations. This means your business logic in your IoHandler
> implementation will run in the I/O processor thread. We call this thread
> model a 'single thread model'. The single thread model is known to be
> adequate for low-latency network applications with CPU-bound business logic
> (e.g. game servers).
>
>
>
> Do we need to add the ExecutorFilter for above mentioned NioSocketConnector
> to System-B so that I/O processor thread will only do the I/O job and all
> other business logic will be given to worker threads in ExecutorFilter.
>
>
>
> Is there any way to increase the number of threads for my I/O to do the
> read/write as we have restriction to create only one session with System-B.
>
>
>
> If we add ExecutorFilter  with IoEventType for read/write then will that
> make the read/write faster with System-B on same single session.
>
>
>
> Or we add ExecutorFilter  with default IoEventType (All Events) will that
> make the read/write faster with System-B on same single session.
>
>
>
> Pls help us understand the working on ExecutorFilter in above scenario
> where
> NioSocketConnector has only single session to do all the read/write with
> huge load of almost 60-70 messages per second with average message size of
> 3-4KiloBytes.
>
>
>
> Currently we have server having 16 Core with 64 GB RAM.
>
>
>
> Thanks And Regards,
>
> Nitin Phuria
>
>
>
> Confidentiality Disclaimer: "The information contained in this electronic
> message (email) and any attachments to this email are intended for the
> exclusive use of the addressee(s) and access to this email by anyone else
> is
> unauthorized. The email may contain proprietary, confidential or privileged
> information or information relating to Integra Group. If you are not the
> intended recipient, please notify the sender by telephone, fax, or return
> email and delete this communication and any attachments thereto,
> immediately
> from your computer. Any dissemination, distribution, or copying of this
> communication and the attachments thereto (in whole or part), in any
> manner,
> is strictly prohibited and actionable at law. The recipient acknowledges
> that emails are susceptible to alteration and their integrity cannot be
> guaranteed and that Company does not guarantee that any e-mail is
> virus-free
> and accept no liability for any 

RE: MINA: ExecutorFilter on NioSocketConnector

[Nitin Phuria]

Dear Jonathan,

Thanks for the reply, can you provide more details on the "All writes 
are queued up and scheduled
later using the poll/kqueue system." How this poll & kqueue system works? Do we 
have any control on this from MINA framework?

We are not facing any problem with the higher CPU consumption but we see that 
when there is high load from System-B to our server the reads from System-B and 
write back to System-B on the single persistence session is becoming slow. So 
we were looking at i/o operations and how to make them fast.

With your explanation provided in below email about ExecutorFilter, If we add 
it in our connector after codec filter will help us reduce the I/O overhead on 
reactor/processor thread. I hope my understanding is right kindly confirm?


Thanks And Regards,
Nitin Phuria

Confidentiality Disclaimer: “The information contained in this electronic 
message (email) and any attachments to this email are intended for the 
exclusive use of the addressee(s) and access to this email by anyone else is 
unauthorized. The email may contain proprietary, confidential or privileged 
information or information relating to Integra Group. If you are not the 
intended recipient, please notify the sender by telephone, fax, or return email 
and delete this communication and any attachments thereto, immediately from 
your computer. Any dissemination, distribution, or copying of this 
communication and the attachments thereto (in whole or part), in any manner, is 
strictly prohibited and actionable at law. The recipient acknowledges that 
emails are susceptible to alteration and their integrity cannot be guaranteed 
and that Company does not guarantee that any e-mail is virus-free and accept no 
liability for any damage caused by any virus transmitted by this email.”

-Original Message-
From: Jonathan Valliere [mailto:jon.valli...@emoten.com] 
Sent: 09 April 2020 18:09
To: Nitin Phuria
Cc: users@mina.apache.org
Subject: Re: MINA: ExecutorFilter on NioSocketConnector

Mina is a queue and flush framework. All writes are queued up and scheduled
later using the poll/kqueue system.

The ExecutorFilter uses threading to process incoming messages using a pool
of threads instead of the reactor/processor thread.  This frees the
processor thread to do IO.

Have you profiled the application to figure out which code is consuming the
CPU?

On Thu, Apr 9, 2020 at 3:39 AM Nitin Phuria 
wrote:

> Dear All,
>
>
>
> We have Server developed using MINA 2.0.16 where it
> connects
> to two other systems say System-A and System-B.
>
>
>
> The connectivity with both these system is persistence which means our
> server connects and create only single session with each of these systems
> and on the same session multiple message exchanges (Request/Response)
> happens.
>
>
>
> The current flow of message is as below:
>
> Once we connect with System-B and have persistence connection (single
> session), System-B will send us message which we have to process in our
> server and then send processed message to System-A then get response from
> System-A and then pass it back to System-B
>
>
>
> We have provision that if load increases we can have multiple instances of
> System-A and our server can establish persistence connection (single
> session) with each of these System-A instances and manage the load.
>
>
>
> We are facing problem with load coming from System-B to our server as it is
> coming on single session and we could see that the read/write on the
> session
> are slowed down and lot of queue build-up is happening for read/write.
>
>
>
> Below is the code for establishing the Connectivity with System-B where we
> have not added the ExecutorFilter
>
>
>
> SocketAddress socketAddress = new InetSocketAddress(this.hostName,
> this.port);
>
> NioSocketConnector connector = new NioSocketConnector();
>
> connector.setConnectTimeoutMillis(30 * 1000L);
>
> connector.getSessionConfig().setTcpNoDelay(true);
>
>
> connector.getSessionConfig().setIdleTime(IdleStatus.BOTH_IDLE,this.echoTime)
> ;
>
> this.connector.setHandler(systemBHandler); // systemBHandler is our
> IoHandler implementationfor System-B
>
> IoFilter codecFilter = new
> ProtocolCodecFilter((ProtocolCodecFactory)systemBProcFactory);//
> systemBProcFactory is our Codec Implementation
>
> DefaultIoFilterChainBuilder chain = this.connector.getFilterChain();
>
> chain.clear();
>
> chain.addLast("codec", codecFilter);
>
>
>
> https://cwiki.apache.org/confluence/display/MINA/Configuring+Thread+Model
>
>
> From this link we could understand as below
>
>
>
> If We didn't add any ExecutorFilter events are forwarded to an IoHandler
> via
> method invocations. This means your business logic in your IoHandler
> implementation will run in the I/O processor thread. We call this thread
> model a 'single thread model'. The single thread model is known to be
> adequate for low-latency network applications with CPU-bound business logic
> (e.g. game 

Re: MINA: ExecutorFilter on NioSocketConnector

[Jonathan Valliere]

The ExecutorFilter only works for the Server receiver side.

When you say the responses are slow, what do you consider slow?

All writes are put into a queue dedicated for each session.  The Session is
then enabled for writing by adding the write interest to the poll
mechanism.  Reads and Writes IO happen in the same thread.  After the
Session finishes reading it will then call the poll mechanism for new
events to process. Hopefully it should get the write event for your Session
then begin to flush the queue.

Processor Thread Loop —>   Select Poll Events —>  Process Events
(read/write) —> Loop

On Thu, Apr 9, 2020 at 8:53 AM Nitin Phuria  wrote:

> Dear Jonathan,
>
> Thanks for the reply, can you provide more details on the "All
> writes are queued up and scheduled
> later using the poll/kqueue system." How this poll & kqueue system works?
> Do we have any control on this from MINA framework?
>
> We are not facing any problem with the higher CPU consumption but we see
> that when there is high load from System-B to our server the reads from
> System-B and write back to System-B on the single persistence session is
> becoming slow. So we were looking at i/o operations and how to make them
> fast.
>
> With your explanation provided in below email about ExecutorFilter, If we
> add it in our connector after codec filter will help us reduce the I/O
> overhead on reactor/processor thread. I hope my understanding is right
> kindly confirm?
>
>
> Thanks And Regards,
> Nitin Phuria
>
> Confidentiality Disclaimer: “The information contained in this electronic
> message (email) and any attachments to this email are intended for the
> exclusive use of the addressee(s) and access to this email by anyone else
> is unauthorized. The email may contain proprietary, confidential or
> privileged information or information relating to Integra Group. If you are
> not the intended recipient, please notify the sender by telephone, fax, or
> return email and delete this communication and any attachments thereto,
> immediately from your computer. Any dissemination, distribution, or copying
> of this communication and the attachments thereto (in whole or part), in
> any manner, is strictly prohibited and actionable at law. The recipient
> acknowledges that emails are susceptible to alteration and their integrity
> cannot be guaranteed and that Company does not guarantee that any e-mail is
> virus-free and accept no liability for any damage caused by any virus
> transmitted by this email.”
>
> -Original Message-
> From: Jonathan Valliere [mailto:jon.valli...@emoten.com]
> Sent: 09 April 2020 18:09
> To: Nitin Phuria
> Cc: users@mina.apache.org
> Subject: Re: MINA: ExecutorFilter on NioSocketConnector
>
> Mina is a queue and flush framework. All writes are queued up and scheduled
> later using the poll/kqueue system.
>
> The ExecutorFilter uses threading to process incoming messages using a pool
> of threads instead of the reactor/processor thread.  This frees the
> processor thread to do IO.
>
> Have you profiled the application to figure out which code is consuming the
> CPU?
>
> On Thu, Apr 9, 2020 at 3:39 AM Nitin Phuria  .invalid>
> wrote:
>
> > Dear All,
> >
> >
> >
> > We have Server developed using MINA 2.0.16 where it
> > connects
> > to two other systems say System-A and System-B.
> >
> >
> >
> > The connectivity with both these system is persistence which means our
> > server connects and create only single session with each of these systems
> > and on the same session multiple message exchanges (Request/Response)
> > happens.
> >
> >
> >
> > The current flow of message is as below:
> >
> > Once we connect with System-B and have persistence connection (single
> > session), System-B will send us message which we have to process in our
> > server and then send processed message to System-A then get response from
> > System-A and then pass it back to System-B
> >
> >
> >
> > We have provision that if load increases we can have multiple instances
> of
> > System-A and our server can establish persistence connection (single
> > session) with each of these System-A instances and manage the load.
> >
> >
> >
> > We are facing problem with load coming from System-B to our server as it
> is
> > coming on single session and we could see that the read/write on the
> > session
> > are slowed down and lot of queue build-up is happening for read/write.
> >
> >
> >
> > Below is the code for establishing the Connectivity with System-B where
> we
> > have not added the ExecutorFilter
> >
> >
> >
> > SocketAddress socketAddress = new InetSocketAddress(this.hostName,
> > this.port);
> >
> > NioSocketConnector connector = new NioSocketConnector();
> >
> > connector.setConnectTimeoutMillis(30 * 1000L);
> >
> > connector.getSessionConfig().setTcpNoDelay(true);
> >
> >
> >
> connector.getSessionConfig().setIdleTime(IdleStatus.BOTH_IDLE,this.echoTime)
> > ;
> >
> > this.connector.setHandler(systemBHandler); 

RE: MINA: ExecutorFilter on NioSocketConnector

[Nitin Phuria]

Dear Jonathan,

Our Server and System-B are on WAN network with leased line. 

 

One more clarification required As per documentation

 

NioSocketConnector 

 () Constructor for  

 NioSocketConnector with default configuration (multiple thread model).

 

So when we create NioSocketConnector connector = new NioSocketConnector(); I 
expect it to create n+1 processor

where n is number of core on the hardware. In our case it is 17 (16 core 
machine)

 

When we took threaddump of our Server process I could see only one thread for 
this connector and in the log (log4j) when we print the thread name it prints 
NioProcessor-x where x is some number.

 

So does it means that n+1 processors use single thread to work.  

 

Also you mentioned that The ExecutorFilter only works for the Server receiver 
side.

 

http://apache-mina.10907.n7.nabble.com/OrderedThreadPoolExecutor-shutdown-problem-in-MINA-2-0-0-M4-td34417.html

 

But I could see from this above URL that we can add it to Connector also and it 
says "My understanding is that the Executor filter just adds an executor in the 
chain which will be used to spread the load on many threads. This is an 
optimization, rather than something you need to use. If you don't use it, your 
program will work fine."

 

Could you provide more insight and your thoughts.

 

Thanks And Regards,

Nitin Phuria

 

Confidentiality Disclaimer: “The information contained in this electronic 
message (email) and any attachments to this email are intended for the 
exclusive use of the addressee(s) and access to this email by anyone else is 
unauthorized. The email may contain proprietary, confidential or privileged 
information or information relating to Integra Group. If you are not the 
intended recipient, please notify the sender by telephone, fax, or return email 
and delete this communication and any attachments thereto, immediately from 
your computer. Any dissemination, distribution, or copying of this 
communication and the attachments thereto (in whole or part), in any manner, is 
strictly prohibited and actionable at law. The recipient acknowledges that 
emails are susceptible to alteration and their integrity cannot be guaranteed 
and that Company does not guarantee that any e-mail is virus-free and accept no 
liability for any damage caused by any virus transmitted by this email.”

 

From: Jonathan Valliere [mailto:jon.valli...@emoten.com] 
Sent: 09 April 2020 19:34
To: Nitin Phuria
Cc: users@mina.apache.org
Subject: Re: MINA: ExecutorFilter on NioSocketConnector

 

How far are the servers?  TCP Packet loss could explain this.

 

You can get the Queue size from the IoSession object. 

 

On Thu, Apr 9, 2020 at 10:02 AM Nitin Phuria  wrote:

Dear Jonathan,

 

Anytime we write message to System-B session with session.write method the 
messageSent event has to be generated with it actually gets written on TCP/IP

 

org.apache.mina.core.session.IoSession.write(Object)

 

We see that there is time difference of 10-14 seconds between timestamp of 
session.write for particular message and messageSent event getting generated 
for the same message.

 

In non-peak load this time difference is in 10-15 milliseconds and in peak load 
(60-70 messages per second) it goes till 10-14 seconds.

 

Is there anything that we can log to see what is happening or see the queue 
size etc..  for further better understaning. 

 

Thanks And Regards,

Nitin Phuria

 

Confidentiality Disclaimer: “The information contained in this electronic 
message (email) and any attachments to this email are intended for the 
exclusive use of the addressee(s) and access to this email by anyone else is 
unauthorized. The email may contain proprietary, confidential or privileged 
information or information relating to Integra Group. If you are not the 
intended recipient, please notify the sender by telephone, fax, or return email 
and delete this communication and any attachments thereto, immediately from 
your computer. Any dissemination, distribution, or copying of this 
communication and the attachments thereto (in whole or part), in any manner, is 
strictly prohibited and actionable at law. The recipient acknowledges that 
emails are susceptible to alteration and their integrity cannot be guaranteed 
and that Company does not guarantee that any e-mail is virus-free and accept no 
liability for any damage caused by any virus transmitted by this email.”

 

From: Jonathan Valliere [mailto:jon.valli...@emoten.com] 
Sent: 09 April 2020 18:31
To: Nitin Phuria
Cc: Nitin Phuria; users@mina.apache.org
Subject: Re: MINA: ExecutorFilter on NioSocketConnector

 

The ExecutorFilter only works for the Server receiver side.

 

When you say the 

RE: MINA: ExecutorFilter on NioSocketConnector

[Nitin Phuria]

Dear Jonathan,

 

Anytime we write message to System-B session with session.write method the 
messageSent event has to be generated with it actually gets written on TCP/IP

 

org.apache.mina.core.session.IoSession.write(Object)

 

We see that there is time difference of 10-14 seconds between timestamp of 
session.write for particular message and messageSent event getting generated 
for the same message.

 

In non-peak load this time difference is in 10-15 milliseconds and in peak load 
(60-70 messages per second) it goes till 10-14 seconds.

 

Is there anything that we can log to see what is happening or see the queue 
size etc..  for further better understaning. 

 

Thanks And Regards,

Nitin Phuria

 

Confidentiality Disclaimer: “The information contained in this electronic 
message (email) and any attachments to this email are intended for the 
exclusive use of the addressee(s) and access to this email by anyone else is 
unauthorized. The email may contain proprietary, confidential or privileged 
information or information relating to Integra Group. If you are not the 
intended recipient, please notify the sender by telephone, fax, or return email 
and delete this communication and any attachments thereto, immediately from 
your computer. Any dissemination, distribution, or copying of this 
communication and the attachments thereto (in whole or part), in any manner, is 
strictly prohibited and actionable at law. The recipient acknowledges that 
emails are susceptible to alteration and their integrity cannot be guaranteed 
and that Company does not guarantee that any e-mail is virus-free and accept no 
liability for any damage caused by any virus transmitted by this email.”

 

From: Jonathan Valliere [mailto:jon.valli...@emoten.com] 
Sent: 09 April 2020 18:31
To: Nitin Phuria
Cc: Nitin Phuria; users@mina.apache.org
Subject: Re: MINA: ExecutorFilter on NioSocketConnector

 

The ExecutorFilter only works for the Server receiver side.

 

When you say the responses are slow, what do you consider slow?

 

All writes are put into a queue dedicated for each session.  The Session is 
then enabled for writing by adding the write interest to the poll mechanism.  
Reads and Writes IO happen in the same thread.  After the Session finishes 
reading it will then call the poll mechanism for new events to process. 
Hopefully it should get the write event for your Session then begin to flush 
the queue. 

 

Processor Thread Loop —>   Select Poll Events —>  Process Events 
(read/write) —> Loop

 

On Thu, Apr 9, 2020 at 8:53 AM Nitin Phuria  wrote:

Dear Jonathan,

Thanks for the reply, can you provide more details on the "All writes 
are queued up and scheduled
later using the poll/kqueue system." How this poll & kqueue system works? Do we 
have any control on this from MINA framework?

We are not facing any problem with the higher CPU consumption but we see that 
when there is high load from System-B to our server the reads from System-B and 
write back to System-B on the single persistence session is becoming slow. So 
we were looking at i/o operations and how to make them fast.

With your explanation provided in below email about ExecutorFilter, If we add 
it in our connector after codec filter will help us reduce the I/O overhead on 
reactor/processor thread. I hope my understanding is right kindly confirm?


Thanks And Regards,
Nitin Phuria

Confidentiality Disclaimer: “The information contained in this electronic 
message (email) and any attachments to this email are intended for the 
exclusive use of the addressee(s) and access to this email by anyone else is 
unauthorized. The email may contain proprietary, confidential or privileged 
information or information relating to Integra Group. If you are not the 
intended recipient, please notify the sender by telephone, fax, or return email 
and delete this communication and any attachments thereto, immediately from 
your computer. Any dissemination, distribution, or copying of this 
communication and the attachments thereto (in whole or part), in any manner, is 
strictly prohibited and actionable at law. The recipient acknowledges that 
emails are susceptible to alteration and their integrity cannot be guaranteed 
and that Company does not guarantee that any e-mail is virus-free and accept no 
liability for any damage caused by any virus transmitted by this email.”

-Original Message-
From: Jonathan Valliere [mailto:jon.valli...@emoten.com] 
Sent: 09 April 2020 18:09
To: Nitin Phuria
Cc: users@mina.apache.org
Subject: Re: MINA: ExecutorFilter on NioSocketConnector

Mina is a queue and flush framework. All writes are queued up and scheduled
later using the poll/kqueue system.

The ExecutorFilter uses threading to process incoming messages using a pool
of threads instead of the reactor/processor thread.  This frees the
processor thread to do IO.

Have you profiled the application to figure out which code is consuming the
CPU?

On Thu, 

Re: MINA: ExecutorFilter on NioSocketConnector

[Jonathan Valliere]

Every IoSession is bound to one Processor Thread.  So only by creating more
IoSessions can you take advantage of multiple Processor Threads.

ExecutorFilter only works for received messages.  So it generally does
nothing on the client side however it will cause the received messages to
be executed in multiple threads.

The purpose of the ExecutorFilter is to do potentially blocking tasks in
there so as not to block the Processor Thread from handling tasks from
other IoSessions.

Since you’re seeing a huge jump from 15ms to several minutes and a very
small number of messages it is most likely a problem with your network
connection.

On Thu, Apr 9, 2020 at 10:26 AM Nitin Phuria 
wrote:

> Dear Jonathan,
>
> Our Server and System-B are on WAN network with leased line.
>
>
>
> One more clarification required As per documentation
>
>
>
> NioSocketConnector
> 
> () Constructor for NioSocketConnector
> 
> with default configuration (multiple thread model).
>
>
>
> So when we create NioSocketConnector connector = new NioSocketConnector(); I 
> expect it to create n+1 processor
>
> where n is number of core on the hardware. In our case it is 17 (16 core
> machine)
>
>
>
> When we took threaddump of our Server process I could see only one thread
> for this connector and in the log (log4j) when we print the thread name it
> prints NioProcessor-x where x is some number.
>
>
>
> So does it means that n+1 processors use single thread to work.
>
>
>
> Also you mentioned that The ExecutorFilter only works for the Server
> receiver side.
>
>
>
>
> http://apache-mina.10907.n7.nabble.com/OrderedThreadPoolExecutor-shutdown-problem-in-MINA-2-0-0-M4-td34417.html
>
>
>
> But I could see from this above URL that we can add it to Connector also
> and it says "*My understanding is that the Executor filter just adds an
> executor in the chain which will be used to spread the load on many
> threads. This is an optimization, rather than something you need to use. If
> you don't use it, your program will work fine.*"
>
>
>
> Could you provide more insight and your thoughts.
>
>
>
> Thanks And Regards,
>
> Nitin Phuria
>
>
>
> *Confidentiality Disclaimer**: “The information contained in this
> electronic message (email) and any attachments to this email are intended
> for the exclusive use of the addressee(s) and access to this email by
> anyone else is unauthorized. The email may contain proprietary,
> confidential or privileged information or information relating to Integra
> Group. If you are not the intended recipient, please notify the sender by
> telephone, fax, or return email and delete this communication and any
> attachments thereto, immediately from your computer. Any dissemination,
> distribution, or copying of this communication and the attachments thereto
> (in whole or part), in any manner, is strictly prohibited and actionable at
> law. The recipient acknowledges that emails are susceptible to alteration
> and their integrity cannot be guaranteed and that Company does not
> guarantee that any e-mail is virus-free and accept no liability for any
> damage caused by any virus transmitted by this email.”*
>
>
>
> *From:* Jonathan Valliere [mailto:jon.valli...@emoten.com]
> *Sent:* 09 April 2020 19:34
> *To:* Nitin Phuria
> *Cc:* users@mina.apache.org
> *Subject:* Re: MINA: ExecutorFilter on NioSocketConnector
>
>
>
> How far are the servers?  TCP Packet loss could explain this.
>
>
>
> You can get the Queue size from the IoSession object.
>
>
>
> On Thu, Apr 9, 2020 at 10:02 AM Nitin Phuria 
> wrote:
>
> Dear Jonathan,
>
>
>
> Anytime we write message to System-B session with session.write method the
> messageSent event has to be generated with it actually gets written on
> TCP/IP
>
>
>
> org.apache.mina.core.session.IoSession.write(Object)
>
>
>
> We see that there is time difference of 10-14 seconds between timestamp of
> session.write for particular message and messageSent event getting
> generated for the same message.
>
>
>
> In non-peak load this time difference is in 10-15 milliseconds and in peak
> load (60-70 messages per second) it goes till 10-14 seconds.
>
>
>
> Is there anything that we can log to see what is happening or see the
> queue size etc..  for further better understaning.
>
>
>
> Thanks And Regards,
>
> Nitin Phuria
>
>
>
> *Confidentiality Disclaimer**: “The information contained in this
> electronic message (email) and any attachments to this email are intended
> for the exclusive use of the addressee(s) and access to this email by
> anyone else is unauthorized. The email may contain proprietary,
> confidential or privileged information or information relating to Integra
> Group. If you are not the intended recipient, please notify the 

Re: MINA: ExecutorFilter on NioSocketConnector

[Jonathan Valliere]

How far are the servers?  TCP Packet loss could explain this.

You can get the Queue size from the IoSession object.

On Thu, Apr 9, 2020 at 10:02 AM Nitin Phuria 
wrote:

> Dear Jonathan,
>
>
>
> Anytime we write message to System-B session with session.write method the
> messageSent event has to be generated with it actually gets written on
> TCP/IP
>
>
>
> org.apache.mina.core.session.IoSession.write(Object)
>
>
>
> We see that there is time difference of 10-14 seconds between timestamp of
> session.write for particular message and messageSent event getting
> generated for the same message.
>
>
>
> In non-peak load this time difference is in 10-15 milliseconds and in peak
> load (60-70 messages per second) it goes till 10-14 seconds.
>
>
>
> Is there anything that we can log to see what is happening or see the
> queue size etc..  for further better understaning.
>
>
>
> Thanks And Regards,
>
> Nitin Phuria
>
>
>
> *Confidentiality Disclaimer**: “The information contained in this
> electronic message (email) and any attachments to this email are intended
> for the exclusive use of the addressee(s) and access to this email by
> anyone else is unauthorized. The email may contain proprietary,
> confidential or privileged information or information relating to Integra
> Group. If you are not the intended recipient, please notify the sender by
> telephone, fax, or return email and delete this communication and any
> attachments thereto, immediately from your computer. Any dissemination,
> distribution, or copying of this communication and the attachments thereto
> (in whole or part), in any manner, is strictly prohibited and actionable at
> law. The recipient acknowledges that emails are susceptible to alteration
> and their integrity cannot be guaranteed and that Company does not
> guarantee that any e-mail is virus-free and accept no liability for any
> damage caused by any virus transmitted by this email.”*
>
>
>
> *From:* Jonathan Valliere [mailto:jon.valli...@emoten.com]
> *Sent:* 09 April 2020 18:31
> *To:* Nitin Phuria
> *Cc:* Nitin Phuria; users@mina.apache.org
> *Subject:* Re: MINA: ExecutorFilter on NioSocketConnector
>
>
>
> The ExecutorFilter only works for the Server receiver side.
>
>
>
> When you say the responses are slow, what do you consider slow?
>
>
>
> All writes are put into a queue dedicated for each session.  The Session
> is then enabled for writing by adding the write interest to the poll
> mechanism.  Reads and Writes IO happen in the same thread.  After the
> Session finishes reading it will then call the poll mechanism for new
> events to process. Hopefully it should get the write event for your Session
> then begin to flush the queue.
>
>
>
> Processor Thread Loop —>   Select Poll Events —>  Process Events
> (read/write) —> Loop
>
>
>
> On Thu, Apr 9, 2020 at 8:53 AM Nitin Phuria 
> wrote:
>
> Dear Jonathan,
>
> Thanks for the reply, can you provide more details on the "All
> writes are queued up and scheduled
> later using the poll/kqueue system." How this poll & kqueue system works?
> Do we have any control on this from MINA framework?
>
> We are not facing any problem with the higher CPU consumption but we see
> that when there is high load from System-B to our server the reads from
> System-B and write back to System-B on the single persistence session is
> becoming slow. So we were looking at i/o operations and how to make them
> fast.
>
> With your explanation provided in below email about ExecutorFilter, If we
> add it in our connector after codec filter will help us reduce the I/O
> overhead on reactor/processor thread. I hope my understanding is right
> kindly confirm?
>
>
> Thanks And Regards,
> Nitin Phuria
>
> Confidentiality Disclaimer: “The information contained in this electronic
> message (email) and any attachments to this email are intended for the
> exclusive use of the addressee(s) and access to this email by anyone else
> is unauthorized. The email may contain proprietary, confidential or
> privileged information or information relating to Integra Group. If you are
> not the intended recipient, please notify the sender by telephone, fax, or
> return email and delete this communication and any attachments thereto,
> immediately from your computer. Any dissemination, distribution, or copying
> of this communication and the attachments thereto (in whole or part), in
> any manner, is strictly prohibited and actionable at law. The recipient
> acknowledges that emails are susceptible to alteration and their integrity
> cannot be guaranteed and that Company does not guarantee that any e-mail is
> virus-free and accept no liability for any damage caused by any virus
> transmitted by this email.”
>
> -Original Message-
> From: Jonathan Valliere [mailto:jon.valli...@emoten.com]
> Sent: 09 April 2020 18:09
> To: Nitin Phuria
> Cc: users@mina.apache.org
> Subject: Re: MINA: ExecutorFilter on NioSocketConnector
>
> Mina is a queue and flush framework. 

Re: MINA: ExecutorFilter on NioSocketConnector

[Emmanuel Lécharny]

On 09/04/2020 15:00, Jonathan Valliere wrote:

The ExecutorFilter only works for the Server receiver side.



Actually, it could be used on the client side too, but it makes little 
sense, as the client is not supposed to receive heavy load from the 
server (of course, there might be use cases where it's happening, but 
it's not frequent).




-
To unsubscribe, e-mail: users-unsubscr...@mina.apache.org
For additional commands, e-mail: users-h...@mina.apache.org

RE: MINA: ExecutorFilter on NioSocketConnector

[Nitin Phuria]

Dear Jonathan,

 

Thanks for bringing more clarity on Processor Thread and it's 
relation with IoSession. We are also suspecting the network connection and in a 
crisis one should pull all the strings to check where you get the lead to 
identify the problem and that's what we are doing. 

 

Thanks a tone for all your replies and giving very useful information. We will 
check further on network  connection.

 

Thanks And Regards,

Nitin Phuria

 

Confidentiality Disclaimer: “The information contained in this electronic 
message (email) and any attachments to this email are intended for the 
exclusive use of the addressee(s) and access to this email by anyone else is 
unauthorized. The email may contain proprietary, confidential or privileged 
information or information relating to Integra Group. If you are not the 
intended recipient, please notify the sender by telephone, fax, or return email 
and delete this communication and any attachments thereto, immediately from 
your computer. Any dissemination, distribution, or copying of this 
communication and the attachments thereto (in whole or part), in any manner, is 
strictly prohibited and actionable at law. The recipient acknowledges that 
emails are susceptible to alteration and their integrity cannot be guaranteed 
and that Company does not guarantee that any e-mail is virus-free and accept no 
liability for any damage caused by any virus transmitted by this email.”

 

From: Jonathan Valliere [mailto:jon.valli...@emoten.com] 
Sent: 09 April 2020 20:03
To: Nitin Phuria
Cc: users@mina.apache.org
Subject: Re: MINA: ExecutorFilter on NioSocketConnector

 

Every IoSession is bound to one Processor Thread.  So only by creating more 
IoSessions can you take advantage of multiple Processor Threads. 

 

ExecutorFilter only works for received messages.  So it generally does nothing 
on the client side however it will cause the received messages to be executed 
in multiple threads. 

 

The purpose of the ExecutorFilter is to do potentially blocking tasks in there 
so as not to block the Processor Thread from handling tasks from other 
IoSessions. 

 

Since you’re seeing a huge jump from 15ms to several minutes and a very small 
number of messages it is most likely a problem with your network connection.

 

On Thu, Apr 9, 2020 at 10:26 AM Nitin Phuria  wrote:

Dear Jonathan,

Our Server and System-B are on WAN network with leased line. 

 

One more clarification required As per documentation

 

NioSocketConnector 

 () Constructor for  

 NioSocketConnector with default configuration (multiple thread model).

 

So when we create NioSocketConnector connector = new NioSocketConnector(); I 
expect it to create n+1 processor

where n is number of core on the hardware. In our case it is 17 (16 core 
machine)

 

When we took threaddump of our Server process I could see only one thread for 
this connector and in the log (log4j) when we print the thread name it prints 
NioProcessor-x where x is some number.

 

So does it means that n+1 processors use single thread to work.  

 

Also you mentioned that The ExecutorFilter only works for the Server receiver 
side.

 

http://apache-mina.10907.n7.nabble.com/OrderedThreadPoolExecutor-shutdown-problem-in-MINA-2-0-0-M4-td34417.html

 

But I could see from this above URL that we can add it to Connector also and it 
says "My understanding is that the Executor filter just adds an executor in the 
chain which will be used to spread the load on many threads. This is an 
optimization, rather than something you need to use. If you don't use it, your 
program will work fine."

 

Could you provide more insight and your thoughts.

 

Thanks And Regards,

Nitin Phuria

 

Confidentiality Disclaimer: “The information contained in this electronic 
message (email) and any attachments to this email are intended for the 
exclusive use of the addressee(s) and access to this email by anyone else is 
unauthorized. The email may contain proprietary, confidential or privileged 
information or information relating to Integra Group. If you are not the 
intended recipient, please notify the sender by telephone, fax, or return email 
and delete this communication and any attachments thereto, immediately from 
your computer. Any dissemination, distribution, or copying of this 
communication and the attachments thereto (in whole or part), in any manner, is 
strictly prohibited and actionable at law. The recipient acknowledges that 
emails are susceptible to alteration and their integrity cannot be guaranteed 
and that Company does not guarantee that any e-mail is virus-free and accept no 
liability for any damage caused by any virus transmitted by this email.”

 

Re: MINA: ExecutorFilter on NioSocketConnector

[Jonathan Valliere]

Make sure you enable BBR TCP on your host machines to help mitigate the
network problems.

On Thu, Apr 9, 2020 at 10:58 AM Nitin Phuria 
wrote:

> Dear Jonathan,
>
>
>
> Thanks for bringing more clarity on Processor Thread and
> it's relation with IoSession. We are also suspecting the network connection
> and in a crisis one should pull all the strings to check where you get the
> lead to identify the problem and that's what we are doing.
>
>
>
> Thanks a tone for all your replies and giving very useful information. We
> will check further on network  connection.
>
>
>
> Thanks And Regards,
>
> Nitin Phuria
>
>
>
> *Confidentiality Disclaimer**: “The information contained in this
> electronic message (email) and any attachments to this email are intended
> for the exclusive use of the addressee(s) and access to this email by
> anyone else is unauthorized. The email may contain proprietary,
> confidential or privileged information or information relating to Integra
> Group. If you are not the intended recipient, please notify the sender by
> telephone, fax, or return email and delete this communication and any
> attachments thereto, immediately from your computer. Any dissemination,
> distribution, or copying of this communication and the attachments thereto
> (in whole or part), in any manner, is strictly prohibited and actionable at
> law. The recipient acknowledges that emails are susceptible to alteration
> and their integrity cannot be guaranteed and that Company does not
> guarantee that any e-mail is virus-free and accept no liability for any
> damage caused by any virus transmitted by this email.”*
>
>
>
> *From:* Jonathan Valliere [mailto:jon.valli...@emoten.com]
> *Sent:* 09 April 2020 20:03
> *To:* Nitin Phuria
> *Cc:* users@mina.apache.org
> *Subject:* Re: MINA: ExecutorFilter on NioSocketConnector
>
>
>
> Every IoSession is bound to one Processor Thread.  So only by creating
> more IoSessions can you take advantage of multiple Processor Threads.
>
>
>
> ExecutorFilter only works for received messages.  So it generally does
> nothing on the client side however it will cause the received messages to
> be executed in multiple threads.
>
>
>
> The purpose of the ExecutorFilter is to do potentially blocking tasks in
> there so as not to block the Processor Thread from handling tasks from
> other IoSessions.
>
>
>
> Since you’re seeing a huge jump from 15ms to several minutes and a very
> small number of messages it is most likely a problem with your network
> connection.
>
>
>
> On Thu, Apr 9, 2020 at 10:26 AM Nitin Phuria 
> wrote:
>
> Dear Jonathan,
>
> Our Server and System-B are on WAN network with leased line.
>
>
>
> One more clarification required As per documentation
>
>
>
> NioSocketConnector
> 
> () Constructor for NioSocketConnector
> 
> with default configuration (multiple thread model).
>
>
>
> So when we create NioSocketConnector connector = new NioSocketConnector(); I 
> expect it to create n+1 processor
>
> where n is number of core on the hardware. In our case it is 17 (16 core
> machine)
>
>
>
> When we took threaddump of our Server process I could see only one thread
> for this connector and in the log (log4j) when we print the thread name it
> prints NioProcessor-x where x is some number.
>
>
>
> So does it means that n+1 processors use single thread to work.
>
>
>
> Also you mentioned that The ExecutorFilter only works for the Server
> receiver side.
>
>
>
>
> http://apache-mina.10907.n7.nabble.com/OrderedThreadPoolExecutor-shutdown-problem-in-MINA-2-0-0-M4-td34417.html
>
>
>
> But I could see from this above URL that we can add it to Connector also
> and it says "*My understanding is that the Executor filter just adds an
> executor in the chain which will be used to spread the load on many
> threads. This is an optimization, rather than something you need to use. If
> you don't use it, your program will work fine.*"
>
>
>
> Could you provide more insight and your thoughts.
>
>
>
> Thanks And Regards,
>
> Nitin Phuria
>
>
>
> *Confidentiality Disclaimer**: “The information contained in this
> electronic message (email) and any attachments to this email are intended
> for the exclusive use of the addressee(s) and access to this email by
> anyone else is unauthorized. The email may contain proprietary,
> confidential or privileged information or information relating to Integra
> Group. If you are not the intended recipient, please notify the sender by
> telephone, fax, or return email and delete this communication and any
> attachments thereto, immediately from your computer. Any dissemination,
> distribution, or copying of this communication and the attachments thereto
> (in whole or part), in any 

Re: MINA: ExecutorFilter on NioSocketConnector

[Emmanuel Lécharny]

On 09/04/2020 16:01, Nitin Phuria wrote:

Dear Jonathan,

  


Anytime we write message to System-B session with session.write method the 
messageSent event has to be generated with it actually gets written on TCP/IP



The messageSent event is only generated when the message has been fully 
written into the socket. A message can be written piece by piece, 
depending on the socket capacity. Also messages can be queued until the 
previous messages have been fully written.



What could happen is that if you have a slow reader on the other side, 
then the socket will not be able to accept any data, because it gets full.





  


org.apache.mina.core.session.IoSession.write(Object)

  


We see that there is time difference of 10-14 seconds between timestamp of 
session.write for particular message and messageSent event getting generated 
for the same message.



So you have a problem in the middle. Check that the reader (ie the other 
server) is processing messages fast enough. Or maybe you have network 
issues, as suggests Jonathan.





  


In non-peak load this time difference is in 10-15 milliseconds and in peak load 
(60-70 messages per second) it goes till 10-14 seconds.



60/70 msg/s is not exceptionally high, except if you are managing big 
messages, or do a lot of processing on each message.




-
To unsubscribe, e-mail: users-unsubscr...@mina.apache.org
For additional commands, e-mail: users-h...@mina.apache.org