Re: Individual SSLFilter per connector

[Jonathan Valliere]

 Okay, let me know when you’re able to test with that branch.  I think it
should solve your problem.

On Feb 20, 2024 at 6:32:06 AM, Kishore Mokkarala 
wrote:

> @ Valliere I am not available for a week.
>
> On Tue, 20 Feb, 2024, 8:59 am Jonathan Valliere, 
> wrote:
>
> @Kishore Mokkarala  are you able to test this
>
> branch? https://github.com/apache/mina/compare/2.2.X...bugfix/DIRMINA-1173
>
>
> On Feb 18, 2024 at 2:05:57 PM, Jonathan Valliere 
>
> wrote:
>
>
> > The other diagram possibly had a consumer order issue.  While the Queue
>
> > will guarantee that the messages will be pulled out of the Queue in
> order,
>
> > they do not guarantee that the processing of the messages will happen in
>
> > order.
>
> >
>
> > The new diagram implements 3 synchronization objects.  This will ensure
>
> > that messages are produced and enter the respective queues in guaranteed
>
> > order and that they will be consumed with upstream or downstream in order
>
> > without locking both directions concurrently.
>
> >
>
> > On Feb 18, 2024 at 1:47:58 PM, Jonathan Valliere 
>
> > wrote:
>
> >
>
> >> Emmanuel,
>
> >>
>
> >> The attached diagram is how I figured out we can solve this.  The
>
> >> downside is that it requires more concurrent queues and more lock/unlock
>
> >> but I think it should ensure correct execution order.
>
> >>
>
> >> I’m working on this now as SSLHandlerG1 so it stays separate from the
>
> >> reference implementation.
>
> >>
>
> >> On Feb 17, 2024 at 7:33:21 PM, Jonathan Valliere 
>
> >> wrote:
>
> >>
>
> >>> Okay so I need to figure out how to work it so no lock is held while
>
> >>> calling either the upper or lower filter.
>
> >>>
>
> >>> CONFIDENTIALITY NOTICE: The contents of this email message and any
>
> >>> attachments are intended solely for the addressee(s) and may contain
>
> >>> confidential and/or privileged information and may be legally
>
> >>> protected from disclosure.
>
> >>>
>
> >>>
>
> >>> On Sat, Feb 17, 2024 at 5:06 PM Emmanuel Lécharny  >
>
> >>> wrote:
>
> >>>
>
> >>>> Hi Jonathan,
>
> >>>>
>
> >>>> Kishore provided a thrzad dump a few weeks ago, which shows that there
>
> >>>> is a lock:
>
> >>>>
>
> >>>> NioProcessor-12
>
> >>>> ---
>
> >>>> stackTrace:
>
> >>>> java.lang.Thread.State: BLOCKED (on object monitor)
>
> >>>> at
>
> >>>>
> org.apache.mina.statemachine.StateMachine.handle(StateMachine.java:138)
>
> >>>> - waiting to lock <0x7fc1611faec8> (a
>
> >>>> com.netscout.nsaapp.geo.minaG10Proto.server.G10StateContext)
>
> >>>> at
>
> >>>>
>
> >>>>
> org.apache.mina.statemachine.StateMachineProxyBuilder$MethodInvocationHandler.invoke(StateMachineProxyBuilder.java:261)
>
> >>>> at jdk.proxy4.$Proxy83.event(jdk.proxy4/Unknown Source)
>
> >>>> at
>
> >>>>
>
> >>>>
> org.apache.mina.core.filterchain.DefaultIoFilterChain$TailFilter.event(DefaultIoFilterChain.java:1039)
>
> >>>> at
>
> >>>>
>
> >>>>
> org.apache.mina.core.filterchain.DefaultIoFilterChain.callNextFilterEvent(DefaultIoFilterChain.java:789)
>
> >>>> at
>
> >>>>
>
> >>>>
> org.apache.mina.core.filterchain.DefaultIoFilterChain.access$1700(DefaultIoFilterChain.java:49)
>
> >>>> at
>
> >>>>
>
> >>>>
> org.apache.mina.core.filterchain.DefaultIoFilterChain$EntryImpl$1.event(DefaultIoFilterChain.java:1164)
>
> >>>> at
>
> >>>>
>
> >>>>
> org.apache.mina.core.filterchain.IoFilterAdapter.event(IoFilterAdapter.java:162)
>
> >>>> at
>
> >>>>
>
> >>>>
> org.apache.mina.core.filterchain.DefaultIoFilterChain.callNextFilterEvent(DefaultIoFilterChain.java:789)
>
> >>>> at
>
> >>>>
>
> >>>>
> org.apache.mina.core.filterchain.DefaultIoFilterChain.access$1700(DefaultIoFilterChain.java:49)
>
> >>>> at
>
> >>>>
>
> &g

Re: Individual SSLFilter per connector

[Jonathan Valliere]

@Kishore Mokkarala  are you able to test this
branch? https://github.com/apache/mina/compare/2.2.X...bugfix/DIRMINA-1173

On Feb 18, 2024 at 2:05:57 PM, Jonathan Valliere  wrote:

> The other diagram possibly had a consumer order issue.  While the Queue
> will guarantee that the messages will be pulled out of the Queue in order,
> they do not guarantee that the processing of the messages will happen in
> order.
>
> The new diagram implements 3 synchronization objects.  This will ensure
> that messages are produced and enter the respective queues in guaranteed
> order and that they will be consumed with upstream or downstream in order
> without locking both directions concurrently.
>
> On Feb 18, 2024 at 1:47:58 PM, Jonathan Valliere 
> wrote:
>
>> Emmanuel,
>>
>> The attached diagram is how I figured out we can solve this.  The
>> downside is that it requires more concurrent queues and more lock/unlock
>> but I think it should ensure correct execution order.
>>
>> I’m working on this now as SSLHandlerG1 so it stays separate from the
>> reference implementation.
>>
>> On Feb 17, 2024 at 7:33:21 PM, Jonathan Valliere 
>> wrote:
>>
>>> Okay so I need to figure out how to work it so no lock is held while
>>> calling either the upper or lower filter.
>>>
>>> CONFIDENTIALITY NOTICE: The contents of this email message and any
>>> attachments are intended solely for the addressee(s) and may contain
>>> confidential and/or privileged information and may be legally protected
>>> from disclosure.
>>>
>>>
>>> On Sat, Feb 17, 2024 at 5:06 PM Emmanuel Lécharny 
>>> wrote:
>>>
>>>> Hi Jonathan,
>>>>
>>>> Kishore provided a thrzad dump a few weeks ago, which shows that there
>>>> is a lock:
>>>>
>>>> NioProcessor-12
>>>> ---
>>>> stackTrace:
>>>> java.lang.Thread.State: BLOCKED (on object monitor)
>>>> at
>>>> org.apache.mina.statemachine.StateMachine.handle(StateMachine.java:138)
>>>> - waiting to lock <0x7fc1611faec8> (a
>>>> com.netscout.nsaapp.geo.minaG10Proto.server.G10StateContext)
>>>> at
>>>>
>>>> org.apache.mina.statemachine.StateMachineProxyBuilder$MethodInvocationHandler.invoke(StateMachineProxyBuilder.java:261)
>>>> at jdk.proxy4.$Proxy83.event(jdk.proxy4/Unknown Source)
>>>> at
>>>>
>>>> org.apache.mina.core.filterchain.DefaultIoFilterChain$TailFilter.event(DefaultIoFilterChain.java:1039)
>>>> at
>>>>
>>>> org.apache.mina.core.filterchain.DefaultIoFilterChain.callNextFilterEvent(DefaultIoFilterChain.java:789)
>>>> at
>>>>
>>>> org.apache.mina.core.filterchain.DefaultIoFilterChain.access$1700(DefaultIoFilterChain.java:49)
>>>> at
>>>>
>>>> org.apache.mina.core.filterchain.DefaultIoFilterChain$EntryImpl$1.event(DefaultIoFilterChain.java:1164)
>>>> at
>>>>
>>>> org.apache.mina.core.filterchain.IoFilterAdapter.event(IoFilterAdapter.java:162)
>>>> at
>>>>
>>>> org.apache.mina.core.filterchain.DefaultIoFilterChain.callNextFilterEvent(DefaultIoFilterChain.java:789)
>>>> at
>>>>
>>>> org.apache.mina.core.filterchain.DefaultIoFilterChain.access$1700(DefaultIoFilterChain.java:49)
>>>> at
>>>>
>>>> org.apache.mina.core.filterchain.DefaultIoFilterChain$EntryImpl$1.event(DefaultIoFilterChain.java:1164)
>>>> at
>>>>
>>>> org.apache.mina.core.filterchain.IoFilterAdapter.event(IoFilterAdapter.java:162)
>>>> at
>>>>
>>>> org.apache.mina.core.filterchain.DefaultIoFilterChain.callNextFilterEvent(DefaultIoFilterChain.java:789)
>>>> at
>>>>
>>>> org.apache.mina.core.filterchain.DefaultIoFilterChain.access$1700(DefaultIoFilterChain.java:49)
>>>> at
>>>>
>>>> org.apache.mina.core.filterchain.DefaultIoFilterChain$EntryImpl$1.event(DefaultIoFilterChain.java:1164)
>>>> at
>>>>
>>>> org.apache.mina.core.filterchain.IoFilterAdapter.event(IoFilterAdapter.java:162)
>>>> at
>>>>
>>>> org.apache.mina.core.filterchain.DefaultIoFilterChain.callNextFilterEvent(DefaultIoFilterChain.java:789)
>>>> at
>>>>
>>>> org.apache.mina.core.filterchain.DefaultIoFilterChain.access$1700(DefaultIoFilterChain.java:49)
>>>> at
>>>>
>>>> org.apache.mi

Re: Individual SSLFilter per connector

[Jonathan Valliere]

The other diagram possibly had a consumer order issue.  While the Queue
will guarantee that the messages will be pulled out of the Queue in order,
they do not guarantee that the processing of the messages will happen in
order.

The new diagram implements 3 synchronization objects.  This will ensure
that messages are produced and enter the respective queues in guaranteed
order and that they will be consumed with upstream or downstream in order
without locking both directions concurrently.

On Feb 18, 2024 at 1:47:58 PM, Jonathan Valliere  wrote:

> Emmanuel,
>
> The attached diagram is how I figured out we can solve this.  The downside
> is that it requires more concurrent queues and more lock/unlock but I think
> it should ensure correct execution order.
>
> I’m working on this now as SSLHandlerG1 so it stays separate from the
> reference implementation.
>
> On Feb 17, 2024 at 7:33:21 PM, Jonathan Valliere 
> wrote:
>
>> Okay so I need to figure out how to work it so no lock is held while
>> calling either the upper or lower filter.
>>
>> CONFIDENTIALITY NOTICE: The contents of this email message and any
>> attachments are intended solely for the addressee(s) and may contain
>> confidential and/or privileged information and may be legally protected
>> from disclosure.
>>
>>
>> On Sat, Feb 17, 2024 at 5:06 PM Emmanuel Lécharny 
>> wrote:
>>
>>> Hi Jonathan,
>>>
>>> Kishore provided a thrzad dump a few weeks ago, which shows that there
>>> is a lock:
>>>
>>> NioProcessor-12
>>> ---
>>> stackTrace:
>>> java.lang.Thread.State: BLOCKED (on object monitor)
>>> at
>>> org.apache.mina.statemachine.StateMachine.handle(StateMachine.java:138)
>>> - waiting to lock <0x7fc1611faec8> (a
>>> com.netscout.nsaapp.geo.minaG10Proto.server.G10StateContext)
>>> at
>>>
>>> org.apache.mina.statemachine.StateMachineProxyBuilder$MethodInvocationHandler.invoke(StateMachineProxyBuilder.java:261)
>>> at jdk.proxy4.$Proxy83.event(jdk.proxy4/Unknown Source)
>>> at
>>>
>>> org.apache.mina.core.filterchain.DefaultIoFilterChain$TailFilter.event(DefaultIoFilterChain.java:1039)
>>> at
>>>
>>> org.apache.mina.core.filterchain.DefaultIoFilterChain.callNextFilterEvent(DefaultIoFilterChain.java:789)
>>> at
>>>
>>> org.apache.mina.core.filterchain.DefaultIoFilterChain.access$1700(DefaultIoFilterChain.java:49)
>>> at
>>>
>>> org.apache.mina.core.filterchain.DefaultIoFilterChain$EntryImpl$1.event(DefaultIoFilterChain.java:1164)
>>> at
>>>
>>> org.apache.mina.core.filterchain.IoFilterAdapter.event(IoFilterAdapter.java:162)
>>> at
>>>
>>> org.apache.mina.core.filterchain.DefaultIoFilterChain.callNextFilterEvent(DefaultIoFilterChain.java:789)
>>> at
>>>
>>> org.apache.mina.core.filterchain.DefaultIoFilterChain.access$1700(DefaultIoFilterChain.java:49)
>>> at
>>>
>>> org.apache.mina.core.filterchain.DefaultIoFilterChain$EntryImpl$1.event(DefaultIoFilterChain.java:1164)
>>> at
>>>
>>> org.apache.mina.core.filterchain.IoFilterAdapter.event(IoFilterAdapter.java:162)
>>> at
>>>
>>> org.apache.mina.core.filterchain.DefaultIoFilterChain.callNextFilterEvent(DefaultIoFilterChain.java:789)
>>> at
>>>
>>> org.apache.mina.core.filterchain.DefaultIoFilterChain.access$1700(DefaultIoFilterChain.java:49)
>>> at
>>>
>>> org.apache.mina.core.filterchain.DefaultIoFilterChain$EntryImpl$1.event(DefaultIoFilterChain.java:1164)
>>> at
>>>
>>> org.apache.mina.core.filterchain.IoFilterAdapter.event(IoFilterAdapter.java:162)
>>> at
>>>
>>> org.apache.mina.core.filterchain.DefaultIoFilterChain.callNextFilterEvent(DefaultIoFilterChain.java:789)
>>> at
>>>
>>> org.apache.mina.core.filterchain.DefaultIoFilterChain.access$1700(DefaultIoFilterChain.java:49)
>>> at
>>>
>>> org.apache.mina.core.filterchain.DefaultIoFilterChain$EntryImpl$1.event(DefaultIoFilterChain.java:1164)
>>> at
>>>
>>> org.apache.mina.core.filterchain.IoFilterAdapter.event(IoFilterAdapter.java:162)
>>> at
>>>
>>> org.apache.mina.core.filterchain.DefaultIoFilterChain.callNextFilterEvent(DefaultIoFilterChain.java:789)
>>> at
>>>
>>> org.apache.mina.core.filterchain.DefaultIoFilterChain.access$1700(DefaultIoFilterChain.java:49)
>>> at
>>>
>>> org.apache.mina.core.filterchain.DefaultIoFilterChain$EntryIm

Re: Individual SSLFilter per connector

[Jonathan Valliere]

 Emmanuel,

The attached diagram is how I figured out we can solve this.  The downside
is that it requires more concurrent queues and more lock/unlock but I think
it should ensure correct execution order.

I’m working on this now as SSLHandlerG1 so it stays separate from the
reference implementation.

On Feb 17, 2024 at 7:33:21 PM, Jonathan Valliere  wrote:

> Okay so I need to figure out how to work it so no lock is held while
> calling either the upper or lower filter.
>
> CONFIDENTIALITY NOTICE: The contents of this email message and any
> attachments are intended solely for the addressee(s) and may contain
> confidential and/or privileged information and may be legally protected
> from disclosure.
>
>
> On Sat, Feb 17, 2024 at 5:06 PM Emmanuel Lécharny 
> wrote:
>
>> Hi Jonathan,
>>
>> Kishore provided a thrzad dump a few weeks ago, which shows that there
>> is a lock:
>>
>> NioProcessor-12
>> ---
>> stackTrace:
>> java.lang.Thread.State: BLOCKED (on object monitor)
>> at org.apache.mina.statemachine.StateMachine.handle(StateMachine.java:138)
>> - waiting to lock <0x7fc1611faec8> (a
>> com.netscout.nsaapp.geo.minaG10Proto.server.G10StateContext)
>> at
>>
>> org.apache.mina.statemachine.StateMachineProxyBuilder$MethodInvocationHandler.invoke(StateMachineProxyBuilder.java:261)
>> at jdk.proxy4.$Proxy83.event(jdk.proxy4/Unknown Source)
>> at
>>
>> org.apache.mina.core.filterchain.DefaultIoFilterChain$TailFilter.event(DefaultIoFilterChain.java:1039)
>> at
>>
>> org.apache.mina.core.filterchain.DefaultIoFilterChain.callNextFilterEvent(DefaultIoFilterChain.java:789)
>> at
>>
>> org.apache.mina.core.filterchain.DefaultIoFilterChain.access$1700(DefaultIoFilterChain.java:49)
>> at
>>
>> org.apache.mina.core.filterchain.DefaultIoFilterChain$EntryImpl$1.event(DefaultIoFilterChain.java:1164)
>> at
>>
>> org.apache.mina.core.filterchain.IoFilterAdapter.event(IoFilterAdapter.java:162)
>> at
>>
>> org.apache.mina.core.filterchain.DefaultIoFilterChain.callNextFilterEvent(DefaultIoFilterChain.java:789)
>> at
>>
>> org.apache.mina.core.filterchain.DefaultIoFilterChain.access$1700(DefaultIoFilterChain.java:49)
>> at
>>
>> org.apache.mina.core.filterchain.DefaultIoFilterChain$EntryImpl$1.event(DefaultIoFilterChain.java:1164)
>> at
>>
>> org.apache.mina.core.filterchain.IoFilterAdapter.event(IoFilterAdapter.java:162)
>> at
>>
>> org.apache.mina.core.filterchain.DefaultIoFilterChain.callNextFilterEvent(DefaultIoFilterChain.java:789)
>> at
>>
>> org.apache.mina.core.filterchain.DefaultIoFilterChain.access$1700(DefaultIoFilterChain.java:49)
>> at
>>
>> org.apache.mina.core.filterchain.DefaultIoFilterChain$EntryImpl$1.event(DefaultIoFilterChain.java:1164)
>> at
>>
>> org.apache.mina.core.filterchain.IoFilterAdapter.event(IoFilterAdapter.java:162)
>> at
>>
>> org.apache.mina.core.filterchain.DefaultIoFilterChain.callNextFilterEvent(DefaultIoFilterChain.java:789)
>> at
>>
>> org.apache.mina.core.filterchain.DefaultIoFilterChain.access$1700(DefaultIoFilterChain.java:49)
>> at
>>
>> org.apache.mina.core.filterchain.DefaultIoFilterChain$EntryImpl$1.event(DefaultIoFilterChain.java:1164)
>> at
>>
>> org.apache.mina.core.filterchain.IoFilterAdapter.event(IoFilterAdapter.java:162)
>> at
>>
>> org.apache.mina.core.filterchain.DefaultIoFilterChain.callNextFilterEvent(DefaultIoFilterChain.java:789)
>> at
>>
>> org.apache.mina.core.filterchain.DefaultIoFilterChain.access$1700(DefaultIoFilterChain.java:49)
>> at
>>
>> org.apache.mina.core.filterchain.DefaultIoFilterChain$EntryImpl$1.event(DefaultIoFilterChain.java:1164)
>> at
>>
>> org.apache.mina.core.filterchain.IoFilterAdapter.event(IoFilterAdapter.java:162)
>> at
>>
>> org.apache.mina.core.filterchain.DefaultIoFilterChain.callNextFilterEvent(DefaultIoFilterChain.java:789)
>> at
>>
>> org.apache.mina.core.filterchain.DefaultIoFilterChain.access$1700(DefaultIoFilterChain.java:49)
>> at
>>
>> org.apache.mina.core.filterchain.DefaultIoFilterChain$EntryImpl$1.event(DefaultIoFilterChain.java:1164)
>> at
>>
>> org.apache.mina.filter.ssl.SSLHandlerG0.finish_handshake(SSLHandlerG0.java:589)
>> - locked <0x7fc1611fb470> (a org.apache.mina.filter.ssl.SSLHandlerG0)
>> at
>>
>> org.apache.mina.filter.ssl.SSLHandlerG0.receive_loop(SSLHandlerG0.java:271)
>> at
>>
>> org.apache.mina.filter.ssl.SSLHandlerG0.receive_loop(SSLHandlerG0.

Re: Individual SSLFilter per connector

[Jonathan Valliere]

ssionOpened event, the StateMachine filter takes a
> lock (0x7fc1611faec8), then the IoHandler tries to write some data,
> which ends in the SslFilter.filterWrite method, which takes a lock
> (0x7fc1611fb470)
>
> Another thread is processing the messageReceived method, go through the
> SslFilter.messageReceived method which takes a lock
> (0x7fc1611fb470), the same as in the upper thread, then goes through
> the StateMachine.handle method, which takes a lock (0x7fc1611faec8,
> already seen upper).
>
> The IoHandler should not try to write something on the sessionOpened
> message IMO (until the session has been secured), but in any case, the
> double lock is clearly the root cause of the problem.
>
>
>
>
> On 17/02/2024 17:41, Emmanuel Lécharny wrote:
> > Hi Jonathan,
> >
> > regarding the queing, we did that in MINA 2.1:
> >
> >  public void filterWrite(NextFilter nextFilter, IoSession session,
> > WriteRequest writeRequest) throws SSLException {
> >  if (LOGGER.isDebugEnabled()) {
> >  LOGGER.debug("{}: Writing Message : {}",
> > getSessionInfo(session), writeRequest);
> >  }
> >
> >  boolean needsFlush = true;
> >  SslHandler sslHandler = getSslSessionHandler(session);
> >
> >  try {
> >  synchronized (sslHandler) {
> >  if (!isSslStarted(session)) {
> >  sslHandler.scheduleFilterWrite(nextFilter,
> > writeRequest);
> >  }
> > ...
> >
> > and
> >
> >  /* no qualifier */void scheduleFilterWrite(NextFilter nextFilter,
> > WriteRequest writeRequest) {
> >  filterWriteEventQueue.add(new IoFilterEvent(nextFilter,
> > IoEventType.WRITE, session, writeRequest));
> >  }
> >
> > Sobasically, of the session handshake has not been fully negociated,
> > then we enqueue the write request. Pretty much what you suggest to do
> > instead of using synchronized blocks around read and write.
> >
> > Otherwise, comments inline:
> >
> > On 17/02/2024 14:06, Jonathan Valliere wrote:
> >>   There is also some additional complexity supporting the scenario
> >> where two
> >> different threads are triggering “receive” events on the filter.
> >
> > You mean on the same session?
> >
> >>
> >> We either have to process ALL messages out of the payload, then AFTER
> >> send
> >> them to the downstream filters or we have to perform a kind of
> >> dual-locking
> >> mechanism to ensure that multiple receiver threads to not operate in
> >> parallel.  Either way, we have to guarantee a serial processing of
> >> incoming
> >> messages.
> >
> > Actually, this is not a problem unless someone puts an executor filter
> > *before the SslFilter in the chain. We process the incoming message in a
> > way a given session is associated with one single IoProcessor instance,
> > so all the messages for a given session should be processed sequentially
> > by this Io processor. If we don't have an excutor filter before the
> > Sslfilter, it should be safe.
> >
> >>
> >> The more I think about it the more concerned I am that we’re following a
> >> rabbit down a hole.
> >>
> >> @Kishore Mokkarala  have you tried NOT using
> >> the SSL
> >> and instead creating a dummy filter which simply added synchronized to
> >> the
> >> receive and write method?  I would be interested to know if that
> >> causes the
> >> problem also.  Put this dummy filter in the same place as the SSL
> >> would be.
> >
> > Another way to deal with the lock would be to check for the
> > SessioSnecured event in the IoHandler side:
> > - if the event has not be received, then don't send writes, but enqueue
> > them
> > - if the session has been secured, then you can write at will
> > - when teh sessionSecured even is received, unqueue the write messages.
> >
> > Somehow, that should replace what I think we should do in the SslFilter.
> >
> > But I don't think the burden should be put on the MINA library user...
> >
> >>
> >> On Feb 17, 2024 at 2:00:30 AM, Emmanuel Lécharny 
> >> wrote:
> >>
> >>> Hi Jonathan,
> >>>
> >>> there are two aspects to consider:
> >>>
> >>> - first the establishemnt of the secured session
> >>> - second the standard exchange of data when the session has been
> >>> se

Re: Individual SSLFilter per connector

[Jonathan Valliere]

 I’m trying to understand specifically, if its how the new SSL implements a
synchronized block on the receive and write IoFilter functions that is
causing this problem for you.

Before we go down various paths in trying to figure how to remove the
synchronized blocks, I would like to confirm that is the cause of this
problem and not something else.

public class SyncFilter extends IoFilterAdapter {

@Override
   synchronized public void messageReceived(NextFilter nextFilter,
IoSession session, Object message) throws Exception {
super.messageReceived(nextFilter, session, message);
}

@Override
synchronized public void filterWrite(NextFilter nextFilter,
IoSession session, WriteRequest writeRequest) throws Exception {
super.filterWrite(nextFilter, session, writeRequest);
}
}


If you can use the above in lieu of the SSL and run the tests to see if the
same problem occurs that will help pinpoint the issue.

On Feb 17, 2024 at 10:13:56 AM, Kishore Mokkarala 
wrote:

> My use case is only with SSLFilter. We want  secure communication between
> application and sniffer. We reverted to mina 2.0.25,we are not facing this
> issue now.
>
> I would like to know what is the difference between 2.0.25 and 2.2.1/
> 2.2.3.
>
> On Sat, 17 Feb, 2024, 6:37 pm Jonathan Valliere, 
> wrote:
>
> There is also some additional complexity supporting the scenario where two
>
> different threads are triggering “receive” events on the filter.
>
>
> We either have to process ALL messages out of the payload, then AFTER send
>
> them to the downstream filters or we have to perform a kind of dual-locking
>
> mechanism to ensure that multiple receiver threads to not operate in
>
> parallel.  Either way, we have to guarantee a serial processing of incoming
>
> messages.
>
>
> The more I think about it the more concerned I am that we’re following a
>
> rabbit down a hole.
>
>
> @Kishore Mokkarala  have you tried NOT using the
>
> SSL and instead creating a dummy filter which simply added synchronized to
>
> the receive and write method?  I would be interested to know if that causes
>
> the problem also.  Put this dummy filter in the same place as the SSL would
>
> be.
>
>
> On Feb 17, 2024 at 2:00:30 AM, Emmanuel Lécharny 
>
> wrote:
>
>
> > Hi Jonathan,
>
> >
>
> > there are two aspects to consider:
>
> >
>
> > - first the establishemnt of the secured session
>
> > - second the standard exchange of data when the session has been secured.
>
> >
>
> > In the first case, we should *never* have any write done by the
>
> > IoHandler, or those writes should be enqueued until the session has been
>
> > secured. Here, the upstream filters should not be aware that the session
>
> > has been secured or not (all in all, filters are supposed to be
>
> > independenct)
>
> >
>
> > In the second case, the SslFilter is responsible for handling the
>
> > encoding and decoding of the data, regardless of what the upper filters
>
> > are doing
>
> >
>
> > So there should not be any assumption made on what wcould or should do
>
> > the upstream filters (that's because anyone can add a filter, and we
>
> > should not force the implementors to take care of unerlying filters
>
> > constraints).
>
> >
>
> > So the best way to deal with writes while the Secured session
>
> > establishment is ongoing is to enqueue the writes until the session is
>
> > secured. It adds some complexity, but it's safe for the full stack
>
> > (reads are a different beast: we have to assume that we can only read
>
> > Handshake packets until the HS is completed)
>
> >
>
> > I guess it fits with what you have in mind.
>
> >
>
> >
>
> > On 17/02/2024 04:12, Jonathan Valliere wrote:
>
> >
>
> > I was thinking this over last weekend….
>
> >
>
> >
>
> > If I simply removed the synchronization then there COULD be several
>
> >
>
> > incoming data corruption problems unless the upstream filters/ processors
>
> >
>
> > MUST guarantee FIFO of the data stream.
>
> >
>
> >
>
> > What I would have todo is push OR copy the incoming buffer into a Queue
>
> >
>
> > then perform processing off the head of that Queue.  Essentially, do the
>
> >
>
> > same as the mEncodeQueue but for Decodes.  The lock would then only be
>
> > held
>
> >
>
> > when processing data off of that Decode queue.  Following this pattern, I
>
> >
>
> > could probably remove both the REA

Re: Individual SSLFilter per connector

[Jonathan Valliere]

 There is also some additional complexity supporting the scenario where two
different threads are triggering “receive” events on the filter.

We either have to process ALL messages out of the payload, then AFTER send
them to the downstream filters or we have to perform a kind of dual-locking
mechanism to ensure that multiple receiver threads to not operate in
parallel.  Either way, we have to guarantee a serial processing of incoming
messages.

The more I think about it the more concerned I am that we’re following a
rabbit down a hole.

@Kishore Mokkarala  have you tried NOT using the SSL
and instead creating a dummy filter which simply added synchronized to the
receive and write method?  I would be interested to know if that causes the
problem also.  Put this dummy filter in the same place as the SSL would be.

On Feb 17, 2024 at 2:00:30 AM, Emmanuel Lécharny 
wrote:

> Hi Jonathan,
>
> there are two aspects to consider:
>
> - first the establishemnt of the secured session
> - second the standard exchange of data when the session has been secured.
>
> In the first case, we should *never* have any write done by the
> IoHandler, or those writes should be enqueued until the session has been
> secured. Here, the upstream filters should not be aware that the session
> has been secured or not (all in all, filters are supposed to be
> independenct)
>
> In the second case, the SslFilter is responsible for handling the
> encoding and decoding of the data, regardless of what the upper filters
> are doing
>
> So there should not be any assumption made on what wcould or should do
> the upstream filters (that's because anyone can add a filter, and we
> should not force the implementors to take care of unerlying filters
> constraints).
>
> So the best way to deal with writes while the Secured session
> establishment is ongoing is to enqueue the writes until the session is
> secured. It adds some complexity, but it's safe for the full stack
> (reads are a different beast: we have to assume that we can only read
> Handshake packets until the HS is completed)
>
> I guess it fits with what you have in mind.
>
>
> On 17/02/2024 04:12, Jonathan Valliere wrote:
>
> I was thinking this over last weekend….
>
>
> If I simply removed the synchronization then there COULD be several
>
> incoming data corruption problems unless the upstream filters/ processors
>
> MUST guarantee FIFO of the data stream.
>
>
> What I would have todo is push OR copy the incoming buffer into a Queue
>
> then perform processing off the head of that Queue.  Essentially, do the
>
> same as the mEncodeQueue but for Decodes.  The lock would then only be held
>
> when processing data off of that Decode queue.  Following this pattern, I
>
> could probably remove both the READ and WRITE synchronization blocks.
>
>
> My understanding, for MINA, that the assumption is that it SHOULD be safe
>
> to hold on to any object passed into a Filter since object lifecycles
>
> SHOULD be controlled by the VM and not the User’s code.
>
>
> If we’re in agreement, I can make the change.
>
>
>
> On Feb 10, 2024 at 2:14:01 PM, Kishore Mokkarala 
>
> wrote:
>
>
> > Any time line for removing synchronization block in SSLFilter?
>
> >
>
> > On Sat, 10 Feb, 2024, 9:48 pm Emmanuel Lécharny, 
>
> > wrote:
>
> >
>
> > Netty is not Apache, but Eclipse.
>
> >
>
> >
>
> > We are discussing the error at the moment, trying to move away the
>
> >
>
> > SSLFilter synchronized block.
>
> >
>
> >
>
> > On 10/02/2024 08:10, Kishore Mokkarala wrote:
>
> >
>
> >> We had to revert mina version to 2.0.25 from 2.2.1 to make it work in
>
> >
>
> >> production and  trying for other alternatives like apache netty.
>
> >
>
> >>
>
> >
>
> >> On Fri, 9 Feb, 2024, 1:59 pm Emmanuel Lécharny, 
> >
>
> >> <mailto:elecha...@gmail.com>> wrote:
>
> >
>
> >>
>
> >
>
> >>  Hi Jonathan,
>
> >
>
> >>
>
> >
>
> >>  in this very case, I think there is a race condition when using the
>
> >
>
> >>  SSLFilter in conjonction with the StateMachine filter.
>
> >
>
> >>
>
> >
>
> >>  On 09/02/2024 05:33, Jonathan Valliere wrote:
>
> >
>
> >>   >   No, you should not have to create multiple instances.  The
>
> >
>
> >>  necessary
>
> >
>
> >>   > stateful data is saved to the Connection

Re: Individual SSLFilter per connector

[Jonathan Valliere]

I was thinking this over last weekend….

If I simply removed the synchronization then there COULD be several
incoming data corruption problems unless the upstream filters/ processors
MUST guarantee FIFO of the data stream.

What I would have todo is push OR copy the incoming buffer into a Queue
then perform processing off the head of that Queue.  Essentially, do the
same as the mEncodeQueue but for Decodes.  The lock would then only be held
when processing data off of that Decode queue.  Following this pattern, I
could probably remove both the READ and WRITE synchronization blocks.

My understanding, for MINA, that the assumption is that it SHOULD be safe
to hold on to any object passed into a Filter since object lifecycles
SHOULD be controlled by the VM and not the User’s code.

If we’re in agreement, I can make the change.


On Feb 10, 2024 at 2:14:01 PM, Kishore Mokkarala 
wrote:

> Any time line for removing synchronization block in SSLFilter?
>
> On Sat, 10 Feb, 2024, 9:48 pm Emmanuel Lécharny, 
> wrote:
>
> Netty is not Apache, but Eclipse.
>
>
> We are discussing the error at the moment, trying to move away the
>
> SSLFilter synchronized block.
>
>
> On 10/02/2024 08:10, Kishore Mokkarala wrote:
>
> > We had to revert mina version to 2.0.25 from 2.2.1 to make it work in
>
> > production and  trying for other alternatives like apache netty.
>
> >
>
> > On Fri, 9 Feb, 2024, 1:59 pm Emmanuel Lécharny, 
> > <mailto:elecha...@gmail.com>> wrote:
>
> >
>
> > Hi Jonathan,
>
> >
>
> > in this very case, I think there is a race condition when using the
>
> > SSLFilter in conjonction with the StateMachine filter.
>
> >
>
> > On 09/02/2024 05:33, Jonathan Valliere wrote:
>
> >  >   No, you should not have to create multiple instances.  The
>
> > necessary
>
> >  > stateful data is saved to the Connection.
>
> >  >
>
> >  >
>
> >  > On Feb 1, 2024 at 5:22:36 AM, Kishore Mokkarala
>
> > mailto:kishore@gmail.com>>
>
> >  > wrote:
>
> >  >
>
> >  >> Any response would be greatly appreciated.
>
> >  >> --
>
> >  >> M.V.S.Kishore
>
> >  >> 91-9886412814
>
> >  >>
>
> >  >>
>
> >  >> On Wed, 31 Jan 2024 at 22:17, Kishore Mokkarala
>
> > mailto:kishore@gmail.com>>
>
> >  >> wrote:
>
> >  >>
>
> >  >> Hi Emmanuel,
>
> >  >>
>
> >  >>
>
> >  >> Do we need to create a new instance of SSLFilter per tcp ip
>
> > connection or
>
> >  >>
>
> >  >> can we reuse it ?
>
> >  >>
>
> >  >>
>
> >  >> For example, do we need to repeat the same code for each tcp ip
>
> > connection
>
> >  >>
>
> >  >> ?
>
> >  >>
>
> >  >>
>
> >  >> *Approach 1:*
>
> >  >>
>
> >  >> for(int i=0;i< 500;i++)
>
> >  >>
>
> >  >> {
>
> >  >>
>
> >  >> NioSocketConnector connector = new NioSocketConnector();
>
> >  >>
>
> >  >> connector.getFilterChain().addLast("LoggingFilter",
>
> >  >>
>
> >  >> G10CaptureService.loggingFilter);
>
> >  >>
>
> >  >> connector.getFilterChain().addLast("codecFilter",
>
> >  >>
>
> >  >> G10CaptureService.probeCodecFilter);
>
> >  >>
>
> >  >> connector.getFilterChain().addLast("executorFilter",
>
> >  >>
>
> >  >> G10CaptureService.executorFilter);
>
> >  >>
>
> >  >> connector.getFilterChain().addLast("gpbMessageFilter",
>
> >  >>
>
> >  >> G10CaptureService.gpbMessageFilter);
>
> >  >>
>
> >  >> connector.getFilterChain().addLast("keepAliveFilter",
>
> >  >>
>
> >  >> G10CaptureService.keepAliveFilter);
>
> >  >>
>
> >  >> SslFilter sslFilter;
>
> >  >>
>
> >  >> try {
>
> >  >>
>
> >  >> SSLContext sslContext = 

Re: Individual SSLFilter per connector

[Jonathan Valliere]

 No, you should not have to create multiple instances.  The necessary
stateful data is saved to the Connection.


On Feb 1, 2024 at 5:22:36 AM, Kishore Mokkarala 
wrote:

> Any response would be greatly appreciated.
> --
> M.V.S.Kishore
> 91-9886412814
>
>
> On Wed, 31 Jan 2024 at 22:17, Kishore Mokkarala 
> wrote:
>
> Hi Emmanuel,
>
>
> Do we need to create a new instance of SSLFilter per tcp ip connection or
>
> can we reuse it ?
>
>
> For example, do we need to repeat the same code for each tcp ip connection
>
> ?
>
>
> *Approach 1:*
>
> for(int i=0;i< 500;i++)
>
> {
>
> NioSocketConnector connector = new NioSocketConnector();
>
> connector.getFilterChain().addLast("LoggingFilter",
>
> G10CaptureService.loggingFilter);
>
> connector.getFilterChain().addLast("codecFilter",
>
> G10CaptureService.probeCodecFilter);
>
> connector.getFilterChain().addLast("executorFilter",
>
> G10CaptureService.executorFilter);
>
> connector.getFilterChain().addLast("gpbMessageFilter",
>
> G10CaptureService.gpbMessageFilter);
>
> connector.getFilterChain().addLast("keepAliveFilter",
>
> G10CaptureService.keepAliveFilter);
>
> SslFilter sslFilter;
>
> try {
>
> SSLContext sslContext = TLSUtil.getSSLContext();
>
> sslFilter = new CustomSslFilter(sslContext);
>
> connector.getFilterChain().addFirst("sslFilter", sslFilter);
>
> } catch (Exception e) {
>
> e.printStackTrace();
>
> LOG.error("Exception during creating SSL context..." +
>
> XError.getStackTrace(e));
>
> }
>
> //io handler creation
>
> StateMachine stateMachine =
>
> StateMachineFactory.getInstance(IoHandlerTransition.class).create(
>
> G10MinaClient.CONNECTED, new G10MinaClient(processor));
>
>
> IoHandler ioHandler = new
>
> StateMachineProxyBuilder().setStateContextLookup(
>
> new IoSessionStateContextLookup(new StateContextFactory() {
>
> @Override
>
> public StateContext create() {
>
> final G10StateContext stateContext = new
>
> G10StateContext();
>
> stateContext.setStartedTime(new Date());
>
> LOG.info("G10StateContext initialized at:{}
>
> ",System.currentTimeMillis());
>
> return stateContext;
>
> }
>
> })).create(IoHandler.class, stateMachine);
>
> connector.setHandler(ioHandler);
>
> connector.connect(primaryAddress);
>
> connectFuture.awaitUninterruptibly();
>
> if (connectFuture.isConnected()) {
>
> IoSession session = connectFuture.getSession();
>
> // Do something with the session if needed
>
> } else {
>
> System.err.println("Connection failed for iteration: " + i);
>
> }
>
> }
>
>
> *Approach 2:*
>
> Reuse the generic connector implemented above for opening all TCP/IP
>
> connections.
>
> //just do the below for getting connections for example
>
> NioSocketConnector connector = new NioSocketConnector();
>
> //filter chain creation
>
> //add SSLFilter to filer chain
>
>
> for(int i=0;i< 500;i++)
>
> {
>
> ConnectFuture connectFuture = connector.connect(serverAddress);
>
> connectFuture.awaitUninterruptibly();
>
> if (connectFuture.isConnected()) {
>
> IoSession session = connectFuture.getSession();
>
> // Do something with the session if needed
>
> } else {
>
> System.err.println("Connection failed for iteration: " + i);
>
> }
>
> }
>
>
> Which approach is better ?
>
>
> Regards,
>
> --
>
> M.V.S.Kishore
>
> 91-9886412814
>
>
>

Re: Apache MINA Mastodon account is on!

[Jonathan Valliere]



On Sun, Jul 2, 2023 at 6:32 PM Emmanuel Lécharny 
wrote:

> Hi!
>
> we just have created a Mastodon account to inform the world abouut the
> Apache MINA project evolution.
>
> Feel free to follow @apachem...@fosstodon.org  !
> --
> *Emmanuel Lécharny*
>
> -
> To unsubscribe, e-mail: users-unsubscr...@mina.apache.org
> For additional commands, e-mail: users-h...@mina.apache.org
>
>

Re: Apache mina 2.2.1 thread blocking issue.

[Jonathan Valliere]

Please make a Jira. I also need an example to run which reproduces the
error.

On Tue, Apr 25, 2023 at 6:35 AM Kishore Mokkarala 
wrote:

> Hi,
> I have migrated from 2.0.21 to 2.023 for solving CVE, i have seen thread
> blocking issue, So used latest mina verstion 2.2.1 but still threads were
> blocked here is the sample code.Thread hungs at awaitUninterruptibly.  Once
> this issue comes  in sub sequest launches nothing will work all threads
> will be blocked forever,i have to restart the process to make it work. For
> single thread working fine,if i start 50-100 threads this thread blocking
> issue will surface.
>
> *Here is the sample thread dump:*
> "pool-118-thread-6" #508 prio=5 os_prio=0 cpu=345.84ms elapsed=*1929.48s*
> tid=0x7ec6fc001800 nid=0x4b5d4 in Object.wait()  [0x7ec7792d4000]
>java.lang.Thread.State: TIMED_WAITING (on object monitor)
> at java.lang.Object.wait(java.base@11.0.14.1/Native Method)
> - waiting on 
> at
>
> org.apache.mina.core.future.DefaultIoFuture.await0(DefaultIoFuture.java:218)
> - waiting to re-lock in wait() <0x7ed90d1b2c40> (a
> org.apache.mina.core.polling.AbstractPollingIoConnector$ConnectionRequest)
> at
>
> org.apache.mina.core.future.DefaultIoFuture.awaitUninterruptibly(DefaultIoFuture.java:148)
> at
>
> org.apache.mina.core.future.DefaultConnectFuture.awaitUninterruptibly(DefaultConnectFuture.java:149)
> at
>
> com.netscout.nsaapp.geo.g10Plugin.g10.service.G10CaptureService.startRecordCapture(G10CaptureService.java:622)
> at
>
> com.netscout.nsaapp.geo.g10Plugin.geoblade.service.GeoBladeCaptureService.startRecordCapture(GeoBladeCaptureService.java:67)
> at
>
> com.netscout.nsaapp.geo.g10Plugin.g10.processor.G10PluginCaptureProcessor.processGatewaySrQueryResponseSuccess(G10PluginCaptureProcessor.java:2156)
> at
>
> com.netscout.nsaapp.geo.minaG10Proto.server.G10MinaClient.doHandleGatewaySrQueryResponse(G10MinaClient.java:283)
> at
>
> com.netscout.nsaapp.geo.minaG10Proto.server.G10MinaClient.handleGatewaySrQueryResponse(G10MinaClient.java:268)
> at jdk.internal.reflect.GeneratedMethodAccessor201.invoke(Unknown
> Source)
> at jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(
> java.base@11.0.14.1/DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(
> java.base@11.0.14.1/Method.java:566)
> at
>
> org.apache.mina.statemachine.transition.MethodTransition.invokeMethod(MethodTransition.java:281)
> at
>
> org.apache.mina.statemachine.transition.MethodTransition.doExecute(MethodTransition.java:232)
> at
>
> org.apache.mina.statemachine.transition.AbstractTransition.execute(AbstractTransition.java:100)
> at
> org.apache.mina.statemachine.StateMachine.handle(StateMachine.java:183)
> at
> org.apache.mina.statemachine.StateMachine.handle(StateMachine.java:273)
> at
>
> org.apache.mina.statemachine.StateMachine.processEvents(StateMachine.java:170)
> at
> org.apache.mina.statemachine.StateMachine.handle(StateMachine.java:158)
> - locked <0x7ed92a951cd8> (a
> com.netscout.nsaapp.geo.minaG10Proto.server.G10StateContext)
> at
>
> org.apache.mina.statemachine.StateMachineProxyBuilder$MethodInvocationHandler.invoke(StateMachineProxyBuilder.java:261)
> at com.sun.proxy.$Proxy85.messageReceived(Unknown Source)
> at
>
> org.apache.mina.core.filterchain.DefaultIoFilterChain$TailFilter.messageReceived(DefaultIoFilterChain.java:1015)
> at
>
> org.apache.mina.core.filterchain.DefaultIoFilterChain.callNextMessageReceived(DefaultIoFilterChain.java:650)
> at
>
> org.apache.mina.core.filterchain.DefaultIoFilterChain.access$1300(DefaultIoFilterChain.java:49)
> at
>
> org.apache.mina.core.filterchain.DefaultIoFilterChain$EntryImpl$1.messageReceived(DefaultIoFilterChain.java:1128)
> at
>
> org.apache.mina.filter.keepalive.KeepAliveFilter.messageReceived(KeepAliveFilter.java:414)
> at
>
> org.apache.mina.core.filterchain.DefaultIoFilterChain.callNextMessageReceived(DefaultIoFilterChain.java:650)
> at
>
> org.apache.mina.core.filterchain.DefaultIoFilterChain.access$1300(DefaultIoFilterChain.java:49)
> at
>
> org.apache.mina.core.filterchain.DefaultIoFilterChain$EntryImpl$1.messageReceived(DefaultIoFilterChain.java:1128)
> at
>
> com.netscout.nsaapp.geo.minaG10Proto.server.G10GPBMessageIoFilter.messageReceived(G10GPBMessageIoFilter.java:100)
> at
>
> org.apache.mina.core.filterchain.DefaultIoFilterChain.callNextMessageReceived(DefaultIoFilterChain.java:650)
> at
>
> org.apache.mina.core.filterchain.DefaultIoFilterChain.access$1300(DefaultIoFilterChain.java:49)
> at
>
> org.apache.mina.core.filterchain.DefaultIoFilterChain$EntryImpl$1.messageReceived(DefaultIoFilterChain.java:1128)
> at
> 

Re: Fwd: migration from apache mina 2.0.21 to 2.0.23 issue

[Jonathan Valliere]

Cool. That was easy.

On Mon, Apr 17, 2023 at 11:05 AM Kishore Mokkarala 
wrote:

> Thank you all for the help.Here is my SSL implementation for making it work
> with 2.2.1 for passing PEER ADDRESS (SNI host name) in the SSL engine.
>
> public class CustomSslFilter {
> public CustomSslFilter(SSLContext sslContext) {
> super(sslContext);
> }
> //Override CreateEngine
>  protected SSLEngine createEngine(IoSession session, InetSocketAddress
> addr) {
> //Add your SNI host name and port in the IOSession
> SNIHostNames   = (String)session.getAttribute( SNIHostNames );
>   PortNumber =   (String)session.getAttribute(  PortNumber  );
> InetSocketAddress peer =
> InetSocketAddress.createUnresolved(SNIHostNames,PortNumber);
>SSLEngine sslEngine = (addr != null) ?
> sslContext.createSSLEngine(peer.getHostString(), peer.getPort())
>: sslContext.createSSLEngine();
>
>// Always start with WANT, which will be squashed by NEED if NEED is
> true.
>// Actually, it makes not a lot of sense to select NEED and WANT.
> NEED >> WANT...
>if (wantClientAuth) {
>sslEngine.setWantClientAuth(true);
>}
>
>if (needClientAuth) {
>sslEngine.setNeedClientAuth(true);
>}
>
>if (enabledCipherSuites != null) {
>sslEngine.setEnabledCipherSuites(enabledCipherSuites);
>}
>
>if (enabledProtocols != null) {
>sslEngine.setEnabledProtocols(enabledProtocols);
>}
>
>sslEngine.setUseClientMode(!session.isServer());
>
>return sslEngine;
>}
> }
>
>
> IoSessionInitializer initializer = new
> IoSessionInitializer() {
>
> @Override
> public void initializeSession(IoSession session, ConnectFuture
> future) {
>
> session.setAttribute( SNIHostNames , "example.com");
> session.setAttribute( PortNumber  , 8443);
> }
> };
>
> try {
> NioSocketConnector connector = getConnector();
> ioSession = connector.connect(address,
> initializer).awaitUninterruptibly().getSession();
> } catch (RuntimeIoException eio) {
> initializationException = eio;
> }
>
> --
> M.V.S.Kishore
> 91-9886412814
>
>
> On Fri, 14 Apr 2023 at 18:43, Jonathan Valliere 
> wrote:
>
> > Looking at the code for your existing filter it appears like you’re just
> > trying to create the SSLEngine so it can be reused for subsequent
> > connections by passing in the IP address and Port?
> >
> > This is already a feature in the new filter.
> >
> >
> https://github.com/apache/mina/blob/a8dc2c56ec43ac67d64d0dab39a65958579debbb/mina-core/src/main/java/org/apache/mina/filter/ssl/SslFilter.java#L281
> >
> > If you want to perform any customization during the SSL Engine setup,
> just
> > override createEngine
> >
> >
> > On Fri, Apr 14, 2023 at 7:23 AM Kishore Mokkarala  >
> > wrote:
> >
> > > Currently we are using the following custom SSL filter for passing SNI
> > host
> > > name. For doing this we are using PEER_ADDRESS.
> > > This was available in apache mina 2.0.21 SslHandler.java,but this
> > attribute
> > > is not available in 2.2.10.
> > > This PEER_ADDRESS is *eid.17.cid.0* different from the actual IP
> address
> > to
> > > which it connects ,but this information is needed for the destination
> > > server.
> > >
> > > *Existing implementation : *
> > >
> > > SslFilter sslFilter;
> > > try {
> > > SSLContext sslContext = javax.net.ssl.SSLContext.getDefault();
> > > * sslFilter = new CustomSslFilter(sslContext); //passing *
> *PEER_ADDRESS
> > > in overridden onPreAdd*.
> > > sslFilter.setUseClientMode(true);
> > > connector.getFilterChain().addFirst("sslFilter", sslFilter);
> > > } catch (Exception e) {
> > > e.printStackTrace();
> > > LOG.error("Exception during creating SSL context..." +
> > > XError.getStackTrace(e));
> > > }
> > > connector.setHandler(ioHandler);
> > >
> > > *CustomSslFilter.java:*
> > >
> > > public class CustomSslFilter extends SslFilter
> > > {
> > >
> > > public CustomSslFilter(SSLContext sslContext) {
> > > super(sslContext, true);
> > > }
> > >
> > > @Override
> > > public void onPreAdd(IoFilterChain parent, String name,
> > >  

Re: Fwd: migration from apache mina 2.0.21 to 2.0.23 issue

[Jonathan Valliere]

Looking at the code for your existing filter it appears like you’re just
trying to create the SSLEngine so it can be reused for subsequent
connections by passing in the IP address and Port?

This is already a feature in the new filter.
https://github.com/apache/mina/blob/a8dc2c56ec43ac67d64d0dab39a65958579debbb/mina-core/src/main/java/org/apache/mina/filter/ssl/SslFilter.java#L281

If you want to perform any customization during the SSL Engine setup, just
override createEngine


On Fri, Apr 14, 2023 at 7:23 AM Kishore Mokkarala 
wrote:

> Currently we are using the following custom SSL filter for passing SNI host
> name. For doing this we are using PEER_ADDRESS.
> This was available in apache mina 2.0.21 SslHandler.java,but this attribute
> is not available in 2.2.10.
> This PEER_ADDRESS is *eid.17.cid.0* different from the actual IP address to
> which it connects ,but this information is needed for the destination
> server.
>
> *Existing implementation : *
>
> SslFilter sslFilter;
> try {
> SSLContext sslContext = javax.net.ssl.SSLContext.getDefault();
> * sslFilter = new CustomSslFilter(sslContext); //passing * *PEER_ADDRESS
> in overridden onPreAdd*.
> sslFilter.setUseClientMode(true);
> connector.getFilterChain().addFirst("sslFilter", sslFilter);
> } catch (Exception e) {
> e.printStackTrace();
> LOG.error("Exception during creating SSL context..." +
> XError.getStackTrace(e));
> }
> connector.setHandler(ioHandler);
>
> *CustomSslFilter.java:*
>
> public class CustomSslFilter extends SslFilter
> {
>
> public CustomSslFilter(SSLContext sslContext) {
> super(sslContext, true);
> }
>
> @Override
> public void onPreAdd(IoFilterChain parent, String name,
> NextFilter nextFilter) throws SSLException {
> // Check that we don't have a SSL filter already present in the
> chain
> if (parent.contains(SslFilter.class)) {
> String msg = "Only one SSL filter is permitted in a chain.";
> LOGGER.error(msg);
> throw new IllegalStateException(msg);
> }
> IoSession session = parent.getSession();
> Provider provider =
> (Provider)session.getAttribute(G10MinaClient.PROVIDER_KEY);
> InetSocketAddress probeAddress =
> InetSocketAddress.createUnresolved(
> *eid.17.cid.0*,Integer.parseInt(provider.getProbe().getPortNumber()));
> session.setAttribute(PEER_ADDRESS, probeAddress);
> super.onPreAdd(parent, name, nextFilter);
> }
> }
>
> We are planning to migrate from 2.0.21 to 2.2.10. Here is the changes I did
> but it is not working.Please do the needful.
> *Question:*
> How to pass this sni host name for creating SSLEngine?
>
> *Here is the new implementation changed as per new Mina 2.2.10 API:*
> try{
> sslContext = javax.net.ssl.SSLContext.getDefault();
> SNIServerName sniHostName = new SNIHostName("*eid.17.cid.0*");
> List sniHostNames = new ArrayList<>();
> sniHostNames.add(sniHostName);
> SSLParameters sslParams = sslContext.getDefaultSSLParameters();
> sslParams.setServerNames(sniHostNames);
> sslFilter = new SslFilter(sslContext);
> //sslFilter.setUseClientMode(true); //This is not required in 2.2.1 hence
> commented.
> connector.getFilterChain().addFirst("sslFilter", sslFilter);
> } catch (Exception e) {
> e.printStackTrace();
> LOG.error("Exception during creating SSL context..." +
> XError.getStackTrace(e));
> }
> connector.setHandler(ioHandler);
>
> Here is the Apache mina 2.0.21 with PEER_ADDRESS in SslHandler.java code :
>
>  /* no qualifier */void init() throws SSLException {
> if (sslEngine != null) {
> // We already have a SSL engine created, no need to create a
> new one
> return;
> }
> if (LOGGER.isDebugEnabled()) {
> LOGGER.debug("{} Initializing the SSL Handler",
> sslFilter.getSessionInfo(session));
> }
> InetSocketAddress peer = (InetSocketAddress)
> session.getAttribute(SslFilter.PEER_ADDRESS);
> // Create the SSL engine here
> if (peer == null) {
> sslEngine = sslFilter.sslContext.createSSLEngine();
> } else {
> sslEngine =
> sslFilter.sslContext.createSSLEngine(peer.getHostName(), peer.getPort());
> }
> // Initialize the engine in client mode if necessary
> sslEngine.setUseClientMode(sslFilter.isUseClientMode());
>
>
> Regards,
> --
> M.V.S.Kishore
> 91-9886412814
>
>
> On Wed, 12 Apr 2023 at 23:08, Emmanuel Lécharny 
> wrote:
>
> > Hi,
> >
> > On 12/04/2023 18:00, Kishore Mokkarala wrote:
> > > Thanks  Emmanuel for the quick response.I have few more questions on
> the
> > > upgrade.Please do the needful.
> > > If i want to upgrade from Apache mina 2.0.21 to mina 2.2.1 what all
> steps
> > > do i need to follow ?
> >
> > There are two pages that explains the diffence between 2.0 and 2.1, and
> > 2. and 2.2:
> > * https://mina.apache.org/mina-project/2.1-vs-2.0.html
> > * 

Re: Upgrading from mina-core ( 1.7.1 ) to Mina-core 2.2.1

[Jonathan Valliere]

We also likely broke some things between 2.1 and 2.2 (some debate about
whether we should be at 3.0 despite it already exists in a very old branch)

On Sun, Apr 9, 2023 at 9:54 AM Isaac M  wrote:

> A lot has changed since the old version and likely refactored to a
> different package. See ProxyTcpSessionConfig.java
> <
> https://github.com/apache/mina/blob/trunk/core/src/main/java/org/apache/mina/transport/tcp/ProxyTcpSessionConfig.java
> >
>
> On Sat, Apr 8, 2023 at 11:48 AM Chandan Singh <
> mailbox.chandansi...@gmail.com> wrote:
>
> > Hi All ,
> >
> > I am looking to upgrade  from mina-core  1.7.1  version to  2.2.1  but
> > getting compilation issue stating packages not found as shown below ,
> > Please let me know how to resolve these
> >
> > *package org.apache.mina.common does not exist*
> >
> >
> >
> > * cannot find symbol[ERROR]   symbol:   class SocketSessionConfig[ERROR]
> > location: package org.apache.mina.transport.socket.nio*
> >
> >
> > *Regards*
> > *Chandan*
> >
>

Re: Re: How to access UDP Headers in IOHandlerAdapter

[Jonathan Valliere]

If it’s showing the wrong address then getting the more raw data won’t get
you a different address.

Likely the kubernetes mid plane is doing a reverse proxy or something for
the traffic which is obfuscating the addresses.

On Mon, Jan 2, 2023 at 9:27 PM Bobby R. Harsono
 wrote:

> Remote Address
>
> I know i can get it via session.getRemoteAddress(), but since i deployed
> using kube8, for some reason, that method returns ip address of the
> worker machine, not actual source machine, this is probably known things
> / issue and we can't do anyhing currently; I was thinking maybe i can
> solve this by getting remote address from headers directly,
>
> The remote address will be used for heartbeat / keep alive implementation
>
>
>
> On 2023/01/02 17:17:48 Jonathan Valliere wrote:
>  > What in the header are you looking for?
>  >
>  > On Mon, Jan 2, 2023 at 12:00 PM Bobby R. Harsono
>  >  wrote:
>  >
>  > > Hello,
>  > >
>  > > How can i access UDP Headers details in IOHandlerAdapter? Since the
>  > > documentations doesn't mention about this neither i can found any
>  > > references out there,
>  > >
>  > > I need this to implement keep-alive or heartbeat mechanism for my UDP
>  > > Server; Also, are there references i can read about keep-alive
> filter or
>  > > should i implement this mechanism from scratch
>  > >
>  > > Thank you
>  > >
>  > >
>  > > --
>  > > ---
>  > > Bobby R. Harsono
>  > > Software Developer
>  > > Tricada Intronik - Bandung
>  > >
>  > >
>  > > -
>  > > To unsubscribe, e-mail: users-unsubscr...@mina.apache.org
>  > > For additional commands, e-mail: users-h...@mina.apache.org
>  > >
>  > >
>  >
>
> --
> ---
> Bobby R. Harsono
> Software Developer
> Tricada Intronik - Bandung
>
>
> -
> To unsubscribe, e-mail: users-unsubscr...@mina.apache.org
> For additional commands, e-mail: users-h...@mina.apache.org
>
>

Re: How to access UDP Headers in IOHandlerAdapter

[Jonathan Valliere]

What in the header are you looking for?

On Mon, Jan 2, 2023 at 12:00 PM Bobby R. Harsono
 wrote:

> Hello,
>
> How can i access UDP Headers details in IOHandlerAdapter? Since the
> documentations doesn't mention about this neither i can found any
> references out there,
>
> I need this to implement keep-alive or heartbeat mechanism for my UDP
> Server; Also, are there references i can read about keep-alive filter or
> should i implement this mechanism from scratch
>
> Thank you
>
>
> --
> ---
> Bobby R. Harsono
> Software Developer
> Tricada Intronik - Bandung
>
>
> -
> To unsubscribe, e-mail: users-unsubscr...@mina.apache.org
> For additional commands, e-mail: users-h...@mina.apache.org
>
>

Re: New deadlock potential in MINA 2.2's SSLHandler

[Jonathan Valliere]

Check my graphic in the previous email.

On Mon, Dec 12, 2022 at 3:53 AM Emmanuel Lécharny 
wrote:

> Hi Guus,
>
> there is something missing in the stacck trace: the two threads are
> blocked on different monitors:
> * SSLHandlerG0@99f49e2 owned by socket_c2s_ssl-thread-2
> * StreamManager@7e024771 owned by TaskEngine-pool-3
>
> they are not related, they are not blocking each one other.
>
> Can you provide the full stack trace ?
>
> Thanks !
>
>
> On 09/12/2022 10:02, Guus der Kinderen wrote:
> > Hello,
> >
> > I wonder if MINA 2.2.1 introduces a new potential for deadlock in its
> > SSLHandler implementation. Did something change there, as compared to
> MINA
> > 2.1.3?
> >
> > Since this upgrade, we consistently run into a thread deadlock. One of
> the
> > threads that is deadlocking is sending data to the peer, while the other
> is
> > processing data from the peer. The deadlock involves a MINA-based lock
> (in
> > SSLHandler), and a lock in our code (StreamManager), that is responsible
> > for recording what packets that are exchanged have been acknowledged by
> the
> > peer. The latter did not change, so I'm suspecting a change in
> SSLHandler.
> >
> > The stack traces of two deadlocked threads are added to this email.
> >
> > Kind regards,
> >
> >Guus
> >
> >
> > "TaskEngine-pool-3" #40 daemon prio=5 waiting on lock
> > java.lang.Thread.State: BLOCKED
> >- blocked on org.apache.mina.filter.ssl.SSLHandlerG0@99f49e2 (owned
> by
> > socket_c2s_ssl-thread-2 id=78)
> >at
> org.apache.mina.filter.ssl.SSLHandlerG0.write(SSLHandlerG0.java:312)
> >at
> org.apache.mina.filter.ssl.SslFilter.filterWrite(SslFilter.java:380)
> >at
> >
> org.apache.mina.core.filterchain.DefaultIoFilterChain.callPreviousFilterWrite(DefaultIoFilterChain.java:753)
> >at
> >
> org.apache.mina.core.filterchain.DefaultIoFilterChain.access$1500(DefaultIoFilterChain.java:49)
> >at
> >
> org.apache.mina.core.filterchain.DefaultIoFilterChain$EntryImpl$1.filterWrite(DefaultIoFilterChain.java:1146)
> >at
> >
> org.apache.mina.filter.codec.ProtocolCodecFilter.filterWrite(ProtocolCodecFilter.java:301)
> >at
> >
> org.apache.mina.core.filterchain.DefaultIoFilterChain.callPreviousFilterWrite(DefaultIoFilterChain.java:753)
> >at
> >
> org.apache.mina.core.filterchain.DefaultIoFilterChain.access$1500(DefaultIoFilterChain.java:49)
> >at
> >
> org.apache.mina.core.filterchain.DefaultIoFilterChain$EntryImpl$1.filterWrite(DefaultIoFilterChain.java:1146)
> >at
> >
> org.apache.mina.core.filterchain.IoFilterAdapter.filterWrite(IoFilterAdapter.java:138)
> >at
> > org.jivesoftware.openfire.net
> .StalledSessionsFilter.filterWrite(StalledSessionsFilter.java:61)
> >at
> >
> org.apache.mina.core.filterchain.DefaultIoFilterChain.callPreviousFilterWrite(DefaultIoFilterChain.java:753)
> >at
> >
> org.apache.mina.core.filterchain.DefaultIoFilterChain.access$1500(DefaultIoFilterChain.java:49)
> >at
> >
> org.apache.mina.core.filterchain.DefaultIoFilterChain$EntryImpl$1.filterWrite(DefaultIoFilterChain.java:1146)
> >at
> >
> org.apache.mina.core.filterchain.IoFilterAdapter.filterWrite(IoFilterAdapter.java:138)
> >at
> >
> org.apache.mina.core.filterchain.DefaultIoFilterChain.callPreviousFilterWrite(DefaultIoFilterChain.java:753)
> >at
> >
> org.apache.mina.core.filterchain.DefaultIoFilterChain.fireFilterWrite(DefaultIoFilterChain.java:746)
> >at
> >
> org.apache.mina.core.session.AbstractIoSession.write(AbstractIoSession.java:575)
> >at
> >
> org.apache.mina.core.session.AbstractIoSession.write(AbstractIoSession.java:520)
> >at
> >
> org.jivesoftware.openfire.nio.NIOConnection.deliver(NIOConnection.java:349)
> >at
> >
> org.jivesoftware.openfire.session.LocalClientSession.deliver(LocalClientSession.java:928)
> >- locked
> org.jivesoftware.openfire.streammanagement.StreamManager@7e024771
> >at
> >
> org.jivesoftware.openfire.session.LocalSession.process(LocalSession.java:407)
> >at
> >
> org.jivesoftware.openfire.spi.RoutingTableImpl.routeToLocalDomain(RoutingTableImpl.java:439)
> >at
> >
> org.jivesoftware.openfire.spi.RoutingTableImpl.routePacket(RoutingTableImpl.java:350)
> >at org.jivesoftware.openfire.IQRouter.handle(IQRouter.java:426)
> >at org.jivesoftware.openfire.IQRouter.route(IQRouter.java:106)
> >at
> >
> org.jivesoftware.openfire.spi.PacketRouterImpl.route(PacketRouterImpl.java:74)
> >at
> >
> org.jivesoftware.openfire.pubsub.PubSubEngine.publishItemsToNode(PubSubEngine.java:428)
> >at
> > org.jivesoftware.openfire.pubsub.PubSubEngine$1.run(PubSubEngine.java:96)
> >at java.base@11.0.17
> > /java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
> >at java.base@11.0.17
> > /java.util.concurrent.FutureTask.run(FutureTask.java:264)
> >at java.base@11.0.17
> >
> /java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
> >at java.base@11.0.17
> >
> 

Re: SSL Issue with Mina NioConnector

[Jonathan Valliere]

Please update to Mina 2.2

On Thu, Nov 10, 2022 at 1:41 AM Nitin Phuria
 wrote:

> Dear All,
>
>
>
> We are trying to use Mina library to connect to server which is running the
> on SSL TLSv1.2. We are using JDK8u341 on client side with Mina 2.1.6 core
> library.
>
>
>
> Client Log
>
> =
>
> NioProcessor-2, called closeInbound()
>
> NioProcessor-2, fatal error: 80: Inbound closed before receiving peer's
> close_notify: possible truncation attack?
>
> javax.net.ssl.SSLException: Inbound closed before receiving peer's
> close_notify: possible truncation attack?
>
> %% Invalidated:  [Session-1, TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384]
>
> NioProcessor-2, SEND TLSv1.2 ALERT:  fatal, description = internal_error
>
> NioProcessor-2, Exception sending alert: java.io.IOException: writer side
> was already closed.
>
> NioProcessor-2, called closeOutbound()
>
> NioProcessor-2, closeOutboundInternal()
>
>
>
> We are getting the above mentioned error.  Any other info required then let
> me know.
>
>
>
> Thanks And Regards,
>
> Nitin Phuria
>
>
>
> Confidentiality Disclaimer: "The information contained in this electronic
> message (email) and any attachments to this email are intended for the
> exclusive use of the addressee(s) and access to this email by anyone else
> is
> unauthorized. The email may contain proprietary, confidential or privileged
> information or information relating to Integra Group. If you are not the
> intended recipient, please notify the sender by telephone, fax, or return
> email and delete this communication and any attachments thereto,
> immediately
> from your computer. Any dissemination, distribution, or copying of this
> communication and the attachments thereto (in whole or part), in any
> manner,
> is strictly prohibited and actionable at law. The recipient acknowledges
> that emails are susceptible to alteration and their integrity cannot be
> guaranteed and that Company does not guarantee that any e-mail is
> virus-free
> and accept no liability for any damage caused by any virus transmitted by
> this email."
>
>
>
>
> --
> **
>
> ***
> *
> *Confidentiality Disclaimer**: "The information contained in this
> electronic message
> (email) and any attachments to this email are intended
> for the exclusive use of
> the addressee(s) and access to this email by
> anyone else is unauthorized. The
> email may contain proprietary,
> confidential or privileged information or
> information relating to Integra
> Group. If you are not the intended recipient,
> please notify the sender by
> telephone, fax, or return email and delete this
> communication and any
> attachments thereto, immediately from your computer. Any
> dissemination,
> distribution, or copying of this communication and the
> attachments thereto
> (in whole or part), in any manner, is strictly prohibited
> and actionable at
> law. The recipient acknowledges that emails are susceptible
> to alteration
> and their integrity cannot be guaranteed and that Company does
> not
> guarantee that any e-mail is virus-free and accept no liability for any
> damage caused by any virus transmitted by this email."*
>
>

Re: Does MINA support TCP Option 28 for forwarded IP?

[Jonathan Valliere]

TCP_SAVED_SYN = 28 is the 28 option for the TCP level in Linux
SO_PEERNAME probably does the same thing as getpeername
<https://man7.org/linux/man-pages/man2/getpeername.2.html>

I'm at a loss to understand how to read that Host Identity option from TCP
in C

On Fri, Sep 17, 2021 at 11:21 AM Jonathan Valliere 
wrote:

> I see SO_PEERNAME = 28 but can't find any documentation of how to use it.
>
> Anyway, Java does not support this option.  Therefore, MINA does not.
>
>
>
> On Fri, Sep 17, 2021 at 10:55 AM Marc Boorshtein 
> wrote:
>
>>
>> On Fri, Sep 17, 2021 at 10:44 AM Jonathan Valliere 
>> wrote:
>>
>>> That has to be implemented at the kernel.
>>>
>>>>
>>>>
>> like, in linux?  i think its there already.  There's
>> https://www.ebower.com/w/index.php/Client_IP_Visibility that shows this
>> being done manually using tcpdump.  Is this data available from IoSession
>> or at all to MINA?
>>
>> Thanks
>> Marc
>>
>

Re: Does MINA support TCP Option 28 for forwarded IP?

[Jonathan Valliere]

I see SO_PEERNAME = 28 but can't find any documentation of how to use it.

Anyway, Java does not support this option.  Therefore, MINA does not.



On Fri, Sep 17, 2021 at 10:55 AM Marc Boorshtein 
wrote:

>
> On Fri, Sep 17, 2021 at 10:44 AM Jonathan Valliere 
> wrote:
>
>> That has to be implemented at the kernel.
>>
>>>
>>>
> like, in linux?  i think its there already.  There's
> https://www.ebower.com/w/index.php/Client_IP_Visibility that shows this
> being done manually using tcpdump.  Is this data available from IoSession
> or at all to MINA?
>
> Thanks
> Marc
>

Re: Does MINA support TCP Option 28 for forwarded IP?

[Jonathan Valliere]

That has to be implemented at the kernel.

On Fri, Sep 17, 2021 at 10:05 AM Marc Boorshtein 
wrote:

> I'm capturing the remote IP from the IoSession, but I've been asked to
> capture an IP address using TCP option 28 (
> https://datatracker.ietf.org/doc/rfc7974/).  Is this something that's
> possible in MINA?
>
> Thanks
> Marc
>

MINA SSL improvements

[Jonathan Valliere]

Everybody,

Last week I redesigned the SSL system in MINA to fix DIRMINA-1132 and other
TLSv1.3 related issues with the old code.  It's currently referred to as
SSL2, however it will eventually replace the original
org.apache.mina.filter.ssl.* package (probably in MINA 2.2.X).

The new version, while being substantially easier to understand, implements
write streaming.  With SSL write streaming, you no longer have to encode
the entire message all at once.  Large messages can be encoded and written
to the socket in chunks as needed.  I also plan on adding support for
writing files.

I would appreciate it if some of you can pull the current development
version from
https://gitbox.apache.org/repos/asf?p=mina.git;a=shortlog;h=refs/heads/bugfix/DIRMINA1132
and try your applications with it.

Neither Emmanual nor myself can get Java to handshake TLSv1.3 without
handshaking TLSv1.2 first.  I don't think this has anything to do with my
code but any attempts to check this is greatly appreciated.

Regards,
Jonathan

Re: delay in session.write and messageSent event for same message

[Jonathan Valliere]

All writes to the actual OS socket are delayed by reading from other
sockets attached to the same IoProcessor.

Here is an example:

   1. the event poller wakes up and has events for 20 sockets.
   2. Socket A - one of those events is a read, your socket reads and
   processes the data
   3. Socket A - you write to IoSession.write
   4. Socket A - the WriteRequest is copied to the session write queue
   5. Socket A - the IoSession subscribes to the write event from the poller
   6. the remaining events for the 19 other sockets are with events are
   processed
   7. the IoProcessor selects new events from the poller
   8. 20 new events are returned
   9. assuming Socket A is not backlogged, it should appear in the event
   list as writable but the order is random
   10. all 20 events must be processed until Socket A is found and the
   enqueued WriteRequest is written to the underlying socket


On Mon, Jul 5, 2021 at 8:57 AM Nitin Phuria  wrote:

> Dear Jonathan,
>
>
>
> The issue is not persistent and when it comes it is not
> for a moment. When it comes the issue will be for 2-3 hours and we see the
> time gap only in the session.write and messageSent event and that to with
> System B only.  If at all it has to be linked with any memory management it
> should not affect only this part alone.  I need some info on how I can get
> some internal details to find out why their is delay happening. Is there
> way to print some internal queue details and when MINA framework will be
> writting to actual TCP/IP etc..
>
>
>
> But to answer your question we did the heap memory analysis and we didn't
> find anything in it.
>
>
>
> Message written to System B session (Session.write method)
>
> 2021-06-30 10:57:10,427
>
> Message sent to System B (messageSent event)
>
> 2021-06-30 10:57:21,749
>
>
>
> Thanks And Regards,
>
> Nitin Phuria
>
>
>
> *Confidentiality Disclaimer**: “The information contained in this
> electronic message (email) and any attachments to this email are intended
> for the exclusive use of the addressee(s) and access to this email by
> anyone else is unauthorized. The email may contain proprietary,
> confidential or privileged information or information relating to Integra
> Group. If you are not the intended recipient, please notify the sender by
> telephone, fax, or return email and delete this communication and any
> attachments thereto, immediately from your computer. Any dissemination,
> distribution, or copying of this communication and the attachments thereto
> (in whole or part), in any manner, is strictly prohibited and actionable at
> law. The recipient acknowledges that emails are susceptible to alteration
> and their integrity cannot be guaranteed and that Company does not
> guarantee that any e-mail is virus-free and accept no liability for any
> damage caused by any virus transmitted by this email.”*
>
>
>
> *From:* Jonathan Valliere [mailto:john...@apache.org]
> *Sent:* 05 July 2021 18:15
> *To:* nit...@integramicro.com
> *Cc:* users@mina.apache.org
> *Subject:* Re: delay in session.write and messageSent event for same
> message
>
>
>
> Have you profiled the garbage collector and heap memory while this happens?
>
>
>
> On Mon, Jul 5, 2021 at 7:35 AM Nitin Phuria
>  wrote:
>
> Dear All,
>
>
>
> We have Server developed using MINA 2.0.9 where it connects
> to two other systems say System-A and System-B.
>
>
>
> The connectivity with both these system is persistence which means our
> server connects and create only single session with each of these systems
>
> and on the same session multiple message exchanges (Request/Response)
> happens.
>
>
>
> The current flow of message is as below:
>
> Once we connect with System-B and have persistence connection (single
> session), System-B will send us message which we have to process in our
> server and then send processed message to System-A then get response from
> System-A and then pass it back to System-B
>
>
>
> We have provision that if load increases we can have multiple instances of
> System-A and our server can establish persistence connection (single
> session) with each of these System-A instances and manage the load.
>
>
>
> Below is the code for establishing the Connectivity with System-B and it is
> persistence connection (Single connection)
>
>
>
> SocketAddress socketAddress = new
> InetSocketAddress(this.hostName,this.port);
>
> NioSocketConnector connector = new NioSocketConnector();
>
> connector.setConnectTimeoutMillis(30 * 1000L);
>
> connector.getSessionConfig().setTcpNoDelay(true);
>
>
> connector.getSessionConfig()

Re: delay in session.write and messageSent event for same message

[Jonathan Valliere]

Have you profiled the garbage collector and heap memory while this happens?

On Mon, Jul 5, 2021 at 7:35 AM Nitin Phuria 
wrote:

> Dear All,
>
>
>
> We have Server developed using MINA 2.0.9 where it connects
> to two other systems say System-A and System-B.
>
>
>
> The connectivity with both these system is persistence which means our
> server connects and create only single session with each of these systems
>
> and on the same session multiple message exchanges (Request/Response)
> happens.
>
>
>
> The current flow of message is as below:
>
> Once we connect with System-B and have persistence connection (single
> session), System-B will send us message which we have to process in our
> server and then send processed message to System-A then get response from
> System-A and then pass it back to System-B
>
>
>
> We have provision that if load increases we can have multiple instances of
> System-A and our server can establish persistence connection (single
> session) with each of these System-A instances and manage the load.
>
>
>
> Below is the code for establishing the Connectivity with System-B and it is
> persistence connection (Single connection)
>
>
>
> SocketAddress socketAddress = new
> InetSocketAddress(this.hostName,this.port);
>
> NioSocketConnector connector = new NioSocketConnector();
>
> connector.setConnectTimeoutMillis(30 * 1000L);
>
> connector.getSessionConfig().setTcpNoDelay(true);
>
>
> connector.getSessionConfig().setIdleTime(IdleStatus.BOTH_IDLE,this.echoTime)
> ;
>
> this.connector.setHandler(systemBHandler); // systemB Handler is our
> IoHandler implementation for System-B
>
> IoFilter codecFilter = new
> ProtocolCodecFilter((ProtocolCodecFactory)systemBProcFactory);// systemB
> ProcFactory is our Codec Implementation
>
> DefaultIoFilterChainBuilder chain = this.connector.getFilterChain();
>
> chain.clear();
>
> chain.addLast("codec", codecFilter);
>
>
>
> With above when we connect to System B we get session object on which we
> exchange multiple req/res messages.
>
>
>
> We have added logs for the message time tracking as below
>
> - when we write message to session via session.write we log the message id
> (internal to our system) in logger with timestamp
>
> - When the message is sent the messageSent event will be triggered so we
> log
> the timestamp in that event along with message id (internal to our system)
>
>  We have seen that the time gap between the session.write and messageSent
> event is mostly in few milliseconds (1 to 5 milliseconds)
>
> But Sometimes we have seen that this gap is increasing to almost 10-12
> seconds which is surprising
>
>
>
>
> #
>
> Time Stamp Details
>
> Sample Message Id
>
>
> 118320938392 (normal behaviour)
>
> 118320743879 (abnormal behaviour)
>
>
> 1
>
> Received message from System B
>
> 2021-07-02 20:31:59,917
>
> 2021-06-30 10:57:10,233
>
>
> 2
>
> Message sent to System A
>
> 2021-07-02 20:31:59,917
>
> 2021-06-30 10:57:10,328
>
>
> 3
>
> Received message from System A
>
> 2021-07-02 20:32:00,005
>
> 2021-06-30 10:57:10,426
>
>
> 4
>
> Message written to System B session (Session.write method)
>
> 2021-07-02 20:32:00,006
>
> 2021-06-30 10:57:10,427
>
>
> 5
>
> Message sent to System B (messageSent event)
>
> 2021-07-02 20:32:00,006
>
> 2021-06-30 10:57:21,749
>
>
>
> Can someone provide more details on why this is happening. This is not a
> regular behaviour but suddenly one bad day it happens and we are clueless
> to
> identify the root-cause. After session.write it has to go to Codec encoder
> which is our implementation as below and when the actual message is written
> to tcp/ip the messageSent event will be triggered by Mina framework. So
> what
> is going wrong and why it is taking so long to write to TCP/IP is driving
> me
> crazy. It will be good to get some clarity or soem guideline to trace the
> issue.
>
>
>
> public void encode(IoSession session, Object msg, ProtocolEncoderOutput
> out)
> throws Exception
>
> {
>
> if(msg instanceof AppMsg)
>
> {
>
> AppMsg appMsg = (AppMsg) msg;
>
> IoBuffer buffer =
>
> IoBuffer.allocate(appMsg.getLengthBytes().length+appMsg.getMsgBytes().length
> );
>
> buffer.put(appMsg.getLengthBytes());
>
> buffer.put(appMsg.getMsgBytes());
>
> buffer.flip();
>
> out.write(buffer);
>
> }
>
> else if(msg instanceof NullMessage)
>
> {
>
> NullMessage m=(NullMessage)msg;
>
> final byte[] bytes = m.getBytes();
>
> IoBuffer buffer=
> IoBuffer.allocate(bytes.length);
>
> buffer.put(bytes);
>
> buffer.flip();
>
> out.write(buffer);
>
>

Re: Mina Server losing messages

[Jonathan Valliere]

Thanks for the report. Please make an issue in JIRA.

On Sun, Jun 27, 2021 at 12:14 PM Itzhaki, Guy
 wrote:

> Hi all,
>
>
>
> During our tests we found that in some circumstances Mina server loses
> messages.
>
> From what we understand this happens when client opens connection and then
> sends message(s) and the server is still busy processing the sessionOpened
> event while the message the client sent went through the server’s
> filterChain.
>
>
>
> Attached is a simple client and server that will help you reproduce the
> problem, before you run it please perform the following steps:
>
> *Configuration:*
>
>1. In order to reproduce the problem SSL filter must be defined
>(already implemented in the attached example)
>2. Update the keystore and truststore files locations and passwords in
>*addSSLFilter()* method in *MinaClient* and *MinaServer*
>3. Set server break points at:
>   1. MinaServerHandler#sessionOpened
>   2. MinaServerHandler#messageReceived
>   3.
>   org.apache.mina.core.filterchain.IoFilter.NextFilter#messageReceived
>4. Set client break points at:
>   1. MinaClientHandler#sessionOpened
>
>
>
> *Since this is a timing problem you need to run a debugger as described
> below:*
>
>1. Start server
>2. Start client
>3. At the client at *MinaClientHandler#sessionOpened* release the
>break point
>4. Release all break points *except* server’s
>*MinaServerHandler#sessionOpened*
>5. After
>org.apache.mina.core.filterchain.IoFilter.NextFilter#messageReceived
>completed to process all events, you can release
>*MinaServerHandler#sessionOpened*
>
>
>
> Expected result:
>
> MinaServerHandler#messageReceived will be invoked with the message sent by
> the client
>
>
>
> Actual result:
>
> MinaServerHandler#messageReceived is not invoked.
>
>
>
> To use the example, add the following jars to your class path:
>
> commons-lang3-3.9.jar
>
> log4j-api-2.13.3.jar
>
> log4j-core-2.13.3.jar
>
> log4j-jcl-2.13.3.jar
>
> mina-core-2.1.4.jar
>
> slf4j-api-1.7.26.jar
>
> spring-jcl-5.2.12.RELEASE.jar
>
>
>
>
>
> We are using mina 2.1.4
>
>
>
> Your help is highly appreciated as this prevents us from releasing the
> product.
>
>
>
> Thanks,
>
> Guy
>
>
>
> -
> To unsubscribe, e-mail: users-unsubscr...@mina.apache.org
> For additional commands, e-mail: users-h...@mina.apache.org

Re: userauth_pubkey: unsupported public key algorithm: rsa-sha2-512

[Jonathan Valliere]

With all the vulnerabilities that get discovered. Running OpenSSH from 2013
is a problem.

On Thu, Apr 8, 2021 at 6:12 AM uma shankar  wrote:

> *Environment details:*
>
> *Server OS* : CentOS release 6.9 (Final)
>
> $ ssh -V
>
> OpenSSH_5.3p1, OpenSSL 1.0.1e-fips 11 Feb 2013
>
> $ sshd -T
>
> port 22
> protocol 2
> addressfamily any
> listenaddress 0.0.0.0:22
> listenaddress [::]:22
> usepam yes
> serverkeybits 1024
> logingracetime 120
> keyregenerationinterval 3600
> x11displayoffset 10
> maxauthtries 6
> maxsessions 10
> clientaliveinterval 0
> clientalivecountmax 3
> permitrootlogin yes
> ignorerhosts yes
> ignoreuserknownhosts no
> rhostsrsaauthentication no
> hostbasedauthentication no
> hostbasedusesnamefrompacketonly no
> rsaauthentication yes
> pubkeyauthentication yes
> kerberosauthentication no
> kerberosorlocalpasswd yes
> kerberosticketcleanup yes
> gssapiauthentication yes
> gssapikeyexchange no
> gssapicleanupcredentials yes
> gssapistrictacceptorcheck yes
> gssapistorecredentialsonrekey no
> gssapikexalgorithms gss-gex-sha1-,gss-group1-sha1-,gss-group14-sha1-
> passwordauthentication yes
> kbdinteractiveauthentication no
> challengeresponseauthentication no
> printmotd yes
> printlastlog yes
> x11forwarding yes
> x11uselocalhost yes
> strictmodes yes
> tcpkeepalive yes
> permitemptypasswords no
> permituserenvironment no
> uselogin no
> compression delayed
> gatewayports no
> showpatchlevel no
> usedns yes
> allowtcpforwarding yes
> allowagentforwarding yes
> useprivilegeseparation yes
> kerberosusekuserok yes
> pidfile /var/run/sshd.pid
> xauthlocation /usr/bin/xauth
> ciphers
> aes128-ctr,aes192-ctr,aes256-ctr,arcfour256,arcfour128,aes128-cbc,3des-cbc,blowfish-cbc,cast128-cbc,aes192-cbc,aes256-cbc,arcfour,
> rijndael-...@lysator.liu.se
> macs hmac-md5,hmac-sha1,umac...@openssh.com
> ,hmac-sha2-256,hmac-sha2-512,hmac-ripemd160,hmac-ripemd...@openssh.com
> ,hmac-sha1-96,hmac-md5-96
> kexalgorithms
> diffie-hellman-group-exchange-sha256,diffie-hellman-group-exchange-sha1,diffie-hellman-group14-sha1,diffie-hellman-group1-sha1
> banner none
> authorizedkeysfile .ssh/authorized_keys
> authorizedkeysfile2 .ssh/authorized_keys2
> loglevel DEBUG
> syslogfacility AUTHPRIV
> hostkey /etc/ssh/ssh_host_rsa_key
> hostkey /etc/ssh/ssh_host_dsa_key
> acceptenv LANG
> acceptenv LC_CTYPE
> acceptenv LC_NUMERIC
> acceptenv LC_TIME
> acceptenv LC_COLLATE
> acceptenv LC_MONETARY
> acceptenv LC_MESSAGES
> acceptenv LC_PAPER
> acceptenv LC_NAME
> acceptenv LC_ADDRESS
> acceptenv LC_TELEPHONE
> acceptenv LC_MEASUREMENT
> acceptenv LC_IDENTIFICATION
> acceptenv LC_ALL
> acceptenv LANGUAGE
> acceptenv XMODIFIERS
> subsystem sftp /usr/libexec/openssh/sftp-server
> maxstartups 10:30:100
> permittunnel no
> permitopen any
>
> sshd-common : 2.6.0
>
> sshd-core : 2.6.0
>
> I am using Client protocol version 2.0; client software version APACHE-
> SSHD-2 .6.0
>
> I am trying to ssh my server(RHEL6) using APACHE-SSHD-2
> .6.0 using below code
> snippet.
>
>  String send = "HOST:" + host + " " + command;
> InputStream inputStream = new
> ByteArrayInputStream(send.getBytes());
> SshClient client = SshClient.setUpDefaultClient();
> client.start();
> ConnectFuture cf = client.connect(username, host, port);
> try (ClientSession session = cf.verify().getSession();) {
>
> session.addPublicKeyIdentity(loadKeypair(privateKey.getAbsolutePath()));
> session.auth().verify(defaultTimeoutSeconds,
> TimeUnit.SECONDS);
>
> This is working fine with RHEL8, Ubuntu14, Ubuntu16, Ubuntu18 but not
> working with RHEL6 and RHEL7, getting below exception.
>
> *unsupported public key algorithm: rsa-sha2-512* in sshd log
>
>
>
> Caused by: org.apache.sshd.common.SshException: No more authentication
> methods available
> at
> org.apache.sshd.common.future.AbstractSshFuture.verifyResult(AbstractSshFuture.java:126)
> at
> org.apache.sshd.client.future.DefaultAuthFuture.verify(DefaultAuthFuture.java:39)
> at
> org.apache.sshd.client.future.DefaultAuthFuture.verify(DefaultAuthFuture.java:32)
> at
> org.apache.sshd.common.future.VerifiableFuture.verify(VerifiableFuture.java:56)
> at
> com.zimbra.cs.rmgmt.RemoteManager.executeRemoteCommand(RemoteManager.java:170)
> at
> com.zimbra.cs.rmgmt.RemoteManager.execute(RemoteManager.java:147)
> ... 70 more
> Caused by: org.apache.sshd.common.SshException: No more authentication
> methods available
> at
> org.apache.sshd.client.session.ClientUserAuthService.tryNext(ClientUserAuthService.java:342)
> at
> org.apache.sshd.client.session.ClientUserAuthService.processUserAuth(ClientUserAuthService.java:277)
> at
> org.apache.sshd.client.session.ClientUserAuthService.process(ClientUserAuthService.java:224)
> at
> 

Re: Datagram Session closed automatically ?

[Jonathan Valliere]

They’re probably being automatically culled if you haven’t used it in a
while.

On Sun, Oct 4, 2020 at 4:53 AM Alex Buechel  wrote:

> Hello together,
>
> We have a scenario using a Unix-Machine, sending Datagram-packages from a
> java-based application to another machine(Windows). On this second machine
> nobody is listening on this specific port, so we receive "Destination
> unreachable"-Package back. We have activated
> setCloseOnUnreachablePort(false). After a few seconds/maybe 1-2 minutes,
> this session was closed suddenly, and is not part of "getManagedSessions"
> anymore.
>
> Can somebody explain some situations, where a DatagramConnector-Session
> will be closed automatically?
>
> Thanks, bests
> Alexander
>

Re: Question about sending a message via udp to multiple clients

[Jonathan Valliere]

That's basically how it works.  Only thing is, when you share a buffer
between multiple IoSessions you need to call #slice() on your IoBuffer for
every IoSession you write the message to.

On Wed, Sep 30, 2020 at 10:06 AM Alex Buechel 
wrote:

> Hello guys,
>
> Imagine i have a String-message, which i want to send to multiple udp
> clients given by their ip addresses and ports. Do you have any short
> example of how to publish this message efficiently to all udp clients?
>
> My attempt (Pseudocode):
>
> NioDatagramConnector connector = new NioDatagramConnector();
>
> foreach (client in clients) {
>
>   connector.connect(new InetSocketAddress(client.ip, client.port));
>
>   connector.broadcast(message);
>
> }
>
> Would this be a correct attempt, which even works for a large number of
> items/second?
>
> Thanks for any advice.
> Alex
>

Re: Does mina-core-2.1.2.jar supported over Open JDK platform or not?

[Jonathan Valliere]

Should work with any compliant JDK

On Wed, Jul 15, 2020 at 3:23 AM Ravi Verma 
wrote:

> Hi, We are using mina-core-2.1.2.jar for one of our products. Need to
> confirm whether mina-core-2.1.2.jar is supported on Open JDK platform or
> not? Any suggestions are highly appreciated. Thanks, Ravi
>

Re: Need SSH Client example

[Jonathan Valliere]

So you want an SFTP example?

On Mon, Jun 1, 2020 at 11:44 AM Steve Pruitt 
wrote:

> I am getting started with SSHD / Mina.  My use case is simply copying
> files to a Linux machine.
>
> I would appreciate is someone can show a simple getting started code for a
> file copy.  There seems to be several classes involved.
>
> Thanks in advance.
>
> -S
>

Re: Vysper support for Xmpp over websockets (RFC 7395)

[Jonathan Valliere]

Vysper is in maintenance only mode right now but if you are interested in
contributing, just let us know.

On Sat, May 16, 2020 at 3:35 PM technologist.kj 
wrote:

> Hi team, is there any development happening in Vysper for supporting RFC
> 7395
> (https://tools.ietf.org/html/rfc7395) ?
>
> For the current implementation of xmpp over websockets in Vysper (which is
> https://tools.ietf.org/id/draft-ietf-xmpp-websocket-01.html)  there are no
> javascript client libraries available supporting it.
>
> Can we expect an implementation of rfc7395 from the contributors ? or is
> there a JS client library available for the current implementation of xmpp
> over websockets ?
>
>
>
> --
> Sent from:
> http://apache-mina.10907.n7.nabble.com/Apache-MINA-User-Forum-f31345.html
>
> -
> To unsubscribe, e-mail: users-unsubscr...@mina.apache.org
> For additional commands, e-mail: users-h...@mina.apache.org
>
>

Re: MINA vs a NIO based protocol stack

[Jonathan Valliere]

Deals with fragmentation?  UDP fragmentation is handled in the kernel.

On Tue, Apr 14, 2020 at 9:19 AM Emmanuel Lécharny 
wrote:

>
> On 14/04/2020 14:31, Simha N wrote:
> > Hi,
> >
> > I am evaluating MINA for development of Java based networking clients
> based
> > on protocols such as SNMP and Netconf.
> >
> > Protocol stacks are available for these protocols. Some of the stacks are
> > based on Java NIO (example: SNMP4J) and some are not based on Java NIO
> > (Example: JoeSNMP and JNC).
> >
> > In order to develop a networking client using MINA, Application developer
> > shall write a Codec by implementing the interface ProtocolDecoderAdapter
> > for each protocol. Application developer has to understand chosen
> protocol
> > stack API well to write the codec. This needs good amount of protocol
> > knowledge.
> >
> > If my protocol stack is based on Java BIO (not a NIO), then I see a valid
> > reason to move to MINA to get performance gain in spite effort needed to
> > writing codec.
> >
> > But, if my protocol stack is based on Java NIO, what is the advantage of
> > using MINA? Does it bring in any performance gain?
>
> Simplicity. It alleviates you from dealing of all the burden of managing
> the Selector on your own. You just end with dealing with events like
> 'messageReceived' or 'messageSent', plus the few session management
> messages (sessionOpened, sessionClosed, etc)
>
> It also deals with fragmentation, which is quite a relief.
>
> It won't bring you any performance, because you are adding a stack on
> top of NIO.
>
> -
> To unsubscribe, e-mail: users-unsubscr...@mina.apache.org
> For additional commands, e-mail: users-h...@mina.apache.org
>
>

Re: MINA: ExecutorFilter on NioSocketConnector

[Jonathan Valliere]

Huh?

On Thu, Apr 9, 2020 at 1:09 PM Weact  wrote:

>
>
>
>  Is this server ready for deployment for a chat application supporting 1mm
> users or is it more in the prototype phase?
>
>
>
> We will be using group chat, roles, moderators and adding some unique chat
> threading models for distributed groups.
>
>
>
> Thanks.
>
>
>
>
>
>
>
> >
> > On Apr 9, 2020 at 11:21 AM,   elecha...@gmail.com)>  wrote:
> >
> >
> >
> >  On 09/04/2020 16:01, Nitin Phuria wrote:  >  Dear Jonathan,  >   >   >
>  >  Anytime we write message to System-B session with session.write method
> the messageSent event has to be generated with it actually gets written on
> TCP/IP The messageSent event is only generated when the message has been
> fully written into the socket. A message can be written piece by piece,
> depending on the socket capacity. Also messages can be queued until the
> previous messages have been fully written. What could happen is that if you
> have a slow reader on the other side, then the socket will not be able to
> accept any data, because it gets full.  >   >   >   >
> org.apache.mina.core.session.IoSession.write(Object)  >   >   >   >  We see
> that there is time difference of 10-14 seconds between timestamp of
> session.write for particular message and messageSent event getting
> generated for the same message. So you have a problem in the middle. Check
> that the reader (ie the other server) is processing messages fast enough.
> Or maybe you have network issues, as suggests Jonathan.  >   >   >   >  In
> non-peak load this time difference is in 10-15 milliseconds and in peak
> load (60-70 messages per second) it goes till 10-14 seconds. 60/70 msg/s is
> not exceptionally high, except if you are managing big messages, or do a
> lot of processing on each message.
> - To
> unsubscribe, e-mail: users-unsubscr...@mina.apache.org For additional
> commands, e-mail: users-h...@mina.apache.org
> >
> >
>
>
>

-- 

CONFIDENTIALITY NOTICE: The contents of this email message and any
attachments are intended solely for the addressee(s) and may contain
confidential and/or privileged information and may be legally protected
from disclosure.

Re: MINA: ExecutorFilter on NioSocketConnector

[Jonathan Valliere]

Make sure you enable BBR TCP on your host machines to help mitigate the
network problems.

On Thu, Apr 9, 2020 at 10:58 AM Nitin Phuria 
wrote:

> Dear Jonathan,
>
>
>
> Thanks for bringing more clarity on Processor Thread and
> it's relation with IoSession. We are also suspecting the network connection
> and in a crisis one should pull all the strings to check where you get the
> lead to identify the problem and that's what we are doing.
>
>
>
> Thanks a tone for all your replies and giving very useful information. We
> will check further on network  connection.
>
>
>
> Thanks And Regards,
>
> Nitin Phuria
>
>
>
> *Confidentiality Disclaimer**: “The information contained in this
> electronic message (email) and any attachments to this email are intended
> for the exclusive use of the addressee(s) and access to this email by
> anyone else is unauthorized. The email may contain proprietary,
> confidential or privileged information or information relating to Integra
> Group. If you are not the intended recipient, please notify the sender by
> telephone, fax, or return email and delete this communication and any
> attachments thereto, immediately from your computer. Any dissemination,
> distribution, or copying of this communication and the attachments thereto
> (in whole or part), in any manner, is strictly prohibited and actionable at
> law. The recipient acknowledges that emails are susceptible to alteration
> and their integrity cannot be guaranteed and that Company does not
> guarantee that any e-mail is virus-free and accept no liability for any
> damage caused by any virus transmitted by this email.”*
>
>
>
> *From:* Jonathan Valliere [mailto:jon.valli...@emoten.com]
> *Sent:* 09 April 2020 20:03
> *To:* Nitin Phuria
> *Cc:* users@mina.apache.org
> *Subject:* Re: MINA: ExecutorFilter on NioSocketConnector
>
>
>
> Every IoSession is bound to one Processor Thread.  So only by creating
> more IoSessions can you take advantage of multiple Processor Threads.
>
>
>
> ExecutorFilter only works for received messages.  So it generally does
> nothing on the client side however it will cause the received messages to
> be executed in multiple threads.
>
>
>
> The purpose of the ExecutorFilter is to do potentially blocking tasks in
> there so as not to block the Processor Thread from handling tasks from
> other IoSessions.
>
>
>
> Since you’re seeing a huge jump from 15ms to several minutes and a very
> small number of messages it is most likely a problem with your network
> connection.
>
>
>
> On Thu, Apr 9, 2020 at 10:26 AM Nitin Phuria 
> wrote:
>
> Dear Jonathan,
>
> Our Server and System-B are on WAN network with leased line.
>
>
>
> One more clarification required As per documentation
>
>
>
> NioSocketConnector
> <http://mina.apache.org/mina-project/gen-docs/2.0.10/apidocs/org/apache/mina/transport/socket/nio/NioSocketConnector.html#NioSocketConnector-->
> () Constructor for NioSocketConnector
> <http://mina.apache.org/mina-project/gen-docs/2.0.10/apidocs/org/apache/mina/transport/socket/nio/NioSocketConnector.html>
> with default configuration (multiple thread model).
>
>
>
> So when we create NioSocketConnector connector = new NioSocketConnector(); I 
> expect it to create n+1 processor
>
> where n is number of core on the hardware. In our case it is 17 (16 core
> machine)
>
>
>
> When we took threaddump of our Server process I could see only one thread
> for this connector and in the log (log4j) when we print the thread name it
> prints NioProcessor-x where x is some number.
>
>
>
> So does it means that n+1 processors use single thread to work.
>
>
>
> Also you mentioned that The ExecutorFilter only works for the Server
> receiver side.
>
>
>
>
> http://apache-mina.10907.n7.nabble.com/OrderedThreadPoolExecutor-shutdown-problem-in-MINA-2-0-0-M4-td34417.html
>
>
>
> But I could see from this above URL that we can add it to Connector also
> and it says "*My understanding is that the Executor filter just adds an
> executor in the chain which will be used to spread the load on many
> threads. This is an optimization, rather than something you need to use. If
> you don't use it, your program will work fine.*"
>
>
>
> Could you provide more insight and your thoughts.
>
>
>
> Thanks And Regards,
>
> Nitin Phuria
>
>
>
> *Confidentiality Disclaimer**: “The information contained in this
> electronic message (email) and any attachments to this email are intended
> for the exclusive use of the addressee(s) and access to this email by
> anyone else is unautho

Re: MINA: ExecutorFilter on NioSocketConnector

[Jonathan Valliere]

Every IoSession is bound to one Processor Thread.  So only by creating more
IoSessions can you take advantage of multiple Processor Threads.

ExecutorFilter only works for received messages.  So it generally does
nothing on the client side however it will cause the received messages to
be executed in multiple threads.

The purpose of the ExecutorFilter is to do potentially blocking tasks in
there so as not to block the Processor Thread from handling tasks from
other IoSessions.

Since you’re seeing a huge jump from 15ms to several minutes and a very
small number of messages it is most likely a problem with your network
connection.

On Thu, Apr 9, 2020 at 10:26 AM Nitin Phuria 
wrote:

> Dear Jonathan,
>
> Our Server and System-B are on WAN network with leased line.
>
>
>
> One more clarification required As per documentation
>
>
>
> NioSocketConnector
> <http://mina.apache.org/mina-project/gen-docs/2.0.10/apidocs/org/apache/mina/transport/socket/nio/NioSocketConnector.html#NioSocketConnector-->
> () Constructor for NioSocketConnector
> <http://mina.apache.org/mina-project/gen-docs/2.0.10/apidocs/org/apache/mina/transport/socket/nio/NioSocketConnector.html>
> with default configuration (multiple thread model).
>
>
>
> So when we create NioSocketConnector connector = new NioSocketConnector(); I 
> expect it to create n+1 processor
>
> where n is number of core on the hardware. In our case it is 17 (16 core
> machine)
>
>
>
> When we took threaddump of our Server process I could see only one thread
> for this connector and in the log (log4j) when we print the thread name it
> prints NioProcessor-x where x is some number.
>
>
>
> So does it means that n+1 processors use single thread to work.
>
>
>
> Also you mentioned that The ExecutorFilter only works for the Server
> receiver side.
>
>
>
>
> http://apache-mina.10907.n7.nabble.com/OrderedThreadPoolExecutor-shutdown-problem-in-MINA-2-0-0-M4-td34417.html
>
>
>
> But I could see from this above URL that we can add it to Connector also
> and it says "*My understanding is that the Executor filter just adds an
> executor in the chain which will be used to spread the load on many
> threads. This is an optimization, rather than something you need to use. If
> you don't use it, your program will work fine.*"
>
>
>
> Could you provide more insight and your thoughts.
>
>
>
> Thanks And Regards,
>
> Nitin Phuria
>
>
>
> *Confidentiality Disclaimer**: “The information contained in this
> electronic message (email) and any attachments to this email are intended
> for the exclusive use of the addressee(s) and access to this email by
> anyone else is unauthorized. The email may contain proprietary,
> confidential or privileged information or information relating to Integra
> Group. If you are not the intended recipient, please notify the sender by
> telephone, fax, or return email and delete this communication and any
> attachments thereto, immediately from your computer. Any dissemination,
> distribution, or copying of this communication and the attachments thereto
> (in whole or part), in any manner, is strictly prohibited and actionable at
> law. The recipient acknowledges that emails are susceptible to alteration
> and their integrity cannot be guaranteed and that Company does not
> guarantee that any e-mail is virus-free and accept no liability for any
> damage caused by any virus transmitted by this email.”*
>
>
>
> *From:* Jonathan Valliere [mailto:jon.valli...@emoten.com]
> *Sent:* 09 April 2020 19:34
> *To:* Nitin Phuria
> *Cc:* users@mina.apache.org
> *Subject:* Re: MINA: ExecutorFilter on NioSocketConnector
>
>
>
> How far are the servers?  TCP Packet loss could explain this.
>
>
>
> You can get the Queue size from the IoSession object.
>
>
>
> On Thu, Apr 9, 2020 at 10:02 AM Nitin Phuria 
> wrote:
>
> Dear Jonathan,
>
>
>
> Anytime we write message to System-B session with session.write method the
> messageSent event has to be generated with it actually gets written on
> TCP/IP
>
>
>
> org.apache.mina.core.session.IoSession.write(Object)
>
>
>
> We see that there is time difference of 10-14 seconds between timestamp of
> session.write for particular message and messageSent event getting
> generated for the same message.
>
>
>
> In non-peak load this time difference is in 10-15 milliseconds and in peak
> load (60-70 messages per second) it goes till 10-14 seconds.
>
>
>
> Is there anything that we can log to see what is happening or see the
> queue size etc..  for further better understaning.
>
>
>
> Thanks And Regards,
>
> Nitin Phuria
>

Re: MINA: ExecutorFilter on NioSocketConnector

[Jonathan Valliere]

How far are the servers?  TCP Packet loss could explain this.

You can get the Queue size from the IoSession object.

On Thu, Apr 9, 2020 at 10:02 AM Nitin Phuria 
wrote:

> Dear Jonathan,
>
>
>
> Anytime we write message to System-B session with session.write method the
> messageSent event has to be generated with it actually gets written on
> TCP/IP
>
>
>
> org.apache.mina.core.session.IoSession.write(Object)
>
>
>
> We see that there is time difference of 10-14 seconds between timestamp of
> session.write for particular message and messageSent event getting
> generated for the same message.
>
>
>
> In non-peak load this time difference is in 10-15 milliseconds and in peak
> load (60-70 messages per second) it goes till 10-14 seconds.
>
>
>
> Is there anything that we can log to see what is happening or see the
> queue size etc..  for further better understaning.
>
>
>
> Thanks And Regards,
>
> Nitin Phuria
>
>
>
> *Confidentiality Disclaimer**: “The information contained in this
> electronic message (email) and any attachments to this email are intended
> for the exclusive use of the addressee(s) and access to this email by
> anyone else is unauthorized. The email may contain proprietary,
> confidential or privileged information or information relating to Integra
> Group. If you are not the intended recipient, please notify the sender by
> telephone, fax, or return email and delete this communication and any
> attachments thereto, immediately from your computer. Any dissemination,
> distribution, or copying of this communication and the attachments thereto
> (in whole or part), in any manner, is strictly prohibited and actionable at
> law. The recipient acknowledges that emails are susceptible to alteration
> and their integrity cannot be guaranteed and that Company does not
> guarantee that any e-mail is virus-free and accept no liability for any
> damage caused by any virus transmitted by this email.”*
>
>
>
> *From:* Jonathan Valliere [mailto:jon.valli...@emoten.com]
> *Sent:* 09 April 2020 18:31
> *To:* Nitin Phuria
> *Cc:* Nitin Phuria; users@mina.apache.org
> *Subject:* Re: MINA: ExecutorFilter on NioSocketConnector
>
>
>
> The ExecutorFilter only works for the Server receiver side.
>
>
>
> When you say the responses are slow, what do you consider slow?
>
>
>
> All writes are put into a queue dedicated for each session.  The Session
> is then enabled for writing by adding the write interest to the poll
> mechanism.  Reads and Writes IO happen in the same thread.  After the
> Session finishes reading it will then call the poll mechanism for new
> events to process. Hopefully it should get the write event for your Session
> then begin to flush the queue.
>
>
>
> Processor Thread Loop —>   Select Poll Events —>  Process Events
> (read/write) —> Loop
>
>
>
> On Thu, Apr 9, 2020 at 8:53 AM Nitin Phuria 
> wrote:
>
> Dear Jonathan,
>
> Thanks for the reply, can you provide more details on the "All
> writes are queued up and scheduled
> later using the poll/kqueue system." How this poll & kqueue system works?
> Do we have any control on this from MINA framework?
>
> We are not facing any problem with the higher CPU consumption but we see
> that when there is high load from System-B to our server the reads from
> System-B and write back to System-B on the single persistence session is
> becoming slow. So we were looking at i/o operations and how to make them
> fast.
>
> With your explanation provided in below email about ExecutorFilter, If we
> add it in our connector after codec filter will help us reduce the I/O
> overhead on reactor/processor thread. I hope my understanding is right
> kindly confirm?
>
>
> Thanks And Regards,
> Nitin Phuria
>
> Confidentiality Disclaimer: “The information contained in this electronic
> message (email) and any attachments to this email are intended for the
> exclusive use of the addressee(s) and access to this email by anyone else
> is unauthorized. The email may contain proprietary, confidential or
> privileged information or information relating to Integra Group. If you are
> not the intended recipient, please notify the sender by telephone, fax, or
> return email and delete this communication and any attachments thereto,
> immediately from your computer. Any dissemination, distribution, or copying
> of this communication and the attachments thereto (in whole or part), in
> any manner, is strictly prohibited and actionable at law. The recipient
> acknowledges that emails are susceptible to alteration and their integrity
> cannot be guaranteed and that Company does not guarantee 

Re: MINA: ExecutorFilter on NioSocketConnector

[Jonathan Valliere]

The ExecutorFilter only works for the Server receiver side.

When you say the responses are slow, what do you consider slow?

All writes are put into a queue dedicated for each session.  The Session is
then enabled for writing by adding the write interest to the poll
mechanism.  Reads and Writes IO happen in the same thread.  After the
Session finishes reading it will then call the poll mechanism for new
events to process. Hopefully it should get the write event for your Session
then begin to flush the queue.

Processor Thread Loop —>   Select Poll Events —>  Process Events
(read/write) —> Loop

On Thu, Apr 9, 2020 at 8:53 AM Nitin Phuria  wrote:

> Dear Jonathan,
>
> Thanks for the reply, can you provide more details on the "All
> writes are queued up and scheduled
> later using the poll/kqueue system." How this poll & kqueue system works?
> Do we have any control on this from MINA framework?
>
> We are not facing any problem with the higher CPU consumption but we see
> that when there is high load from System-B to our server the reads from
> System-B and write back to System-B on the single persistence session is
> becoming slow. So we were looking at i/o operations and how to make them
> fast.
>
> With your explanation provided in below email about ExecutorFilter, If we
> add it in our connector after codec filter will help us reduce the I/O
> overhead on reactor/processor thread. I hope my understanding is right
> kindly confirm?
>
>
> Thanks And Regards,
> Nitin Phuria
>
> Confidentiality Disclaimer: “The information contained in this electronic
> message (email) and any attachments to this email are intended for the
> exclusive use of the addressee(s) and access to this email by anyone else
> is unauthorized. The email may contain proprietary, confidential or
> privileged information or information relating to Integra Group. If you are
> not the intended recipient, please notify the sender by telephone, fax, or
> return email and delete this communication and any attachments thereto,
> immediately from your computer. Any dissemination, distribution, or copying
> of this communication and the attachments thereto (in whole or part), in
> any manner, is strictly prohibited and actionable at law. The recipient
> acknowledges that emails are susceptible to alteration and their integrity
> cannot be guaranteed and that Company does not guarantee that any e-mail is
> virus-free and accept no liability for any damage caused by any virus
> transmitted by this email.”
>
> -Original Message-
> From: Jonathan Valliere [mailto:jon.valli...@emoten.com]
> Sent: 09 April 2020 18:09
> To: Nitin Phuria
> Cc: users@mina.apache.org
> Subject: Re: MINA: ExecutorFilter on NioSocketConnector
>
> Mina is a queue and flush framework. All writes are queued up and scheduled
> later using the poll/kqueue system.
>
> The ExecutorFilter uses threading to process incoming messages using a pool
> of threads instead of the reactor/processor thread.  This frees the
> processor thread to do IO.
>
> Have you profiled the application to figure out which code is consuming the
> CPU?
>
> On Thu, Apr 9, 2020 at 3:39 AM Nitin Phuria  .invalid>
> wrote:
>
> > Dear All,
> >
> >
> >
> > We have Server developed using MINA 2.0.16 where it
> > connects
> > to two other systems say System-A and System-B.
> >
> >
> >
> > The connectivity with both these system is persistence which means our
> > server connects and create only single session with each of these systems
> > and on the same session multiple message exchanges (Request/Response)
> > happens.
> >
> >
> >
> > The current flow of message is as below:
> >
> > Once we connect with System-B and have persistence connection (single
> > session), System-B will send us message which we have to process in our
> > server and then send processed message to System-A then get response from
> > System-A and then pass it back to System-B
> >
> >
> >
> > We have provision that if load increases we can have multiple instances
> of
> > System-A and our server can establish persistence connection (single
> > session) with each of these System-A instances and manage the load.
> >
> >
> >
> > We are facing problem with load coming from System-B to our server as it
> is
> > coming on single session and we could see that the read/write on the
> > session
> > are slowed down and lot of queue build-up is happening for read/write.
> >
> >
> >
> > Below is the code for establishing the Connectivity with System-B where
> we
> > have

Re: MINA: ExecutorFilter on NioSocketConnector

[Jonathan Valliere]

Mina is a queue and flush framework. All writes are queued up and scheduled
later using the poll/kqueue system.

The ExecutorFilter uses threading to process incoming messages using a pool
of threads instead of the reactor/processor thread.  This frees the
processor thread to do IO.

Have you profiled the application to figure out which code is consuming the
CPU?

On Thu, Apr 9, 2020 at 3:39 AM Nitin Phuria 
wrote:

> Dear All,
>
>
>
> We have Server developed using MINA 2.0.16 where it
> connects
> to two other systems say System-A and System-B.
>
>
>
> The connectivity with both these system is persistence which means our
> server connects and create only single session with each of these systems
> and on the same session multiple message exchanges (Request/Response)
> happens.
>
>
>
> The current flow of message is as below:
>
> Once we connect with System-B and have persistence connection (single
> session), System-B will send us message which we have to process in our
> server and then send processed message to System-A then get response from
> System-A and then pass it back to System-B
>
>
>
> We have provision that if load increases we can have multiple instances of
> System-A and our server can establish persistence connection (single
> session) with each of these System-A instances and manage the load.
>
>
>
> We are facing problem with load coming from System-B to our server as it is
> coming on single session and we could see that the read/write on the
> session
> are slowed down and lot of queue build-up is happening for read/write.
>
>
>
> Below is the code for establishing the Connectivity with System-B where we
> have not added the ExecutorFilter
>
>
>
> SocketAddress socketAddress = new InetSocketAddress(this.hostName,
> this.port);
>
> NioSocketConnector connector = new NioSocketConnector();
>
> connector.setConnectTimeoutMillis(30 * 1000L);
>
> connector.getSessionConfig().setTcpNoDelay(true);
>
>
> connector.getSessionConfig().setIdleTime(IdleStatus.BOTH_IDLE,this.echoTime)
> ;
>
> this.connector.setHandler(systemBHandler); // systemBHandler is our
> IoHandler implementationfor System-B
>
> IoFilter codecFilter = new
> ProtocolCodecFilter((ProtocolCodecFactory)systemBProcFactory);//
> systemBProcFactory is our Codec Implementation
>
> DefaultIoFilterChainBuilder chain = this.connector.getFilterChain();
>
> chain.clear();
>
> chain.addLast("codec", codecFilter);
>
>
>
> https://cwiki.apache.org/confluence/display/MINA/Configuring+Thread+Model
>
>
> From this link we could understand as below
>
>
>
> If We didn't add any ExecutorFilter events are forwarded to an IoHandler
> via
> method invocations. This means your business logic in your IoHandler
> implementation will run in the I/O processor thread. We call this thread
> model a 'single thread model'. The single thread model is known to be
> adequate for low-latency network applications with CPU-bound business logic
> (e.g. game servers).
>
>
>
> Do we need to add the ExecutorFilter for above mentioned NioSocketConnector
> to System-B so that I/O processor thread will only do the I/O job and all
> other business logic will be given to worker threads in ExecutorFilter.
>
>
>
> Is there any way to increase the number of threads for my I/O to do the
> read/write as we have restriction to create only one session with System-B.
>
>
>
> If we add ExecutorFilter  with IoEventType for read/write then will that
> make the read/write faster with System-B on same single session.
>
>
>
> Or we add ExecutorFilter  with default IoEventType (All Events) will that
> make the read/write faster with System-B on same single session.
>
>
>
> Pls help us understand the working on ExecutorFilter in above scenario
> where
> NioSocketConnector has only single session to do all the read/write with
> huge load of almost 60-70 messages per second with average message size of
> 3-4KiloBytes.
>
>
>
> Currently we have server having 16 Core with 64 GB RAM.
>
>
>
> Thanks And Regards,
>
> Nitin Phuria
>
>
>
> Confidentiality Disclaimer: "The information contained in this electronic
> message (email) and any attachments to this email are intended for the
> exclusive use of the addressee(s) and access to this email by anyone else
> is
> unauthorized. The email may contain proprietary, confidential or privileged
> information or information relating to Integra Group. If you are not the
> intended recipient, please notify the sender by telephone, fax, or return
> email and delete this communication and any attachments thereto,
> immediately
> from your computer. Any dissemination, distribution, or copying of this
> communication and the attachments thereto (in whole or part), in any
> manner,
> is strictly prohibited and actionable at law. The recipient acknowledges
> that emails are susceptible to alteration and their integrity cannot be
> guaranteed and that Company does not guarantee that any e-mail is
> virus-free
> and accept no liability for any 

Re: Handling MQTT messages using apache mina library is possible?

[Jonathan Valliere]

Mina uses the Socket API for TCP and UDP.

On Mon, Jul 22, 2019 at 5:17 AM Dhananjay Kandari <
dk00453...@techmahindra.com> wrote:

> Hi,
> Is it possible to use apache mina library to handle MQTT messages?
> In my project, we are using mina library to handle request coming over
> TCP/IP and now have to handle MQTT request so thing whether can use same
> architecture to handle MQTT or need to install separate broker to handle it
> like mosquito
>
> Thanks.
> 
> Disclaimer: This message and the information contained herein is
> proprietary and confidential and subject to the Tech Mahindra policy
> statement, you may review the policy at
> http://www.techmahindra.com/Disclaimer.html externally
> http://tim.techmahindra.com/tim/disclaimer.html internally within
> TechMahindra.
> 
>
>
> -
> To unsubscribe, e-mail: users-unsubscr...@mina.apache.org
> For additional commands, e-mail: users-h...@mina.apache.org

-- 

CONFIDENTIALITY NOTICE: The contents of this email message and any
attachments are intended solely for the addressee(s) and may contain
confidential and/or privileged information and may be legally protected
from disclosure.

MINA 3.0 DEPRECATED

[Jonathan Valliere]

Hello MINA developers & users,

I'm sure your inboxes have been flooded with emails from JIRA regarding
issues being closed.  Emmanuel and I have decided to and are beginning the
process of deprecating the 3.0 branch.  This means that support for the 3.0
branch has officially ended.  All outstanding issues have been marked
"Abandoned" in JIRA and the 3.0 branch in GIT will be marked similar.

Support for the 2.0.X and 2.1.X branches will continue until further notice.

Thank you for your patience.

Re: High CPU Utilization because of Mina-core jar

[Jonathan Valliere]

See
https://issues.apache.org/jira/browse/DIRMINA-1114?jql=project%20%3D%20DIRMINA%20AND%20resolution%20%3D%20Unresolved%20AND%20text%20~%20%22cpu%22%20ORDER%20BY%20priority%20DESC%2C%20updated%20DESC

On Fri, May 24, 2019 at 10:17 AM Guus der Kinderen <
guus.der.kinde...@gmail.com> wrote:

> I don't know anything about the release schedule. You could, however,
> compile the project from source.
>
> I've don't that, and made the artifacts available here:
>
> https://discourse.igniterealtime.org/t/openfire-4-3-2-cpu-goes-at-100-after-a-few-hours-on-linux/85119/27
>
> On Fri, 24 May 2019 at 15:58, Hrushikesh Agrawal
> 
> wrote:
>
> > Thank Guus. Do you know the expected date the 2.1.3 would be available? I
> > wanted to try with this jar as well.
> >
> > Regards,
> > Hrushi
> >
> > On Fri, May 24, 2019 at 6:28 PM Guus der Kinderen <
> > guus.der.kinde...@gmail.com> wrote:
> >
> > > This sounds like https://issues.apache.org/jira/browse/DIRMINA-
> but
> > I
> > > think the MINA devs told me that this was limited to the 2.1.X branch.
> If
> > > that's true, then I cannot explain why you found the issue in 2.0.18
> and
> > > 2.0.21.
> > >
> > > On Fri, 24 May 2019 at 14:36, Hrushikesh Agrawal
> > > 
> > > wrote:
> > >
> > > > Hi,
> > > >
> > > > We are have our own product  through which we are trying to
> communicate
> > > > with LDAP server by using the mina-core-2.1.2.jar and
> api-all-1.0.3.jar
> > > > jars. When we have invalid connection parameter, and tried to get the
> > > > connection, CPU core utilization is 100%.
> > > > Then we have seen in JVisualVM and in Jprofiler that
> > > > "org.apache.mina.transport.socket.nio.NioSocketConnector" is in CPU
> > more
> > > > time.
> > > >
> > > > We can see below in thread dump-
> > > >
> > > > "NioSocketConnector-3" #152 prio=5 os_prio=0 tid=0x7f95940d4000
> > > > nid=0x77fb runnable [0x7f956f1fe000]
> > > >java.lang.Thread.State: RUNNABLE
> > > > at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
> > > > at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)
> > > > at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93)
> > > > at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
> > > > - locked <0xebdf89e8> (a sun.nio.ch.Util$3)
> > > > - locked <0xebdf89d8> (a
> java.util.Collections$UnmodifiableSet)
> > > > - locked <0xebdf88c0> (a sun.nio.ch.EPollSelectorImpl)
> > > > at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
> > > > at
> > > >
> > > >
> > >
> >
> org.apache.mina.transport.socket.nio.NioSocketConnector.select(NioSocketConnector.java:292)
> > > > at
> > > >
> > > >
> > >
> >
> org.apache.mina.core.polling.AbstractPollingIoConnector$Connector.run(AbstractPollingIoConnector.java:433)
> > > > at
> > > >
> > > >
> > >
> >
> org.apache.mina.util.NamePreservingRunnable.run(NamePreservingRunnable.java:64)
> > > > at
> > > >
> > > >
> > >
> >
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> > > > at
> > > >
> > > >
> > >
> >
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> > > > at java.lang.Thread.run(Thread.java:748)
> > > >
> > > > We can see same issue with standalone java class and try to get the
> > ldap
> > > > connection. Attaching couple of snapshot to understand the issue.
> > > > Please note this happens in Ubuntu and Centos. Also found same issue
> > with
> > > > 2.0.21, 2.0.18 mina core jar.
> > > >
> > > > Can someone please help me on this.
> > > >
> > > > Thanks,
> > > > Hrushi
> > > >
> > >
> >
>

Re: NioSocketAcceptor grabbing a number of ports when instantiated

[Jonathan Valliere]

MINA only allocates ports the application tells it to. Nothing is
automatic.

On Fri, May 17, 2019 at 10:23 AM Emmanuel Lécharny 
wrote:

>
> On 17/05/2019 16:04, Phil Hudson wrote:
> >
> > Hi All,
> >
> > I have 2 questions about the way NioSocketAcceptor allocates and uses
> > ports.
> >
> > Background
> >
> > We are using a 3^rd party product (QuickFix/J) for client/server
> > communication, which in turn uses
> >
> > MINA for network communication.  For server side communication,
> > QuickFix/J is instantiating a
> >
> > NioSocketAcceptor for listening on a specified port.  We recently
> > noticed an issue where a large number
> >
> > of additional ports are also being allocated when a NioSocketAcceptor
> > is instantiated.We seem to
> >
> > see around 10 ‘pairs’ of ports allocated (20 total) with each
> > instantiation.   As you can see in the sample
> >
> > below, they seem to be in pairs, where they talk to each other. ??
> >
> >   Proto  Local Address  Foreign AddressState
> PID
> >
> >   TCP127.0.0.1:55686 MYSYS:55687 ESTABLISHED 24976
> >
> >   TCP127.0.0.1:55687 MYSYS:55686 ESTABLISHED 24976
> >
> >   TCP127.0.0.1:55688 MYSYS:55689ESTABLISHED 24976
> >
> >   TCP127.0.0.1:55689 MYSYS:55688ESTABLISHED 24976
> >
> >   …
> >
> > Note that the actual port being listened on is a different port
> > altogether… example: port 6073
> >
> > I have searched for why this is occurring, but can’t find any
> > documentation on it.
> >
> > Questions
> >
> > 1) Why do 20 ports get allocated when a NioSocketAcceptor is
> instantiated?
> >
> > 1a) What is it using these ports for?
> >
>
> No idea. It all depends on the application using MINA.
>
> > 2) Is there a way to control this?   (or control the range it
> > allocates from?)
> >
>
> You have to ask the QuickFix/J fellows.
>
> MINA is just a framework, it does what you tell it to do...
>
> Quickly browsing their documentation, it seems they allow users to set a
> failover mechanism, by which they start more than one acceptor:
>
> https://www.quickfixj.org/usermanual/2.1.0/usage/acceptor_failover.html
>
> but that might be unrelated. Just ask QuickFix/J people, they are the
> ones who know :-)
>
>
> And thanks them for using MINA ;-)
>
>
>

Re: IoFilter mark() usage, between 2.0 and 2.1

[Jonathan Valliere]

David,

The `buf.mark()` and `buf.reset()` calls are dangerous when used in
conjunction with a dynamic call stack and are being removed from the MINA
Filter system to improve compatibility with the code you write.

1> Some things aren't being back-ported to 2.0.X, the SSL improvements are
one.
2> What call?  the mark() was removed, so it is not needed.
3> Shouldn't be necessary because you shouldn't have to change any of your
code.

Did removal of this mark() break your code?

On Fri, May 3, 2019 at 6:29 PM David Trott  wrote:

> Hey,
>
> I am working on an IoFilter similar in location/function to the SslFilter
> (but implementing a different protocol). I noted there was a delta between
> 2.0 and 2.1 code in the SslFilter.
>
> Specifically commit 2d08d530961597b9f21dff861725f08e73fb9291
> Has removed the call to  buf.mark(); // Approx Line 654
>
> In my filter (compiled against 2.0) I also need the same buf.mark() call as
> the SslFilter uses.
>
> Could I ask the following questions:
> 1.> Why does the 2.0 code base need this call?
> 2.> Is this call needed in the 2.1 code base?
> 3.> Would it be possible to update the (2.1-vs-2.0) migration web page to
> include a note about this change?
>
>
> Thanks,
> David
>

Re: Logging Problem in ProtocolCodecFilter

[Jonathan Valliere]

Oh I’ll fix that in the morning.

On Wed, Apr 24, 2019 at 3:22 PM Emmanuel Lecharny 
wrote:

> Hi Manuel,
>
> There is a bug in the function that dumps data as hex (
> https://issues.apache.org/jira/plugins/servlet/mobile#issue/DIRMINA-1104)
>
> This should be fixed in a coming version.
>
> Le mar. 23 avr. 2019 à 12:21, Manuel Hegner  a
> écrit :
>
> > Heyo,
> >
> > we have a problem in using the ProtocolCodecFilter class. If anything in
> > messageReceived goes wrong mina creates a ProtocolDecoderException. If
> our
> > tool is not able to ascertain a readable error message we simply log the
> > exception stacktrace and all. In minas case this means that mina tries to
> > write the whole hex dump to the log, which can take forever if the
> message
> > is very large.
> >
> > For now we circumvent this by using a changed version of
> > ProtocolCodecFilterthat only adds the first n bytes to the exception, but
> > this is kind of messy.
> >
> > Is there a better way to deal with that?
> >
> > Kindest regards,
> > Manuel Hegner
> > --
> >  *Manuel Hegner*
> > Data Engineer
> >  +49 30 3406 068 – 47 <+4930340606847>
> >  manuel.heg...@bakdata.com bakdata GmbH
> > Falckensteinstraße 24
> > D-10997 Berlin
> > Geschäftsführer: Alexander Albrecht, Christoph Böhm, Frank Kaufer
> > Sitz Berlin, Amtsgericht Berlin HRB 174303
> >
> --
> Regards,
> Cordialement,
> Emmanuel Lécharny
> www.iktek.com
>

Re: Loopback messages

[Jonathan Valliere]

On a side note, this is the developers mailing list.  We're filling the
inboxes of other developers that probably don't want to be a part of this
conversation.  We should move it to the users@mina.apache.org mailing list.

On Thu, Feb 14, 2019 at 11:36 AM Jonathan Valliere 
wrote:

> so I cannot ensure a message is being received/sent and look at some queue
>> for this.
>
>
> Why do you need to ensure the message is sent?  Once you call write(), it
> will be sent eventually.  The only guarantee that it was actually sent and
> received would to require the Client to reply to the message.  This could
> be done by using some kind of Queue attached to the IoSession which
> contains pending sent requests.
>
> *Note: messageSent* just means it was written to the OS socket, no
> guarantee that it made it anywhere.
>
> On Thu, Feb 14, 2019 at 11:22 AM Emmanuel Lécharny 
> wrote:
>
>> Let's be clear:
>>
>> - reads will result in a messageReceived event and you will be able to
>> process it and do whatever you want to do in your IoHandler, as soon as
>> the message is complete (ie, if the message is read in chunks, you will
>> only receive a messageReceived event once the message is complete)
>>
>> - writes is a bit different: you do write, and you either decide to do
>> something *after* the write has been executed, but without any guarantee
>> that the message has been fully sent to the remote host, *or* you
>> execute your action when you receive the messageSent event - which means
>> you have written a message, *and* it has been fully sent to the remote
>> peer.
>>
>> In any case, you are still in the IoProcessor thread, so you can do
>> whatever you want (except that if you decide to do what you want to do
>> when receiving the messageSent, it will be executed later on, only if
>> the message has been fully sent.
>>
>>
>> On 14/02/2019 16:56, kevintjuh93 wrote:
>> > It's for a game server where actions need to be synchronized with
>> read/write
>> > in order to make sure everything is done in order. Not everything is
>> > executed from read/write methods, so I cannot ensure a message is being
>> > received/sent and look at some queue for this.
>> >
>> > That's why I want to execute something on the same thread a read/write
>> event
>> > is done for a specific session.
>> >
>> >
>> > Jonathan Valliere-3 wrote
>> >> I just read the last email Kevin wrote.
>> >>
>> >> Kevin, if you could execute something on the IO processor thread; you
>> >> understand that It would be a deferred action that could only happen
>> after
>> >> the IO processor is done?  Maybe you could explain the reason why you
>> want
>> >> to do this?
>> >>
>> >> On Thu, Feb 14, 2019 at 10:40 AM Emmanuel Lécharny 
>> >> elecharny@
>> >> 
>> >> wrote:
>> >>
>> >>> I still don't get it.
>> >>>
>> >>> Your IoHandler will be called everytime an event occurs (message
>> >>> received, message written, session created/closed/idling, exception).
>> >>> You have the opportunity to execute some action at this moment.
>> >>>
>> >>>
>> >>> Beside that, I don't see a use case. I'm probably missing something...
>> >>> Unless what you want to do is to have another session to be called
>> while
>> >>> processing an event, using the thread you are in ?
>> >>>
>> >>>
>> >>> On 14/02/2019 16:21, Jonathan Valliere wrote:
>> >>>> There are some examples in the unit tests which accomplish this by
>> >>> creating
>> >>>> a Client and Server connection.  I don't believe there is a true
>> >>> loopback
>> >>>> implementation in Mina without going through the OS networking.
>> >>>>
>> >>>> On Thu, Feb 14, 2019 at 10:16 AM kevintjuh93 
>> >> kevin_kal@
>> >> 
>> >>> wrote:
>> >>>>> Hi guys,
>> >>>>>
>> >>>>> What I mean is that I want a way to execute something for an
>> IoSession
>> >>> in
>> >>>>> the same thread the I/O events run. I figured a good way would be to
>> >>> 'fake'
>> >>>>> an incoming message, called a loopback packet. Like write a message
>> to
>> >>>>> 'yourself'.
>> >>>>>
>> >>>>> I rather like to avoid using an ExecutorFilter or a lock.
>> >>>>>
>> >>>>>
>> >>>>>
>> >>>>> --
>> >>>>> Sent from:
>> >>>>>
>> >>>
>> http://apache-mina.10907.n7.nabble.com/Apache-MINA-Developer-Forum-f6809.html
>> >
>> >
>> >
>> >
>> > --
>> > Sent from:
>> http://apache-mina.10907.n7.nabble.com/Apache-MINA-Developer-Forum-f6809.html
>>
>

Re: MINA filter chains breaking under stress

[Jonathan Valliere]

Upon a quick review of your email, I have the following notes.

The filterchain is not thread safe. You have to write threadsafe filters.
You must also make your encoded message system threadsafe.  Just as with
SSL the actual encoding process must occur within a synchronized block.

As far as ProtocolCodecFilter goes, I will have to look into that.

On Mon, Jan 14, 2019 at 8:37 AM Max Larsson 
wrote:

> I don't yet have a test case for the issues, but i have a description from
> a
> colleague in German under which circumstances this behavior occurs using
> Apache Mina 2.0.19:
>
> - There must be multiple Threads (at least two) calling nearly
> simultaneously session.write and there must be multiple ProtocolCodecFilter
> in the FilterChain, whereas the first encoder doesn't produce messages of
> type IoBuffer
>
> - For example if thread A calls session.write and the first
> ProtocolCodecFilter has processed the write request, there will be an
> encoded message in the WriteRequestQueue of the session. Lets assume that
> it
> is an byte array.
>
> - Now if another thread B even called session.write in such circumstances
> that the call of thread B has passed all FilterChain and it is in progress
> to call AbstractPollingIoProcessor$Processor.flushNow(), the encoded
> message
> from thread A (which still is in process to complete FilterChain) result in
> the known error "Don't know how to handle message of type 'xyz'. Are you
> missing a protocol encoder?'
>
> - all following calls to session.write (regardless which thread) will even
> produce the error, because the following snippet (somewhere in mina):
>
>   // Check for pending writes.
>   req = session.getCurrentWriteRequest();
>   if (req == null) {
> req = writeRequestQueue.poll(session);
> if (req == null) {
>   break;
> }
> session.setCurrentWriteRequest(req);
>   }
>
> always return the same incomplete encoded message at line 'req =
> session.getCurrentWriteRequest();',
> because it isn't removed from initial first faulty processing.
>
>
> I hope someone understand whats going on, and if someone could tell me, if
> it is a bug in mina or a faulty
> usage of mina.
>
> best regards
>
> Max Larsson
>
>
>
> --
> Sent from:
> http://apache-mina.10907.n7.nabble.com/Apache-MINA-User-Forum-f31345.html
>
-- 

CONFIDENTIALITY NOTICE: The contents of this email message and any
attachments are intended solely for the addressee(s) and may contain
confidential and/or privileged information and may be legally protected
from disclosure.

Re: Getting "Too many open files" warnings11

[Jonathan Valliere]

Do you have idle configured to close idle connections?  This is both
configured in Mina and can be set globally in Linux.

On Wed, Dec 5, 2018 at 7:26 AM Krishan Babbar 
wrote:

> Hi Jonathan,
>
> I found following counts in /proc/$PID/fd folder.
>
> Socket: 7807
> Pipe: 76
> anon_inode:[eventpoll]: 38
> /usr/lib/jvm/jdk1.8.0_92/ : 5
> /appl/myapplicationfolder : 6
>
> Thanks & Regards,
> Krishan Babbar
>
>
> 
> Disclaimer: This message and the information contained herein is
> proprietary and confidential and subject to the Tech Mahindra policy
> statement, you may review the policy at
> http://www.techmahindra.com/Disclaimer.html externally
> http://tim.techmahindra.com/tim/disclaimer.html internally within
> TechMahindra.
>
> ===
>
> -Original Message-
> From: Jonathan Valliere 
> Sent: Wednesday, December 5, 2018 4:34 PM
> To: users@mina.apache.org
> Cc: Atul Kandhari 
> Subject: Re: Getting "Too many open files" warnings11
>
> Can you find out the open file by type instead of just the number?  How
> many are storage files, how many are sockets?
>
> On Wed, Dec 5, 2018 at 2:10 AM Stefan Magnus Landrø <
> stefan.lan...@gmail.com>
> wrote:
>
> > Perform a heap dump to find out what’s going on
> >
> > Sendt fra min iPhone
> >
> > > 5. des. 2018 kl. 07:36 skrev Krishan Babbar
> > > > >:
> > >
> > > Any idea why file count is increasing day by day? Can we close any
> > socket/file handle manually?
> > >
> > > Date
> > >
> > > Time
> > >
> > > Open files count
> > >
> > > 2-Dec
> > >
> > > 16:08 PM
> > >
> > > 7047
> > >
> > > 3-Dec
> > >
> > > 8:29 AM
> > >
> > > 7209
> > >
> > > 3-Dec
> > >
> > > 14:45 PM
> > >
> > > 7289
> > >
> > > 3-Dec
> > >
> > > 16:00 PM
> > >
> > > 7308
> > >
> > > 3-Dec
> > >
> > > 16:11 PM
> > >
> > > 7310
> > >
> > > 3-Dec
> > >
> > > 18:01 PM
> > >
> > > 7330
> > >
> > > 3-Dec
> > >
> > > 20:00 PM
> > >
> > > 7358
> > >
> > > 3-Dec
> > >
> > > 22:00 PM
> > >
> > > 7394
> > >
> > > 4-Dec
> > >
> > > 8:25 AM
> > >
> > > 7501
> > >
> > > 4-Dec
> > >
> > > 11:09 AM
> > >
> > > 7543
> > >
> > > 4-Dec
> > >
> > > 12:00 PM
> > >
> > > 7551
> > >
> > > 4-Dec
> > >
> > > 14:00 PM
> > >
> > > 7594
> > >
> > > 4-Dec
> > >
> > > 16:00 PM
> > >
> > > 7614
> > >
> > > 4-Dec
> > >
> > > 18:00 PM
> > >
> > > 7635
> > >
> > > 4-Dec
> > >
> > > 20:00 PM
> > >
> > > 7668
> > >
> > > 4-Dec
> > >
> > > 22:00 PM
> > >
> > > 7689
> > >
> > >
> > >
> > > Thanks & Regards,
> > > Krishan Babbar
> > >
> > >
> > ==
> > ==
> > > Disclaimer: This message and the information contained herein is
> > proprietary and confidential and subject to the Tech Mahindra policy
> > statement, you may review the policy at
> > http://www.techmahindra.com/Disclaimer.html externally
> > http://tim.techmahindra.com/tim/disclaimer.html internally within
> > TechMahindra.
> > >
> > ==
> > =
> > >
> > > From: Krishan Babbar
> > > Sent: Monday, December 3, 2018 11:27 AM
> > > To: users@mina.apache.org
> > > Cc: Atul Kandhari 
> > > Subject: RE: Getting "Too many open files" warnings11
> > >
> > >
> > > Thanks Jonathan,
> > >
> > >
> > >
> > > Ok, but I

Re: Getting "Too many open files" warnings11

[Jonathan Valliere]

Mina is patched to prevent that exception from occurring in a cycle. That
was a long time ago, your version is patched.   You probably noticed a
message saying the acceptor was sleeping.

Do you have any kind of testing utility to create fake users for your
application?  I would recommend lowering the nlimit and trying to reproduce
entirely artificially.   There shouldn’t be any correlation to

On Fri, Nov 30, 2018 at 8:39 AM Krishan Babbar 
wrote:

> Hi Jonathan,
>
> Version - 2.0.16
> Java Application process was using around 1% of CPU and 3.4% of Memory.
>
> After mitigate, were you able to provide the solution?
> I mean what would be the solution if you know?
>
> Thanks & Regards,
> Krishan Babbar
>
> -----Original Message-
> From: Jonathan Valliere 
> Sent: Friday, November 30, 2018 6:56 PM
> To: users@mina.apache.org
> Cc: Atul Kandhari 
> Subject: Re: Getting "Too many open files" warnings11
>
> What version of Mina are you using? We added code to mitigate this
> specific exemption a while back.
>
> Do you know how much cpu Java was consuming at that point in time?
>
> On Fri, Nov 30, 2018 at 8:24 AM Krishan Babbar <
> kb00449...@techmahindra.com>
> wrote:
>
> > Thanks Stefan,
> >
> > It would be great if you could answers other questions too.
> > -What do you think, why application stops receiving packets from devices?
> > Is it only due to exception "Too many open files" or it can be due to
> > any other reasons as well? Please guide.
> > -Is my MINA configuration fine shown in below Java code? Please
> > suggest if I need to fine tune it more.
> >
> > Thanks & Regards,
> > Krishan Babbar
> >
> >
> >
> > ==
> > ==
> > Disclaimer: This message and the information contained herein is
> > proprietary and confidential and subject to the Tech Mahindra policy
> > statement, you may review the policy at
> > http://www.techmahindra.com/Disclaimer.html externally
> > http://tim.techmahindra.com/tim/disclaimer.html internally within
> > TechMahindra.
> >
> > ==
> > =
> >
> > -Original Message-
> > From: Stefan Magnus Landrø 
> > Sent: Friday, November 30, 2018 5:59 PM
> > To: users@mina.apache.org
> > Cc: Atul Kandhari 
> > Subject: Re: Getting "Too many open files" warnings11
> >
> >
> > https://unix.stackexchange.com/questions/108603/do-changes-in-etc-secu
> > rity-limits-conf-require-a-reboot
> >
> >
> > Sendt fra min iPhone
> >
> > > 30. nov. 2018 kl. 11:17 skrev Krishan Babbar
> > > > >:
> > >
> > > Dear All,
> > >
> > > I am working on an IoT project. I have created a Java application
> > > using
> > MINA library. All the devices are connecting to this Java application
> > using TCP protocol. I am getting around 3000 packets per hour from
> > about 50 devices as of now and it will increase soon.
> > >
> > > I am facing one issue on production environment. After running say
> > > 1-2
> > days, my application stops receiving packets from all the devices but
> > application process keep running. To make it working again, I need to
> > restart my Java application. I noticed following warnings in logs.
> > > Warnings
> > > [NioSocketAcceptor-3] WARN
> > > org.apache.mina.util.DefaultExceptionMonitor
> > 30/11/2018 05:25:40 - Unexpected exception.
> > > java.io.IOException: Too many open files
> > >at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
> > >at sun.nio.ch
> > .ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:422)
> > >at sun.nio.ch
> > .ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:250)
> > >at
> > org.apache.mina.transport.socket.nio.NioSocketAcceptor.accept(NioSocke
> > tAcceptor.java:194)
> > >at
> > org.apache.mina.transport.socket.nio.NioSocketAcceptor.accept(NioSocke
> > tAcceptor.java:51)
> > >at
> > org.apache.mina.core.polling.AbstractPollingIoAcceptor$Acceptor.proces
> > sHandles(AbstractPollingIoAcceptor.java:544)
> > >at
> > org.apache.mina.core.polling.AbstractPollingIoAcceptor$Acceptor.run(Ab
> > stractPollingIoAcceptor.java:484)
> > >at
> > org.apache.mina.util.Name

Re: Getting "Too many open files" warnings11

[Jonathan Valliere]

What version of Mina are you using? We added code to mitigate this specific
exemption a while back.

Do you know how much cpu Java was consuming at that point in time?

On Fri, Nov 30, 2018 at 8:24 AM Krishan Babbar 
wrote:

> Thanks Stefan,
>
> It would be great if you could answers other questions too.
> -What do you think, why application stops receiving packets from devices?
> Is it only due to exception "Too many open files" or it can be due to any
> other reasons as well? Please guide.
> -Is my MINA configuration fine shown in below Java code? Please suggest if
> I need to fine tune it more.
>
> Thanks & Regards,
> Krishan Babbar
>
>
>
> 
> Disclaimer: This message and the information contained herein is
> proprietary and confidential and subject to the Tech Mahindra policy
> statement, you may review the policy at
> http://www.techmahindra.com/Disclaimer.html externally
> http://tim.techmahindra.com/tim/disclaimer.html internally within
> TechMahindra.
>
> ===
>
> -Original Message-
> From: Stefan Magnus Landrø 
> Sent: Friday, November 30, 2018 5:59 PM
> To: users@mina.apache.org
> Cc: Atul Kandhari 
> Subject: Re: Getting "Too many open files" warnings11
>
>
> https://unix.stackexchange.com/questions/108603/do-changes-in-etc-security-limits-conf-require-a-reboot
>
>
> Sendt fra min iPhone
>
> > 30. nov. 2018 kl. 11:17 skrev Krishan Babbar  >:
> >
> > Dear All,
> >
> > I am working on an IoT project. I have created a Java application using
> MINA library. All the devices are connecting to this Java application using
> TCP protocol. I am getting around 3000 packets per hour from about 50
> devices as of now and it will increase soon.
> >
> > I am facing one issue on production environment. After running say 1-2
> days, my application stops receiving packets from all the devices but
> application process keep running. To make it working again, I need to
> restart my Java application. I noticed following warnings in logs.
> > Warnings
> > [NioSocketAcceptor-3] WARN  org.apache.mina.util.DefaultExceptionMonitor
> 30/11/2018 05:25:40 - Unexpected exception.
> > java.io.IOException: Too many open files
> >at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
> >at sun.nio.ch
> .ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:422)
> >at sun.nio.ch
> .ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:250)
> >at
> org.apache.mina.transport.socket.nio.NioSocketAcceptor.accept(NioSocketAcceptor.java:194)
> >at
> org.apache.mina.transport.socket.nio.NioSocketAcceptor.accept(NioSocketAcceptor.java:51)
> >at
> org.apache.mina.core.polling.AbstractPollingIoAcceptor$Acceptor.processHandles(AbstractPollingIoAcceptor.java:544)
> >at
> org.apache.mina.core.polling.AbstractPollingIoAcceptor$Acceptor.run(AbstractPollingIoAcceptor.java:484)
> >at
> org.apache.mina.util.NamePreservingRunnable.run(NamePreservingRunnable.java:64)
> >at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> >at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> >at java.lang.Thread.run(Thread.java:745)
> >
> > Due to above warnings in logs, yesterday, after doing some Google, I
> changed file limits in "/etc/security/limits.conf" file to 10 as shown
> below, but I did not restart my Java application.
> > *   hardnofile  10
> > *   softnofile  10
> >
> > Today. I got the same warnings again and I was not getting any packet
> from any of the devices. So I restarted my Java application and devices
> started sending packets again.
> > Now, my questions are
> >
> > -  After application stopped receiving packets, I restarted my
> Java application today. Will it consider new file limits i.e. 10 and
> not give Exceptions again? Or do I need to reboot machine as well?
> >
> > -  What do you think, why application stops receiving packets
> from devices? Is it only due to exception "Too many open files" or it can
> be due to any other reasons as well? Please guide.
> >
> > -  Is my MINA configuration fine shown in below Java code?
> Please suggest if I need to fine tune it more.
> >
> >
> > Below is given our code for MINA configuration.
> > import java.io.IOException;
> > import java.net.InetSocketAddress;
> > import org.apache.log4j.Logger;
> > import org.apache.mina.core.service.AbstractIoAcceptor;
> > import org.apache.mina.core.session.IdleStatus;
> > import org.apache.mina.filter.logging.LoggingFilter;
> > import org.apache.mina.transport.socket.nio.NioDatagramAcceptor;
> >
> > import 

Re: SFTP server whitelist

[Jonathan Valliere]

Unless you need business logic in the whitelist, you could use the Linux
firewall.   Otherwise you could take a look at
org.apache.mina.filter.firewall.*

On Fri, Oct 5, 2018 at 7:22 AM TuanNN  wrote:

> Currently, I'm using sshd-core and mina-core libraries to create sftp
> server.
> I want to create a white list ip filter to restrict the connection for my
> server.
>
> I implement the method check accessable at class MyPasswordAuthenticate
>
> @Override
> public boolean authenticate(String username, String password,
> ServerSession session)
> {
> if (isAllowAccess(session))
> {
> return checkAuthenticate(username, password, session);
> }
> else
> {
> return false;
> }
> }
>
> However, I think this is not a good solution, I think i should check allow
> access when the server start create a connection or start handle request
> from client (For example at I/O Acceptor or the FilterChain) and return the
> message code for access denied.
>
> Is this impossible and how can i implement it. Or any solution to implement
> whitelist IP for my server.
>
> Thank you!
>
>
>
> --
> Sent from:
> http://apache-mina.10907.n7.nabble.com/Apache-MINA-User-Forum-f31345.html
>

Re: Does IoSession.write() do buffering?

[Jonathan Valliere]

There should be no aggregation, especially for Datagrams.

On Tue, Sep 18, 2018 at 1:57 PM Raghavendra Balgi  wrote:

> Hey Krishan, You should probably consider using a CodecFilter as described
> here -
>
> https://mina.apache.org/mina-project/userguide/ch9-codec-filter/ch9-codec-filter.html
>
> On Tue, Sep 18, 2018 at 11:11 PM Krishan Babbar <
> kb00449...@techmahindra.com>
> wrote:
>
> > Hi All,
> >
> > We are working on an IoT project and using "org.apache.mina" library
> > (Version - 2.0.16) in our gateway adaptor (GA - Java Code).
> > Device connects to our GA and GA send acknowledgement back to device for
> > each message.
> > We are using multi-threading for handling each packet from a device(s).
> It
> > is also possible to have multiple messages in a single packet from
> device.
> > It would be based on some delimiter.
> >
> > E.g. Following packet is having 5 messages. Here delimiter is "Header"
> > HeaderMsg1HeaderMsg2HeaderMsg3HeaderMsg4HeaderMsg5
> >
> > Now Device is expecting 5 Acknowledgements one at a time like given below
> > ACK1
> > ACK2
> > ACK3
> > ACK4
> > ACK5
> >
> > But we found following acknowledgements on device side. It is combining 2
> > or more ACKs sometimes.
> > ACK1ACK2
> > ACK3
> > ACK4ACK5
> >
> > Our code is given below. Acknowledgement code is highlighted. Kindly
> > review and let us know what are we doing wrong?
> > How does "session.write()" work? Will it not write immediately? I mean is
> > it using some buffer and appending multiple ACKs before sending message
> > back to device?
> >
> > Kindly suggest the solution.
> >
> >
> > public abstract class AbstractListener {
> > protected static final int BUFFER_SIZE = 1024;
> > public void init(String portKey) throws IOException {
> > AbstractIoAcceptor acceptor =
> > initInternal();
> > acceptor.setHandler(new
> RequestHandler());
> >
> > acceptor.getFilterChain().addLast("logger", new LoggingFilter());
> >
> > acceptor.getSessionConfig().setReadBufferSize(BUFFER_SIZE);
> >
> > acceptor.getSessionConfig().setIdleTime(IdleStatus.BOTH_IDLE, 10);
> > if (acceptor instanceof
> > NioDatagramAcceptor) {
> > ((NioDatagramAcceptor)
> > acceptor).getSessionConfig().setReuseAddress(true);
> > }
> > acceptor.bind(new
> >
> InetSocketAddress(Integer.parseInt(ServerConfigLoader.getInstance().getProperty(portKey;
> > }
> > }
> >
> > public class RequestHandler extends IoHandlerAdapter {
> > public RequestHandler() {
> > super();
> > }
> >
> > @Override
> > public void exceptionCaught(IoSession session, Throwable
> > cause) throws Exception {
> > LOGGER.error("Exception caught in
> > RequestHandler.exceptionCaught()", cause);
> > session.closeNow();
> > }
> >
> > @Override
> > public void messageReceived(IoSession session, Object
> > message) throws Exception {
> > try {
> > SocketAddress
> > remoteAddress = session.getRemoteAddress();
> > String msg =
> > ((IoBuffer)message).getHexDump().replaceAll(" ", "");
> > String strArray[] =
> > remoteAddress.toString().split(":");
> >
> > ThreadAllocator.getInstance().allocateThread(session, msg);
> > } catch (Exception e) {
> > throw e;
> > }
> > }
> > }
> >
> > public class ThreadAllocator {
> >
> > /** The thread pool. */
> > private ExecutorService executorService = Executors
> >
> >
> .newWorkStealingPool(Integer.parseInt(ServerConfigLoader.getInstance().getProperty("threadPoolSize")));
> >
> > public void allocateThread(IoSession session, String
> > message) {
> > executorService.execute(new
> > HelperThread(null,session, message));
> > }
> > }
> >
> >
> > public class HelperThread implements Runnable {
> > private String message;
> > private IoSession session;
> > private SocketChannel socketSession;
> >
> > public HelperThread(SocketChannel socketSession,
> IoSession
> > session, String message) {
> > if (session != null) {
> > this.session = session;
> >
> > this.session.getConfig().setBothIdleTime(1800);
> > } else if (socketSession != null) {
> > 

Re: current Apache MINA state.

[Jonathan Valliere]

Mina 3 will be renamed Mina Future, has no ETA, and might have limited
backwards compatibility with older Mina. I would suggest updating to
support the current 2.X Mina branch.  It is actively supported.

On Thu, Jul 12, 2018 at 9:01 PM Pablo Caballero  wrote:

> Hi community.
>
> Is the MINA framework "lifecycle" active?
>
> Is there an estimated release date for MINA 3?
>
> I have to migrate several software "pieces" builded above an old MINA
> version. I would prefer to continue using MINA but in depends on MINA 3
> being released or not and the project "aliveness"
>
> Thanks in advance
>
> Best regards
>

Re: Maven Plugin Issue FTP Server

[Jonathan Valliere]

Looks like mvn no longer appreciated multiple  tags and using
 inside ; anyway, its fixed and builds locally
now.  Putting all the  tags inside of the
 fixes the problem.

On Mon, May 7, 2018 at 2:53 PM, Emmanuel Lécharny <elecha...@gmail.com>
wrote:

> (the attachement has been dropped. This is standard behaviour of the ASF
> mail server)
>
> Le 07/05/2018 à 18:49, Jonathan Valliere a écrit :
> > I'm trying to make the security fix for the FTP server but I am seeing an
> > issue when building the maven project.
> >
> > apache-rat-plugin:0.12:check (5 errors)
>
>
> Simply ignore those errors ('resolve later')
>
> --
> Emmanuel Lecharny
>
> Symas.com
> directory.apache.org
>
>

Maven Plugin Issue FTP Server

[Jonathan Valliere]

I'm trying to make the security fix for the FTP server but I am seeing an
issue when building the maven project.

apache-rat-plugin:0.12:check (5 errors)

I've attached a screenshot.

Do I need to install this plugin?  I'm not sure what to do here.

Re: Detecting non-idle but non-active either sessions

[Jonathan Valliere]

Are you referring to TCP sockets or some kind of protocol over TCP?  In
MINA idle and non-active are same thing.  Either data is received on the
socket or not.  If data isn’t received and the idle timer is configured,
then idle events are dispatched and the IoHandler can use that event to
take the next step; such as closing the session.

On Sun, Mar 25, 2018 at 7:55 PM, Juan Palacios 
wrote:

> Hi
> I was hoping I could get some support analysing a change I'd like to make
> to our MINA integration.
>
> Our application limits the total number of sessions it will open
> concurrently to keep high load from hurting our performance.
>
> However we've noticed recently that some of our users are configuring
> ControlPersist + KeepAlive settings for their clients. This keeps sessions
> alive for extended periods of time (we've seen sessions going for days).
>
> I was wondering if there was a way of detecting sessions which are
> *only* producing
> keep alive traffic and terminating them after a certain period of time. The
> goal is to prevent these users from exhausting the pool of available
> sessions.
>
> Thanks
>

Re: Too many ports open

[Jonathan Valliere]

It’s not a port it’s a File Descriptor.  The socket events are handled by a
selector which uses 1 File Descriptor. There is usually one per processor.
  You can manually set the number of socket processors.

On Fri, Mar 16, 2018 at 11:49 AM Simo Chiegang, Boris Arthur <
boris.s...@heidelberg.com> wrote:

> Hi,
>
> I made a little test:
>
> test()
> {
> NioSocketConnector connector = new NioSocketConnector();
> Connector.dispose();
> }
>
> When I debug through this code, I can see that the instantiation of the
> connector create exactly 12 ports. Why?. It depends on the number of
> processors on the machine?
> This is for my system critical.
>
> Thanks!
>
> 
>
> Confidentiality note:
> The information in this email and any attachment may contain confidential
> and proprietary information of Heidelberger Druckmaschinen AG and/or its
> affiliates and may be privileged or otherwise protected from disclosure. If
> you are not the intended recipient, you are hereby notified that any
> review, reliance or distribution by others or forwarding without express
> permission is strictly prohibited and may cause liability. In case you have
> received this message due to an error in transmission, we kindly ask you to
> notify the sender immediately and to delete this email and any attachment
> from your system.
>

Re: Data throttling for large window sizes

[Jonathan Valliere]

You can measure the outbound queue length, correct?  Limit it to X number
of Write Requests then throttle it yourself by pausing the inbound.

On Wed, Aug 30, 2017 at 2:55 AM, Juan Palacios 
wrote:

> Hi,
> Today I created DIRMINA-1070
> , but I wanted to
> reach
> out to the list to see if anyone could provide any insight into how we can
> handle this issue in the mean time.
>
> As a short description, when a client connects to our MINA server and
> specifies a large window size (TortoiseGit for instance uses 2GB) it
> results in an OutOfMemoryError. Apparently because the
> DefaultWriteRequestQueue uses an unbounded queue, the server writes data
> without any throttling and if the client is unable to keep up the heap is
> eventually consumed and the error happens.
>
> I've looked at implementing an IoFilter to try and handle filterWrite and
> messageSent (to make sure only a fixed maximum number of bytes are ever
> in-flight) but apparently the calls might be unbalanced because a lot more
> bytes came  through the messageSent call than they did through the
> filterWrite call.
>
> I'll appreciate any help you can provide.
>
> Cheers!
> Juan
>

Re: MINA3.0 recommended

[Jonathan Valliere]

I don't have the code in front of me, but are you saying that the
modification of the Write Queue is no longer concurrent?  Possibly the
Write Queue is a concurrent data structure and the synchronize mechanism is
no longer required.  It seems odd that someone would remove concurrency
from that critical section.

On Thu, Aug 17, 2017 at 11:41 PM, 胡阳  wrote:

> Hi guys:
>
>  I read the source code of MINA3.0M2. The style of the code is
> very good, the structure is clear, the design is concise and efficient,
> especially the use of Selector is unexpected. However, the
> enqueueWriteRequest method and the processWrite method in the
> AbstractNioSession are somewhat flawed.
>
> I see the source code in the enqueueWriteRequest method was originally
> "synchronized (writeQueue)", but was commented out, personal speculation
> may be the author feel that this treatment will affect performance.
>
> My approach is to use CAS to ensure memory visibility and atomic, see I
> see the startSync, finishSync method, feeling that this may be more secure
> after some of the performance will not lose too much.
>
> A little personal humble opinion.
>

Re: Address already in use when restarting server

[Jonathan Valliere]

If a remote tcp connection is open to that port then TCP will always say 
Address already in use. Set the Reuse Address flag and it will work fine. 

Sent from my iPhone

> On Oct 19, 2016, at 5:58 AM, Emmanuel Lécharny  wrote:
> 
> 
> 
>> Le 19/10/16 à 11:27, Michał Gałuszka a écrit :
>> Dear MINA users.
>> 
>> I’m using MINA server acceptor in my Java application that needs to be 
>> restarted from time to time.
>> When I stop and start that application „too fast” I always get an exception 
>> when starting and trying to bind:
>> 
>> java.io.IOException: Error while binding on 0.0.0.0/0.0.0.0:8245
>> original message : Address already in use
>> 
>> It must mean that I’m not closing all connections correctly upon application 
>> termination.
>> I have tried many things but nothing helped.
>> 
>> When terminating application I’m closing all sessions (most often there is 
>> only one session open), wait for the session to close gracefully and then I 
>> fire 
>> 
>> acceptor.dispose(true);
>> 
>> I’m not calling acceptor.unbind() because it is already called inside 
>> dispose(). Right?
>> 
>> After few minutes I can always start my application and then
>> acceptor.bind(new InetSocketAddress(port));
>> will work correctly not throwing any Exception.
> This is inherent to the way TCP works. You can have a look at
> http://hea-www.harvard.edu/~fine/Tech/addrinuse.html.
> 
> You can try to setup the SO_REUSEADRESS flag to mitigate the problem,
> using the setReuseAddress( true ) method on your service.
> 

Re: About apache MINA

[Jonathan Valliere]

That's pretty simple. In Linux set the file descriptor limit to something small 
like 500. Then try to open 1000 connections remotely.   It spins out of control 
because accept() fails due to lack of FDs. My code sleeps the acceptor thread 
when that happens.  This behavior has nothing to do with Java and is a function 
of the underlying OS. 

Sent from my iPhone

> On Jun 10, 2016, at 9:41 AM, Emmanuel Lécharny <elecha...@gmail.com> wrote:
> 
>> Le 10/06/16 à 13:49, Jonathan Valliere a écrit :
>> I've been working with NIO directly for many years and I haven't seen this 
>> issue in a very long time. The selector will spin out of control if there 
>> are no more file descriptors but that's not the fault of NIO. 
>> 
>> What OS is the issue appearing on?  What version of Java?
> 
> This has been seen on various OS, and various Java version. Check
> http://bugs.java.com/view_bug.do?bug_id=6403933.
> 
> If the underlying JVM is not anymore impacted by this problem, then the
> existing code will never be a problem.
> 
> Now, if you can show us some code that makes teh selector spin out of
> control, due to the lack of file descriptors, I would be pleased to test
> it and take care of this specific use case.
> 

Re: About apache MINA

[Jonathan Valliere]

I've been working with NIO directly for many years and I haven't seen this 
issue in a very long time. The selector will spin out of control if there are 
no more file descriptors but that's not the fault of NIO. 

What OS is the issue appearing on?  What version of Java?

Sent from my iPhone

> On Jun 10, 2016, at 4:13 AM, Emmanuel Lécharny  wrote:
> 
>> Le 10/06/16 à 08:23, Claude Warren a écrit :
>> hu_yang3000 mentions 100%  CPU usage. 
> There *is* a 100% CPU usage bug due to some 10 years old bug in the NIO
> library. There is no way to get this fixed, even if this has been asked
> MANY times to Sun/Oracle.
> 
> This occurs because at some point, during some specific conditions, the
> selector is spinning, returning with a 0 value without waiting for the
> given timeout. Obviously, having 0 event to deal with, we reenter the
> select() immediately, and it loops forever, eating 100% of the CPU.
> 
> The workaround is to doom the selector and register all the keys on this
> new selector. That is what the code is doing.
> 
> You can try writing a test for that, but again, there is no
> deterministic ways to get the problem trigerred.
> 
> FTR, this issue did not only affect MINA, Netty was also affected (see
> https://github.com/netty/netty/issues/327). The solution was suggested
> by Jean-François Arcand, who was working on Grizzly
> 
> 

Re: Deadlock when using SslFilter and ProxyFilter together

[Jonathan Valliere]

Obviously something is wrong with the thread-safe operation the write stack.  
If it was thread safe then there is no need to ever have locks in the filter 
chain unless you have an executor filter. 

1027 exposes a bug in the lock routines of IoSession

Sent from my iPhone

> On Feb 9, 2016, at 12:48 PM, Emmanuel Lécharny  wrote:
> 
> Le 09/02/16 18:22, Jon V. a écrit :
>> Its my understanding that Read and Write operations are thread-safe and
>> should never cause deadlocks. (not sure why there are locks in those
>> filters)
>> 
>> Also, I don’t think SSL should be after Proxy.  You cannot initiate an SSL
>> session if you proxy the setup routine.  What is the goal in this setup?
>> 
>> Please send us the deadlock thread trace.(You can get it from VisualVM)
> 
> I think this is a pb that is already exposed in DIRMINA-1027.
> 
> I still have to find a couple of hours to analyse the code and find a
> solution.
>