RE: Proxy example memeory usage going high

[Guy Itzhaki]

Hi Emmanuel,
Thanks for the prompt reply. I appreciate it.

I conducted a test - I created a client that connects to a server through the 
proxy.
The client reads from a file and sends 10kb chunks. 
Using debugger, I noted that the server receives the data as it sent from the 
client. Maybe I'm mistaken but it means that message will NOT have to be fully 
received in order to be written to the server.

Unless I misinterpret what I'm seeing, it looks like the IoBuffer (from 
AbstractProxyIoHandler#messageReceived) does not get bigger 

This is the IoBuffer state during the data transfer (from 
AbstractProxyIoHandler#messageReceived):
HeapBuffer[pos=0 lim=1024 cap=1024: 30 36 2F 30 36 2F 31 37 20 31 39 3A 31 33 
3A 33...]
HeapBuffer[pos=0 lim=2048 cap=2048: 69 74 69 6E 67 20 66 6F 72 20 6E 65 78 74 
20 69...]
HeapBuffer[pos=0 lim=4096 cap=4096: 74 68 72 65 61 64 2D 38 37 5D 20 4D 65 73 
73 61...]
HeapBuffer[pos=0 lim=3072 cap=8192: 69 6F 50 72 6F 63 65 73 73 6F 72 2D 32 5D 
20 53...]
HeapBuffer[pos=0 lim=8192 cap=8192: 75 65 73 74 44 65 63 6F 64 65 72 20 2D 20 
56 65...]
HeapBuffer[pos=0 lim=2048 cap=16384: 73 73 61 67 65 20 6C 65 6E 3A 20 32 38 31 
0D 0A...]
HeapBuffer[pos=0 lim=10240 cap=16384: 73 73 61 67 65 73 20 6F 66 20 31 32 30 31 
31 34...]
HeapBuffer[pos=0 lim=10240 cap=16384: 6F 72 20 6F 72 69 67 69 6E 61 74 6F 72 20 
32 30...]
HeapBuffer[pos=0 lim=10240 cap=16384: 65 64 20 6D 65 73 73 61 67 65 20 6C 65 6E 
3A 20...]
HeapBuffer[pos=0 lim=10240 cap=16384: 74 69 6E 67 20 66 6F 72 20 6E 65 78 74 20 
69 6E...]

Hope this helps.

Many thanks,
Guy

-Original Message-
From: Emmanuel Lécharny [mailto:elecha...@gmail.com] 
Sent: Monday, June 12, 2017 3:43 PM
To: users@mina.apache.org
Subject: Re: Proxy example memeory usage going high

Hi,


Le 12/06/2017 à 11:31, Guy Itzhaki a écrit :
> Dear all,
>
> We are using the proxy example using mina 2.0.15. 
>
> At the beginning everything looks good, but when we tried to funnel a 
> 1GB file the Proxy memory went high till the proxy doesn't receive new 
> connections.
>
> This is strange because I was expecting that every buffer will be 
> cleared once it is sent, so the proxy will not consume too much memory.

This is a really rough implementation of a proxy. Typically, messages will have 
to be fully received in order to be written to the remote server, which means 
we hold it in memory until it's fully received. If you process 1Gb messages, 
then expect at least 1Gb of memory to be ate in the process (actually, expect a 
*lot* or reallocation during the message reception, as MINA will have to resize 
the buffer receiving the
data...)

In other words, you'll have to rework the example to fit your need.

--
Emmanuel Lecharny

Symas.com
directory.apache.org



smime.p7s
Description: S/MIME cryptographic signature

Proxy example memeory usage going high

[Guy Itzhaki]

Dear all,

We are using the proxy example using mina 2.0.15. 

At the beginning everything looks good, but when we tried to funnel a 1GB
file the Proxy memory went high till the proxy doesn't receive new
connections.

This is strange because I was expecting that every buffer will be cleared
once it is sent, so the proxy will not consume too much memory.

The AbstractProxyIoHandler#messageReceived() looks good to me but maybe I'm
missing something. 

 

Any ideas are highly appreciated.

Thanks

Guy



smime.p7s
Description: S/MIME cryptographic signature

Re: verbose logs in mina 2.0.13

[Guy Itzhaki]

Thank you very much Emmanuel for the prompt reply.
Sounds reasonable to wait one more week to get a better and more stable 
release. I'll wait for b15.

Thanks again.
Guy

Sent from iPhone, please excuse typos.

On 21 ? 2016, at 0:01, Emmanuel L?charny 
<elecha...@gmail.com<mailto:elecha...@gmail.com>> wrote:

Le 20/09/16 ? 19:55, Guy Itzhaki a ?crit :
Hi all,
I've upgraded mina from 2.0.9 to 2.0.13 and since then I keep getting this 
entry  in the log
20/09/16 20:20:00 WARN  [NioProcessor-2] IoProcessor - Create a new selector. 
Selected is 0, delta = 0

Is it a code left over or on purpose?

Yes, and it's a Warning, so you can get rid of it by setting the log
level to ERROR.

Now, if you switch to 2.0.14 which has been released 2 weeks ago, the
issue will be less frequent (actually, it should not happen at all),
because we have fixed a bug in this area (beside other few annying bugs).

I would also wait one more week, when 2.0.15 will be out, as it fixes
some other serious issue.


Side note : this log is dumped because there is a bug in NIO, that makes
the select() method returns immediately even if there is no active
selectionKey, leading in some very specific use cases to an infinite
loop, eating 100% of your CPU. In this case, we simply delete the
Selector and recreate a new one registring all the selectionKeys to this
new Selector. Pretty technical...

verbose logs in mina 2.0.13

[Guy Itzhaki]

Hi all,
I've upgraded mina from 2.0.9 to 2.0.13 and since then I keep getting this 
entry  in the log
20/09/16 20:20:00 WARN  [NioProcessor-2] IoProcessor - Create a new selector. 
Selected is 0, delta = 0

Is it a code left over or on purpose?

Guy

Re: Acceptor dispose block indefinitely

[Guy Itzhaki]

I experienced the same with Mina 2.0.9.
When I delve deeper into mina code I saw that the thread is looping endless 
until the sessions' collection is empty. And due to synchronization issue I 
guess it never does...
It was a long time ago and I'm not in front of my laptop right now so I cannot 
share more light.

Regards
Guy

Sent from iPhone, please excuse typos.

On 24 ? 2016, at 11:51, Simo Chiegang, Boris Arthur 
> wrote:

I Guys,

I have a server and I called to stop the server:

Acceptor.dispose(true);

But the call never come back. I don't know why:

  java.lang.Thread.State: TIMED_WAITING (parking)
   at sun.misc.Unsafe.park(Native Method)
   - parking to wait for  <0xf94bcd30> (a 
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
   at 
java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
   at 
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
   at 
java.util.concurrent.ThreadPoolExecutor.awaitTermination(ThreadPoolExecutor.java:1465)
   at 
org.apache.mina.core.service.AbstractIoService.dispose(AbstractIoService.java:307)
   at 
com.heidelberg.lvis.printdevice.sub.samba3.simu.DummySamba3JsonRpcServer.stop(DummySamba3JsonRpcServer.java:203)
   at 
test.heidelberg.lvis.printdevice.devices.TestUtilsSamba3.stopHardwareSimulation(TestUtilsSamba3.java:110)
   at 
test.heidelberg.lvis.printdevice.devices.Samba3HeadTemperatureTest.tearDown(Samba3HeadTemperatureTest.java:58)

I thinks, it must sure that this blocking call come back. Can you tell me in 
which case this can indefinitely block?.



Confidentiality note:
The information in this email and any attachment may contain confidential and 
proprietary information of Heidelberger Druckmaschinen AG and/or its affiliates 
and may be privileged or otherwise protected from disclosure. If you are not 
the intended recipient, you are hereby notified that any review, reliance or 
distribution by others or forwarding without express permission is strictly 
prohibited and may cause liability. In case you have received this message due 
to an error in transmission, we kindly ask you to notify the sender immediately 
and to delete this email and any attachment from your system.

RE: destroying mina gracefully

[Guy Itzhaki]

Thanks for the reply
I tried that, but disposing the ioService might block my thread as well 

Guy

-Original Message-
From: Emmanuel Lécharny [mailto:elecha...@gmail.com] 
Sent: Monday, January 04, 2016 2:56 PM
To: users@mina.apache.org
Subject: Re: destroying mina gracefully

Le 04/01/16 13:17, Guy Itzhaki a écrit :
> Hi all,
> I'm using mina 2.0.9 server in my application and I wonder what is the best 
> way to terminate mina?

Usually, by disposing the IoService :

/**
 * Releases any resources allocated by this service.  Please note that
 * this method might block as long as there are any sessions managed by
 * this service.
 */
void dispose();

destroying mina gracefully

[Guy Itzhaki]

Hi all,
I'm using mina 2.0.9 server in my application and I wonder what is the best way 
to terminate mina?
When my application gets signal to stop it invokes the acceptor.unbind method 
hoping this will do the trick, however I noted that sometimes the unbind() 
method got stuck and never return. As a result my application failed to shut 
down gracefully...

Any idea?

Thanks
Guy

RE: disposing acceptor does not terminate all the OrderedThreadPoolExecutor threads

[Guy Itzhaki]

I found the problem.
I had to destroy the threadpool explicitly just before I unbind and dispose the 
acceptor, apparently it leaves some threads hanging.
I would expect the dispose to do that for me... 

-Original Message-
From: Guy Itzhaki [mailto:g...@nextnine.com] 
Sent: Tuesday, May 12, 2015 6:40 PM
To: users@mina.apache.org
Subject: disposing acceptor does not terminate all the 
OrderedThreadPoolExecutor threads

Dear Forum,
I'm terminating my mina based server using acceptor.unbind(); 
acceptor.dispose(true);

However I noted that the executor's resources are not totally freed, some of 
its threads remains alive. This prevents my application from terminating 
gracefully.

This is the stack trace of one of the threads:
sun.misc.Unsafe.park(Native Method)
java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467)
org.apache.mina.filter.executor.OrderedThreadPoolExecutor$Worker.fetchSession(OrderedThreadPoolExecutor.java:730)
org.apache.mina.filter.executor.OrderedThreadPoolExecutor$Worker.run(OrderedThreadPoolExecutor.java:683)
java.lang.Thread.run(Thread.java:745)

Is there a way to overcome this? What am I doing wrong?
I'm using mina 2.0.9

Thanks a lot
Guy

disposing acceptor does not terminate all the OrderedThreadPoolExecutor threads

[Guy Itzhaki]

Dear Forum,
I'm terminating my mina based server using
acceptor.unbind();
acceptor.dispose(true);

However I noted that the executor's resources are not totally freed, some of 
its threads remains alive. This prevents my application from terminating 
gracefully.

This is the stack trace of one of the threads:
sun.misc.Unsafe.park(Native Method)
java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467)
org.apache.mina.filter.executor.OrderedThreadPoolExecutor$Worker.fetchSession(OrderedThreadPoolExecutor.java:730)
org.apache.mina.filter.executor.OrderedThreadPoolExecutor$Worker.run(OrderedThreadPoolExecutor.java:683)
java.lang.Thread.run(Thread.java:745)

Is there a way to overcome this? What am I doing wrong?
I'm using mina 2.0.9

Thanks a lot
Guy

Best practices for working with files + Anti-Virus (?)

[Guy Itzhaki]

Dear forum,
My question is not directly related to MINA but I hope that you experienced a 
similar issue before.

I used MINA 2.0.7 to write my server. The server receives files from dozens of 
clients and store these files on a designated folders on the hard disk to be 
processed later. The files are zipped and its size are vary from 1K to 1G, the 
majority of the files are rather small, 1-2K in size.

Everything works fine until the Anti-Virus gets into the picture - the files 
scanning literally destroy the server's performance...
The need for scanning incoming files is clear however do you have any idea of 
best practices for this scenario? configuring the AV differently? Maybe there 
is a special tool\AV to scan files more efficiently that the standards 
Anti-Viruses? Something else?

Any help\idea is highly appreciated.

Thanks
Guy

RE: Best practices for working with files + Anti-Virus (?)

[Guy Itzhaki]

We experienced the performance degradation with Symantec and McAfee.
Not to scan is not an option, I guess you mentioned that as a joke :-)
I was under the assumption that this is a general problem that everyone who 
have file server is dealing with... Please let me know what info are you 
missing.

10xs


-Original Message-
From: Alexander Christian [mailto:a...@root1.de] 
Sent: Wednesday, September 24, 2014 11:38 AM
To: users@mina.apache.org
Subject: Re: Best practices for working with files + Anti-Virus (?)

Am 24.09.2014 10:27, schrieb Guy Itzhaki:

 Everything works fine until the Anti-Virus gets into the picture - the files 
 scanning literally destroy the server's performance...
That's quite common :-)
 The need for scanning incoming files is clear however do you have any idea of 
 best practices for this scenario?
Don't scan :-)
 configuring the AV differently?
If you need to scan: Why not? Of course you can try to configure differently. 
It's your system. Check it out.
 Maybe there is a special tool\AV to scan files more efficiently that the 
 standards Anti-Viruses?
Yes, maybe there a more efficient virus file-scanners out there.
And what is a standard ati-virus?! Is there a Standard?
 Something else?
One thing:
How to you except help if you don't provide any details?!

HTTP encoder\filter for MINA server

[Guy Itzhaki]

Dear Forum,
I'm using Mina 2.0.7 for my server and I have implemented my own 
decoders\encoders to handle incoming messages.

I would like to handle http connections coming from web browsers.
It seems to me as a trivial issue and I wonder if anyone in this forum had 
implemented such a filter\decoder that he can share.

Thanks in advance,
Guy

mina 2.0.2 - NullPointerException in AbstractPollingIoProcessor

[Guy Itzhaki]

Dear forum,

I'm using mina 2.0.2 and once in a while I get this error in the
console:

Exception in thread pool-4-thread-1 java.lang.NullPointerException

  at
org.apache.mina.core.polling.AbstractPollingIoProcessor$Processor.run(Ab
stractPollingIoProcessor.java:1151)

  at
org.apache.mina.util.NamePreservingRunnable.run(NamePreservingRunnable.j
ava:64)

  at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(Unknown
Source)

  at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown
Source)

  at java.lang.Thread.run(Unknown Source)

 

 

In my server I have use a NioSocketAcceptor and at some point, although
it might not be relevant, I unbind() the acceptor, replace IoFilter with
a new one and then bind() it again.  

 

BTW, I didn't find line 1151 in AbstractPollingIoProcessor...

Any idea why this might happen?

Thanks in advance,

Guy

 

RE: mina 2.0.2 - NullPointerException in AbstractPollingIoProcessor

[Guy Itzhaki]

No.
I wonder if you experienced such an error before...

Emmanuel, I can debug it myself but the problem is that 
AbstractPollingIoProcessor is shorter than 1151 lines while in the exception 
trace we can see that line 1151 is the cause of the exception.

Can you assist on that?

Thanks,
Guy 
-Original Message-
From: Emmanuel Lécharny [mailto:elecha...@gmail.com] 
Sent: Thursday, March 29, 2012 3:21 PM
To: users@mina.apache.org
Subject: Re: mina 2.0.2 - NullPointerException in AbstractPollingIoProcessor

Le 3/29/12 2:15 PM, Guy Itzhaki a écrit :
 Dear forum,

 I'm using mina 2.0.2 and once in a while I get this error in the
 console:
Have you tried with the latest MINA version (ie 2.0.4) ?


-- 
Regards,
Cordialement,
Emmanuel Lécharny
www.iktek.com



Click here to report this message as SPAM: 
http://vsp.ateranetworks.com/ReportSpam.php?sid=13e0d5ff2545fe302e53c93159ba98e8_40d157ee85b38822031cc4adec725912
-- Powered by ATERA Networks --

RE: where to place the ExecutorFilter?

[Guy Itzhaki]

-Original Message-
From: Emmanuel Lécharny [mailto:elecha...@apache.org] 
Sent: Monday, February 14, 2011 2:36 AM
To: users@mina.apache.org
Subject: Re: where to place the ExecutorFilter?

On 2/13/11 3:23 PM, Guy Itzhaki wrote:
 Emmanuel, thanks for the response.

 1. The server processes incoming messages using the protocolCodecFilter. 
 Currently it is the protocolCodecFilter responsibility to read the ENTIRE 
 message and save it in the hard disk for future processing by the Handler. 
 Reading few hundred MBs is an expensive process and might take a while, so 
 dropping the first ExecutorFilter means that the NIOProcessor thread will 
 handle this (long) task. If I'm not mistaken this is a bad choice because I 
 have only few Processors so I'll easily cause my server to not 
 excepting\processing other requests. Am I right? If I'm, maybe a better 
 design for my server would be to move the (expensive) file reading to the 
 handler...
I think that it's probably better to move the processing of the huge 
incoming messages into the handler, yes. Now, considering that writing 
data on disk is 1000 times more expensive that processing them in memory 
(pulling this number out of my hat, but let's use this number for the 
sake of the demonstration), then your bottleneck will be the disk, not 
the threads : no matter how many threads you will use, you will still be 
bounded by how fast is your disk...
I'll move the reading of the file to the handler as you suggested. Note that my 
message composed of two: a meta data header and the file itself. I'll read the 
meta data in the ProtocolCodec and leave the file reading for the handler. My 
question then is how can I read the rest of the message from the handler? The 
messageReceived method has 2 params one is the session, I didn't find any 
example of reading directly from the session?

 2. In order to be efficient and consume as less memory as it can, the server 
 sends the big files using FileWithHeaderWriteFilter
This class is not a MINA class, AFAICT
   (which extends AbstractStreamWriteFilter) I've noted that even if I have an 
 ExecutorFilter in the chain Mina does not use its threads, instead it uses 
 NIOProcessor threads.
Ultimately, when the message has transited through the chain, it's 
queued into an IoProcessor, so adding an executor expecting that more 
threads will be used to send the data to the client is simply useless. 
Keep in mind that once the socket is opened, it's associated with a 
selector, and what is important is the thread having access to this 
selector. Sharing this selector through many threads would not help...

 (Again,) this is an issue as I would expect my server to process 
 simultaneously large number of files and not only few. How can I force Mina 
 to use other threads than the NIOProcessor threads for that?
Increase the number of IoProcessors. Again, at some point, writing data 
into sockets is fast, compared to the time it takes to send them on the 
physical network, so I'm not sure that having tens of IoProcessor helps 
a lot. An IoProcessor just read the queue, and inject the data into the 
socket, if it's ready for write. It does that repetitively, as soon as 
he select() method signal that at least one socket is ready. This is 
definitively not the bottleneck.

If you need to increase the throughput, then you have to think about 
having more than one server...

-- 
Regards,
Cordialement,
Emmanuel Lécharny
www.iktek.com



Click here to report this message as SPAM: 
http://vsp.ateranetworks.com/ReportSpam.php?sid=a955af5a804820ad90c87dc795acb80f_9e45831b18e75d23118e18ddb0b4646d
-- Powered by ATERA Networks --

jmx with mina 2.0.2

[Guy Itzhaki]

I'm trying to use jmx in my Mina based server, I ran the example andI
get the following error Exception in thread main
java.lang.NoClassDefFoundError: ognl/TypeConverter

 

Help will be appreciated.

Guy

RE: jmx with mina 2.0.2

[Guy Itzhaki]

Working, thanks

 

From: Guy Itzhaki 
Sent: Monday, February 14, 2011 11:41 AM
To: users@mina.apache.org
Subject: jmx with mina 2.0.2

 

I'm trying to use jmx in my Mina based server, I ran the example andI
get the following error Exception in thread main
java.lang.NoClassDefFoundError: ognl/TypeConverter

 

Help will be appreciated.

Guy

where to place the ExecutorFilter?

[Guy Itzhaki]

(Sorry for my pre mature email an hour ago - my mistake)

 

Hi all,

I'm using Mina 2.0.2 for my server which, in the nutshell, is a very
simple server that acts as a message server, clients connect to the
server in order to send and receive messages.

I expect up to 200 clients to connect to the server concurrently and the
messages might be big (up to few hundred Megs a message).

 

The ProtocolCodecFilter does an expensive process much more than the
handler, it is the ProtocolCodecFilter duty to read  persist incoming
messages, same goes for sending messages - the  codec is responsible to
read the message from the disk and send it.

 

I wonder where is best to place the ExecutorFilter? Is it before the
ProtocolCodecFilter or after? It seems to me that given what I explained
before I should use two executorFilters one to be placed before the
codec to digest incoming messages and the other after (for the sake of
the outgoing messages). 

 

chain.addLast(sslFilter, sslFilter)

chain.addLast(logger, new LoggingFilter())

chain.addLast(readPool, new ExecutorFilter(coreSize,
maxSize,keepAliveTime, TimeUnit.MILLISECONDS,
IoEventType.MESSAGE_RECEIVED))

chain.addLast(codec, protocolCodecFilter)

chain.addLast(writePool, new ExecutorFilter(coreSize,
maxSize,keepAliveTime, TimeUnit.MILLISECONDS, IoEventType.WRITE))

 

Can you please approve that using the above chaining is best for my
needs?

 

Emmanuel, I read once your reservation about placing the executor first,
you said: ...as the executor is the fist one in the filters chain, you
may not be able to deal with fragmented messages

Can you explain what did you mean fragmented messages? 

And in other post you said  Executor can leads to some issue if you are
under heavy load  can you elaborate on that? Taking into consideration
these comments of yours what is the best approach in case of heavy load?

 

Thanks in advance,

Guy

RE: where to place the ExecutorFilter?

[Guy Itzhaki]

Emmanuel, thanks for the response.

1. The server processes incoming messages using the protocolCodecFilter. 
Currently it is the protocolCodecFilter responsibility to read the ENTIRE 
message and save it in the hard disk for future processing by the Handler. 
Reading few hundred MBs is an expensive process and might take a while, so 
dropping the first ExecutorFilter means that the NIOProcessor thread will 
handle this (long) task. If I'm not mistaken this is a bad choice because I 
have only few Processors so I'll easily cause my server to not 
excepting\processing other requests. Am I right? If I'm, maybe a better design 
for my server would be to move the (expensive) file reading to the handler...
2. In order to be efficient and consume as less memory as it can, the server 
sends the big files using FileWithHeaderWriteFilter (which extends 
AbstractStreamWriteFilter) I've noted that even if I have an ExecutorFilter in 
the chain Mina does not use its threads, instead it uses NIOProcessor threads. 
(Again,) this is an issue as I would expect my server to process simultaneously 
large number of files and not only few. How can I force Mina to use other 
threads than the NIOProcessor threads for that?

Thanks,
Guy

-Original Message-
From: Emmanuel Lecharny [mailto:elecha...@gmail.com] 
Sent: Sunday, February 13, 2011 1:32 PM
To: users@mina.apache.org
Subject: Re: where to place the ExecutorFilter?

On 2/13/11 11:41 AM, Guy Itzhaki wrote:
 (Sorry for my pre mature email an hour ago - my mistake)

 The ProtocolCodecFilter does an expensive process much more than the
 handler, it is the ProtocolCodecFilter duty to read  persist incoming
 messages, same goes for sending messages - the  codec is responsible to
 read the message from the disk and send it.
ok, two things here :
- I guess the decoding itself is not costly, so no need of using an 
executor for decoding.
- You can add an executor to process the write operation only, but it 
should be placed *after* the codec filter, as the filters are executed 
in the reverse order when writing information.


 I wonder where is best to place the ExecutorFilter? Is it before the
 ProtocolCodecFilter or after? It seems to me that given what I explained
 before I should use two executorFilters one to be placed before the
 codec to digest incoming messages and the other after (for the sake of
 the outgoing messages).
You can't have 2 executors in the chain. That's by design. Probably bad, 
but you have to deal with it...



 chain.addLast(sslFilter, sslFilter)

 chain.addLast(logger, new LoggingFilter())

 chain.addLast(readPool, new ExecutorFilter(coreSize,
 maxSize,keepAliveTime, TimeUnit.MILLISECONDS,
 IoEventType.MESSAGE_RECEIVED))

 chain.addLast(codec, protocolCodecFilter)

 chain.addLast(writePool, new ExecutorFilter(coreSize,
 maxSize,keepAliveTime, TimeUnit.MILLISECONDS, IoEventType.WRITE))



 Can you please approve that using the above chaining is best for my
 needs?
If you remove the first executor, that should be fine.


 Emmanuel, I read once your reservation about placing the executor first,
 you said: ...as the executor is the fist one in the filters chain, you
 may not be able to deal with fragmented messages

 Can you explain what did you mean fragmented messages?
You have no guaranty whatsoever tht the data coming from a client will 
be received in one block. You can even expect to get the 
messageReceived() event be called for every single byte in the message 
(not likely to happen, but it's just to depict the worst possible 
situation). Generally speaking, if you have big messages (ie, more than 
a few Kb), you may get it split in more than one PDU. You have to deal 
with this, using a cumulativeProtocolDecoder, or writing your own logic 
to gather the bytes.
 And in other post you said  Executor can leads to some issue if you are
 under heavy load  can you elaborate on that? Taking into consideration
 these comments of yours what is the best approach in case of heavy load?
As soon as you spread the loads on many threads, at some point, you may 
creates thousands of threads if one of your clients send thousands of 
messages (let's say it's a DDOS). You probably have some ways to deal 
with such issues on a FW level, but it's important to keep in mind that 
such problems can arise.

However, it's more or less a warning, as every application open to the 
outside world can have the exact same issues.

-- 
Regards,
Cordialement,
Emmanuel Lécharny
www.iktek.com



Click here to report this message as SPAM: 
http://vsp.ateranetworks.com/ReportSpam.php?sid=20f993baa0bc4eff02b0f7eae7abc329_887ba19d853cfc2326e0e77f28a509c5
-- Powered by ATERA Networks --

SessionHandler - sessionClosed

[Guy Itzhaki]

All,

Is there a guarantee that once exceptionCaught(...) or sessionIdle(...)
methods are invoked the sessionClosed() method will be invoked too?

 

Thanks

Guy

Can't find SerialConnector

[Guy Itzhaki]

I'm trying to create a serial connector using the example you've
provided in the documentation:

IoConnector connector = new SerialConnector()

SerialAddress portAddress=new SerialAddress( ... );

 

But I'm missing the SerialConnector (I'm missing the SerialAddress as
well).

Where can I find these classes?

 

I'm using the stable GA package Mina 2.0.0

 

Thanks

RE: Can't find SerialConnector

[Guy Itzhaki]

Just found the source code, do you have a binary anywhere or I have to
build it myself?

 

From: Guy Itzhaki 
Sent: Monday, October 25, 2010 12:47 PM
To: users@mina.apache.org
Subject: Can't find SerialConnector

 

I'm trying to create a serial connector using the example you've
provided in the documentation:

IoConnector connector = new SerialConnector()

SerialAddress portAddress=new SerialAddress( ... );

 

But I'm missing the SerialConnector (I'm missing the SerialAddress as
well).

Where can I find these classes?

 

I'm using the stable GA package Mina 2.0.0

 

Thanks

RE: Can't find SerialConnector

[Guy Itzhaki]

I've already done it and I've placed the required dlls under my JRE, but it 
didn't helped.

Maybe I'm missing something but I thought the SerialConnector is part of mina 
project, so I expected to find it in org.apache.mina.transport.serial, right?

Regards,
Guy

-Original Message-
From: Emmanuel Lecharny [mailto:elecha...@gmail.com] 
Sent: Monday, October 25, 2010 12:58 PM
To: users@mina.apache.org
Subject: Re: Can't find SerialConnector

On 10/25/10 12:53 PM, Guy Itzhaki wrote:
 Just found the source code, do you have a binary anywhere or I have to
 build it myself?
You have to download the rxtx.jar from 
http://rxtx.qbang.org/wiki/index.php/Download


-- 
Regards,
Cordialement,
Emmanuel Lécharny
www.iktek.com



Click here to report this message as SPAM: 
http://vsp.ateranetworks.com/ReportSpam.php?sid=c525cca69cacacb3490b9ce46e5ac453_71362a30ac0ace8d791e6c4a1939169a
-- Powered by ATERA Networks --

RE: Possible synchronization issue

[Guy Itzhaki]

Emmanuel,
Thank you very much for the fast and elaborated response.

I've tested the client back and forth, I even sniffed the data that sent from 
the server and I clearly saw the server start processing a message before it 
finished processing the message before. I even looked into the NIOProcessor 
logs and I saw that a message was processed before the previous one was 
finished.
I want to emphasis that it doesn't happen when I am using encoders.
Maybe my FileWithHeaderWriteFilter is causing this problem (although it is a 
very simple one) I just don't know how...

Per your input about ordered execution please note that I care less about the 
order, all I care is that messages will arrive to their destination. 

If you need me to provide you with more details please let me know.
Thanks in advance,
Guy



-Original Message-
From: Emmanuel Lécharny [mailto:elecha...@apache.org] 
Sent: Monday, September 20, 2010 3:34 PM
To: users@mina.apache.org
Subject: Re: Possible synchronization issue

  On 9/20/10 11:35 AM, Guy Itzhaki wrote:

Ok, I have tested the code, but sadly the client blows chuncks at some 
point, as it does not assume the response can be split in pieces. I get 
an OOM because the read size is a huge integer just because what has 
been read is in fact text.

Anyway, I see that in the server sessionOpened() method you are creating 
a thread to send all the responses to the client. Don't expect this 
thread to be fully executed before another call to the server for the 
very same client to be processed, as as soon as the thread on which your 
client session is being procesded will be released as soon as the thread 
has been launched (or a few ms after).

By creating a thread in one of the Handler methods, you are breaking the 
ordered execution of client requests.

Remove this thread, and may be rethink the way your server is processing 
client requests. I would suggest something like :

 public void sessionOpened( final IoSession session )
 {
 log.info( Session Opened ignore incoming messages unless 
'Start' command );

 try
 {
 File[] files = new File( /Users/elecharny/test 
).listFiles();
 System.out.println( there are  + files.length +  
to send );

 for ( File file : files )
 {
 FileWithHeader fileWithHeader = new 
FileWithHeader( file );
 System.out.println( send  + file.getName() + 
 len  + file.length() );
 session.write( fileWithHeader );
 }
 }
 catch ( IOException e )
 {
 e.printStackTrace();
 }
 }
 }

The simple fact to call session.write inside the loop will already keep 
the session busy and this session will have to wait until the loop is 
processed to process another request for this session.

Obvioulsy, it has some drawback, like the client won't be able to stop 
the execution in the middle. You have ways to deal with this : first, 
don't use an OrderedThreadPool executor, otherwise you won't be able to 
process any other request while the previous one has been totally 
processed. Second, you'll have to manage the application's states : when 
a SEND request is being processed, another SEND request will have to 
wait. But this is something you have to deal with in the server, as MINA 
is not smart enough to do that (remember that MINA is *just* a NIO 
framework).

Adding an executor in the chain can help, but again, it's all about how 
your protocol should handle parallel requests. And then as you have to 
use a UnorderedThreadPool executor( *if* of course you want to be able 
to stop some processing on the server on the client request), then you 
still have to manage the application's states the same way.

Does it make sense ?

-- 
Regards,
Cordialement,
Emmanuel Lécharny
www.iktek.com



Click here to report this message as SPAM: 
http://vsp.ateranetworks.com/ReportSpam.php?sid=cdbd0d4e3172e6df2c2a945f36a12534_f2b5b05b50c92ce2398af1c8380c
-- Powered by ATERA Networks --

RE: Possible synchronization issue

[Guy Itzhaki]

Dear Emmanuel,
Do you believe you will take a look on this issue today or still struggling the 
release? 

I can easily reproduce it so please don't hesitate to ask me for any kind of 
help in reproducing or explaining the symptom.

Thanks again,
Guy Itzhaki


-Original Message-
From: Emmanuel Lécharny [mailto:elecha...@apache.org] 
Sent: Wednesday, September 15, 2010 1:27 PM
To: Guy Itzhaki
Subject: Re: Possible synchronization issue

  On 9/15/10 12:51 PM, Guy Itzhaki wrote:
 Emmanuel,
 Did you get my code?

Yes. Had no time to have a look at it atm, as I was fighting with Maven 
to cut a release for MINA.


-- 
Regards,
Cordialement,
Emmanuel Lécharny
www.iktek.com



Click here to report this message as SPAM: 
http://vsp.ateranetworks.com/ReportSpam.php?sid=420cfda2a7a4fe5af328021fc7f52f0b_144fc5596be4c057e97a55ffbc62de41
-- Powered by ATERA Networks --

RE: Re: Possible synchronization issue

[Guy Itzhaki]

Here you go:
public class GetMessageResponse
{
private String messageName;
private long   messageLen;
private long   messageOffset;

   private GetMessageResponse(String messageName, long messageLen, long 
messageOffset)
{
this.messageName  = messageName;
this.messageLen   = messageLen;
this.messageOffset = messageOffset ;
}

public static GetMessageResponse createMessageResponse(String messageName, 
long messageLen, long messageOffset)
{
return new GetMessageResponse(messageName, messageLen, messageOffset);
}

public long getMessageLen()
{
return messageLen;
}

public String getMessageName()
{
return messageName;
}

public long getMessageOffset()
{
return messageOffset;
}
}

Thanks,
Guy

-Original Message-
From: Emmanuel Lecharny [mailto:elecha...@gmail.com] 
Sent: Monday, September 20, 2010 10:46 AM
To: users@mina.apache.org
Cc: d...@mina.apache.org
Subject: Fwd: Re: Possible synchronization issue

  -- Users mailing list, not dev mailing list ! Emmanuel, wake up !

  On 9/20/10 8:59 AM, Guy Itzhaki wrote:
  Dear Emmanuel,
  Do you believe you will take a look on this issue today or still struggling 
 the release?
I did and sent you a response. Or did I ? Damn, I can't find the mail
from my outbox... wtf ?

Ok, anyway, I'm missing the GetMessageResponse class, so I can't test
your code.

Can you send it to me ?

Thaks ?

-- 
Regards,
Cordialement,
Emmanuel Lécharny
www.iktek.com





Click here to report this message as SPAM: 
http://vsp.ateranetworks.com/ReportSpam.php?sid=9452a364e8c58e0d5f8f526f2f47a18e_bab5d7737b3ff2580cfd0ab1cb94775f
-- Powered by ATERA Networks --

RE: Possible synchronization issue

[Guy Itzhaki]

Hi Emmanuel,
Did you have a chance to looked into this issue yet?
Please let me know if there is anything else I can provide you with that can 
assist you during your investigation.

Thanks,
Guy Itzhaki


-Original Message-
From: Emmanuel Lécharny [mailto:elecha...@apache.org] 
Sent: Tuesday, September 14, 2010 3:25 PM
To: users@mina.apache.org
Subject: Re: Possible synchronization issue

  On 9/14/10 3:16 PM, Guy Itzhaki wrote:
 Here is the scrap of the log:

Frankly, I'd rather use the code you use to be able to reproduce the 
problem on my computer, if you don't mind...


-- 
Regards,
Cordialement,
Emmanuel Lécharny
www.iktek.com



Click here to report this message as SPAM: 
http://vsp.ateranetworks.com/ReportSpam.php?sid=ac39132c65ea7c36ee475a0f70001176_5e545fa74d8f0b62cf5dcdffd047640a
-- Powered by ATERA Networks --

RE: Possible synchronization issue

[Guy Itzhaki]

Emmanuel, thanks for the quick response.

OrderedThreadPool didn't help.

The attached log might help you understand better what is happening - as said 
before every sent message has 2 parts, a header (~157 bytes) and a file (around 
1-1.5 kb).
Therefore I would expect the NIOProcessor to process first a small buffer (~157 
bytes) and then a larger one (around 1-1.5 kb) repeatedly. 
As you can see in the log the Thread:NioProcessor-1 indeed processes 157 bytes 
long HeapBuffer (which is the header), and than 1451 long HeapBuffer (which is 
the file content). BUT after that, as you can see in line 22, the second call 
to FileWithHeaderWriteFilter sneaked in...

Thanks,
Guy




-Original Message-
From: Emmanuel Lécharny [mailto:elecha...@apache.org] 
Sent: Tuesday, September 14, 2010 1:15 PM
To: users@mina.apache.org
Subject: Re: Possible synchronization issue

  On 9/14/10 1:07 PM, Guy Itzhaki wrote:
 No.
 This is the filter I'm using:
 new ExecutorFilter(Executors.newFixedThreadPool(2))

Try to use the OrderedThreadPool then.


-- 
Regards,
Cordialement,
Emmanuel Lécharny
www.iktek.com



Click here to report this message as SPAM: 
http://vsp.ateranetworks.com/ReportSpam.php?sid=bac6f639c55903f79d352fe268f89f82_091d39d20cd5550151002e16cad2865b
-- Powered by ATERA Networks --

RE: Possible synchronization issue

[Guy Itzhaki]

Here is the scrap of the log:
14:11:54 14-09-2010 INFO  Thread:main ExampleServer.main - 
==
14:11:54 14-09-2010 INFO  Thread:main ExampleServer.main - java.vm.version 
14.3-b01
14:11:54 14-09-2010 INFO  Thread:main ExampleServer.main - os.name Windows XP
14:11:54 14-09-2010 INFO  Thread:main ExampleServer.main - Configuring the TCP 
Server
14:11:54 14-09-2010 INFO  Thread:main ExampleServer.setDebbugSessionOn - Set 
logging filter on. All the traffic will be logged, are you sure?
14:11:54 14-09-2010 INFO  Thread:main ExampleServer.setIdelTime - Set session 
idle time (sec) 1000
14:11:54 14-09-2010 INFO  Thread:main ExampleServer.start - Binding TCP 
protocol on host 127.0.0.1 on port 8447
14:11:54 14-09-2010 INFO  Thread:main ExampleServer.start - The TCP Server is 
up and running and looking forward for incoming requests :-)
14:12:09 14-09-2010 INFO  Thread:NioProcessor-1 LoggingFilter.info - CREATED
14:12:09 14-09-2010 INFO  Thread:NioProcessor-1 LoggingFilter.info - OPENED
14:12:09 14-09-2010 DEBUG Thread:NioProcessor-1 OrderedThreadPoolExecutor.debug 
- Adding event SESSION_OPENED to session 2
Queue : [SESSION_OPENED, ]

14:12:09 14-09-2010 DEBUG Thread:pool-3-thread-1 IoFilterEvent.debug - Firing a 
SESSION_OPENED event for session 2
14:12:09 14-09-2010 INFO  Thread:pool-3-thread-1 
TesterSessionHandler.sessionOpened - Session Opened ignore incoming messages 
unless 'Start' command
14:12:09 14-09-2010 DEBUG Thread:pool-3-thread-1 IoFilterEvent.debug - Event 
SESSION_OPENED has been fired for session 2
14:12:10 14-09-2010 INFO  Thread:Thread-1 FileWithHeaderWriteFilter.getHeader - 
SENDING header 157
14:12:10 14-09-2010 INFO  Thread:NioProcessor-1 LoggingFilter.info - SENT: 
HeapBuffer[pos=0 lim=157 cap=157: 00 00 00 99 3C 47 65 74 4D 65 73 73 61 67 65 
52...]
14:12:10 14-09-2010 INFO  Thread:NioProcessor-1 
FileWithHeaderWriteFilter.getFile - SENDING file 1451
14:12:10 14-09-2010 INFO  Thread:NioProcessor-1 LoggingFilter.info - SENT: 
HeapBuffer[pos=0 lim=1451 cap=1451: 50 4B 03 04 14 00 08 00 08 00 5C 5D 53 3B 
00 00...]
14:12:10 14-09-2010 INFO  Thread:Thread-1 FileWithHeaderWriteFilter.getHeader - 
SENDING header 157
14:12:10 14-09-2010 INFO  Thread:NioProcessor-1 
FileWithHeaderWriteFilter.getHeader - SENDING header 157
14:12:10 14-09-2010 DEBUG Thread:NioProcessor-1 OrderedThreadPoolExecutor.debug 
- Adding event MESSAGE_SENT to session 2
Queue : [MESSAGE_SENT, ]

14:12:10 14-09-2010 DEBUG Thread:pool-3-thread-1 IoFilterEvent.debug - Firing a 
MESSAGE_SENT event for session 2
14:12:10 14-09-2010 INFO  Thread:pool-3-thread-1 
TesterSessionHandler.messageSent - Message sent successfully
14:12:10 14-09-2010 DEBUG Thread:pool-3-thread-1 IoFilterEvent.debug - Event 
MESSAGE_SENT has been fired for session 2
14:12:10 14-09-2010 INFO  Thread:NioProcessor-1 LoggingFilter.info - SENT: 
HeapBuffer[pos=0 lim=157 cap=157: 00 00 00 99 3C 47 65 74 4D 65 73 73 61 67 65 
52...]
14:12:10 14-09-2010 INFO  Thread:NioProcessor-1 
FileWithHeaderWriteFilter.getFile - SENDING file 1452
14:12:10 14-09-2010 INFO  Thread:NioProcessor-1 LoggingFilter.info - SENT: 
HeapBuffer[pos=0 lim=157 cap=157: 00 00 00 99 3C 47 65 74 4D 65 73 73 61 67 65 
52...]
14:12:10 14-09-2010 INFO  Thread:NioProcessor-1 
FileWithHeaderWriteFilter.getHeader - SENDING header 156
14:12:10 14-09-2010 DEBUG Thread:NioProcessor-1 OrderedThreadPoolExecutor.debug 
- Adding event MESSAGE_SENT to session 2
Queue : [MESSAGE_SENT, ]



-Original Message-
From: Guy Itzhaki [mailto:gitzh...@nextnine.com] 
Sent: Tuesday, September 14, 2010 2:25 PM
To: users@mina.apache.org; elecha...@apache.org
Subject: RE: Possible synchronization issue

Emmanuel, thanks for the quick response.

OrderedThreadPool didn't help.

The attached log might help you understand better what is happening - as said 
before every sent message has 2 parts, a header (~157 bytes) and a file (around 
1-1.5 kb).
Therefore I would expect the NIOProcessor to process first a small buffer (~157 
bytes) and then a larger one (around 1-1.5 kb) repeatedly. 
As you can see in the log the Thread:NioProcessor-1 indeed processes 157 bytes 
long HeapBuffer (which is the header), and than 1451 long HeapBuffer (which is 
the file content). BUT after that, as you can see in line 22, the second call 
to FileWithHeaderWriteFilter sneaked in...

Thanks,
Guy




-Original Message-
From: Emmanuel Lécharny [mailto:elecha...@apache.org] 
Sent: Tuesday, September 14, 2010 1:15 PM
To: users@mina.apache.org
Subject: Re: Possible synchronization issue

  On 9/14/10 1:07 PM, Guy Itzhaki wrote:
 No.
 This is the filter I'm using:
 new ExecutorFilter(Executors.newFixedThreadPool(2))

Try to use the OrderedThreadPool then.


-- 
Regards,
Cordialement,
Emmanuel Lécharny
www.iktek.com



Click here to report this message as SPAM: 
http://vsp.ateranetworks.com/ReportSpam.php?sid=bac6f639c55903f79d352fe268f89f82_091d39d20cd5550151002e16cad2865b
-- Powered by ATERA Networks --

MINA 2.0.0 GA

[Guy Itzhaki]

I noted that 2.0.0-RC1 is unstable and I would like to use mina in our
next version.

Do you have any idea when a GA will be released?

 

Thanks,

Guy Itzhaki

session.getCreationTime - milli vs nano

[Guy Itzhaki]

Dear mina team and users,

 

According to the session.getCreationTime() java doc the method returns
the session's creation time in nanoseconds, however I dig in and found
out that the creation time is actually in milli (which is great as I can
create Date using milli and not using nano)

 

Can you approve that the actual implementation (returning milli) is the
correct one?

(In this case I would recommend you update the java doc accordingly)

 

Thank you very much,

Guy

RE: session.getCreationTime - milli vs nano

[Guy Itzhaki]

Thanks I will

-Original Message-
From: Ashish [mailto:paliwalash...@gmail.com] 
Sent: Thursday, August 26, 2010 4:06 PM
To: users@mina.apache.org
Subject: Re: session.getCreationTime - milli vs nano

check RC1 release.. the docs are updated :)

On Thu, Aug 26, 2010 at 6:33 PM, Guy Itzhaki gitzh...@nextnine.com wrote:
 I am using mina 2.0 M6

 -Original Message-
 From: Ashish [mailto:paliwalash...@gmail.com]
 Sent: Thursday, August 26, 2010 3:46 PM
 To: users@mina.apache.org
 Subject: Re: session.getCreationTime - milli vs nano

 I don't see it as nanoseconds. Which version are u using?

 Here is the function signature from trunk

  /**
     * @return the session's creation time in milliseconds
     */
    long getCreationTime();

 The javadoc on http://mina.apache.org/report/trunk/apidocs/ also shows
 as millis

 HTH
 ashish

 On Thu, Aug 26, 2010 at 5:46 PM, Guy Itzhaki gitzh...@nextnine.com
 wrote:
 Dear mina team and users,



 According to the session.getCreationTime() java doc the method returns
 the session's creation time in nanoseconds, however I dig in and found
 out that the creation time is actually in milli (which is great as I
 can
 create Date using milli and not using nano)



 Can you approve that the actual implementation (returning milli) is
 the
 correct one?

 (In this case I would recommend you update the java doc accordingly)



 Thank you very much,

 Guy


 Click here to report this message as SPAM:
 http://vsp.ateranetworks.com/ReportSpam.php?sid=8cc4be173c25af2f59e22bb7
 045f08b4_d4dabbd59d8fbe68412183003ef57c36
 -- Powered by ATERA Networks --






-- 
thanks
ashish

Blog: http://www.ashishpaliwal.com/blog
My Photo Galleries: http://www.pbase.com/ashishpaliwal

RE: Question about IoSession.write() synchronization

[Guy Itzhaki]

Thank you very much for the fast response

-Original Message-
From: Emmanuel Lecharny [mailto:elecha...@gmail.com] 
Sent: Tuesday, August 10, 2010 9:46 AM
To: users@mina.apache.org
Subject: Re: Question about IoSession.write() synchronization

  On 8/10/10 7:49 AM, Guy Itzhaki wrote:
 Dear mina team and users,



 I have a multi threaded application, my threads as shown below are
 responsible for sending data to the client.

 Here is a scrap of the thread code:



 public class MessageSender implements Runnable

 {

  private final IoSession session;

  public void run()

  {

  /* do some business... */

  session.write(message);


  }

 }



 The server creates multiple such threads with the same IoSession.

 My question is, does mina guarantees that the server will send the
 messages one after another and not mixed up.

 Please note that the session.write()is invoked from 2 different threads
 concurrently. I really don't care about the sending order, all I care is
 that the client will get one message after another.
Messages will be enqueued before being sent back to the client. You have 
a guarantee that once a session has been established between a client 
and the server, then only one thread will process the queue for this client.

-- 
Regards,
Cordialement,
Emmanuel Lécharny
www.iktek.com



Click here to report this message as SPAM: 
http://vsp.ateranetworks.com/ReportSpam.php?sid=79dcfefe4f6adf7554e04cbca593cc50_1798c382d175165a79cdef862eadcf4f
-- Powered by ATERA Networks --

Question about IoSession.write() synchronization

[Guy Itzhaki]

Dear mina team and users,

 

I have a multi threaded application, my threads as shown below are
responsible for sending data to the client. 

Here is a scrap of the thread code:

 

public class MessageSender implements Runnable

{

private final IoSession session;

public void run()

{

/* do some business... */

session.write(message);


}

}

 

The server creates multiple such threads with the same IoSession.

My question is, does mina guarantees that the server will send the
messages one after another and not mixed up.

Please note that the session.write()is invoked from 2 different threads
concurrently. I really don't care about the sending order, all I care is
that the client will get one message after another.

 

Thanks,

Guy