Re: JAMES OOM, maybe the cause of MINA ?

2010-04-20 Thread Eric Charles
Hi, I finally made some tests with mina-rc2 trying to reproduce the oom. I didn't succeed to reproduce it when stressing james with normal mails (well-formed,...). I would tend to follow the pointer Norman gave about the infinite loop in the exceptionCaught() method (writing on the session

Re: JAMES OOM, maybe the cause of MINA ?

2010-04-13 Thread Ashish
Is the OOM fixed after the change? thanks ashish On Mon, Apr 12, 2010 at 1:15 PM, Norman Maurer norman.mau...@googlemail.com wrote: Hi Ashish, I bet its the loop I mention before.. This would make sense when looking at the growing queued writes. I commited a change to trunk which close the

Re: JAMES OOM, maybe the cause of MINA ?

2010-04-13 Thread Norman Maurer
We need to let Eric test it again. Currently he run the netty version of james to see if the cause is really MINA .. Thx, Norman 2010/4/13 Ashish paliwalash...@gmail.com: Is the OOM fixed after the change? thanks ashish On Mon, Apr 12, 2010 at 1:15 PM, Norman Maurer

Re: JAMES OOM, maybe the cause of MINA ?

2010-04-12 Thread Norman Maurer
Hi Ashish, I bet its the loop I mention before.. This would make sense when looking at the growing queued writes. I commited a change to trunk which close the session after write the data to the client. We will see if this helps.. Thx, Norman 2010/4/12 Ashish paliwalash...@gmail.com: On Sun,

Re: JAMES OOM, maybe the cause of MINA ?

2010-04-12 Thread Ashish
On Sun, Apr 11, 2010 at 8:57 PM, Norman Maurer norman.mau...@googlemail.com wrote: Hi Ashish, after thinkin more about the whole stuff I wonder if the exception handling in SMTP could sometimes be an endless loop. In the handler we write a message back to the client on exception. But what

Re: JAMES OOM, maybe the cause of MINA ?

2010-04-12 Thread Ashish
On Mon, Apr 12, 2010 at 1:15 PM, Norman Maurer norman.mau...@googlemail.com wrote: Hi Ashish, I bet its the loop I mention before.. This would make sense when looking at the growing queued writes. I commited a change to trunk which close the session after write the data to the client. We will

Re: JAMES OOM, maybe the cause of MINA ?

2010-04-12 Thread Norman Maurer
Forgot to include JAMES list.. 2010/4/12 Norman Maurer norman.mau...@googlemail.com: Hi Asish, yeah in our case this is fine. But I think even if this fix the problem MINA should at least try to break such loops. Bye, Norman 2010/4/12 Ashish paliwalash...@gmail.com: On Mon, Apr 12,

Re: JAMES OOM, maybe the cause of MINA ?

2010-04-11 Thread Norman Maurer
Hi Ashish, after thinkin more about the whole stuff I wonder if the exception handling in SMTP could sometimes be an endless loop. In the handler we write a message back to the client on exception. But what happens if this throw a new exception ? Could it so loop forever ?

Re: JAMES OOM, maybe the cause of MINA ?

2010-04-09 Thread Ashish
Norman, Couple of more queries 1. The heap dump uses circularqueue class, so seem to be taken for an earlier trunk snapshot. Is my take correct? 2. What's the state of the System? are the clients receiving the messages. The queue seems to be holding a very large number of objects. Essentially

Re: JAMES OOM, maybe the cause of MINA ?

2010-04-09 Thread Ashish
Did some more investigation. Here is one of the strange observations Most of the hanging DefaultWriteRequest have this message 451 Unable to process smtp request.. Can't analyze all of them, but saw content of close to 100 objects. All of them had the same content

Re: JAMES OOM, maybe the cause of MINA ?

2010-04-09 Thread Norman Maurer
Hi Ashish, yeah ofter some more investigation it seems that it's not the StreamIoHandler which cause the problem. It must be something else... Bye Norman 2010/4/9 Ashish paliwalash...@gmail.com: Did some more investigation. Here is one of the strange observations Most of the hanging

Re: JAMES OOM, maybe the cause of MINA ?

2010-04-09 Thread Ashish
Lets dig this further and finish this off.. :) Will try to generate this OOM locally... will keep you all posted On Fri, Apr 9, 2010 at 9:32 PM, Norman Maurer nor...@apache.org wrote: Hi Ashish, yeah ofter some more investigation it seems that it's not the StreamIoHandler which cause the

Re: JAMES OOM, maybe the cause of MINA ?

2010-04-09 Thread Eric Charles
Hi Ashish, I am the user who has many OOM with the trunk and feed Norman with my issues :) 1. Indeed, you have the dump with CircularQueue from mina 2.0.0-RC1. This is the release we normally use in james and give the issue after a certain period of time. I mean, the server can stay working

Re: JAMES OOM, maybe the cause of MINA ?

2010-04-08 Thread Norman Maurer
Hi Ashish, I think we tracked down the source of the problem a bit more.. The OOM seems to be related to IMAP. Our IMAP server component is using the StreamIoHandler (its the only one of our components who use this handler). So I suspect there is the problem. So there are two possible problems:

Re: JAMES OOM, maybe the cause of MINA ?

2010-04-08 Thread Ashish
On Thu, Apr 8, 2010 at 1:28 PM, Norman Maurer norman.mau...@googlemail.com wrote: Hi Ashish, I think we tracked down the source of the problem a bit more.. The OOM seems to be related to IMAP. Our IMAP server component is using the StreamIoHandler (its the only one of our components who use

Re: JAMES OOM, maybe the cause of MINA ?

2010-04-08 Thread Norman Maurer
Maybe Eric can do, cause he is the one who see it very freqently.. So Eric...;) ? Thx, Norman 2010/4/8 Ashish paliwalash...@gmail.com: Can you provide the heapdump for this OOM? thanks ashish On Thu, Apr 8, 2010 at 1:40 PM, Ashish paliwalash...@gmail.com wrote: On Thu, Apr 8, 2010 at

Re: JAMES OOM, maybe the cause of MINA ?

2010-04-08 Thread Ashish
Can you provide the heapdump for this OOM? thanks ashish On Thu, Apr 8, 2010 at 1:40 PM, Ashish paliwalash...@gmail.com wrote: On Thu, Apr 8, 2010 at 1:28 PM, Norman Maurer norman.mau...@googlemail.com wrote: Hi Ashish, I think we tracked down the source of the problem a bit more.. The OOM

Re: JAMES OOM, maybe the cause of MINA ?

2010-04-08 Thread Eric Charles
Hi Ashish, You can find some information to download on http://apache.u-mangate.com/james/oom/index.html Tks, Eric On 04/08/2010 10:26 AM, Ashish wrote: Can you provide the heapdump for this OOM? thanks ashish On Thu, Apr 8, 2010 at 1:40 PM, Ashishpaliwalash...@gmail.com wrote: On

Re: JAMES OOM, maybe the cause of MINA ?

2010-04-05 Thread Ashish
On Sun, Apr 4, 2010 at 12:59 AM, Norman Maurer nor...@apache.org wrote: Hi all, we at JAMES use MINA since a while now for our socket stuff. After doing some stress tests we encountered some OOM Exceptions. The same was seen by one of our users which use JAMES trunk. After debugging stuff it

Re: JAMES OOM, maybe the cause of MINA ?

2010-04-05 Thread Norman Maurer
Hi Ashish, thx for your reply. The problem is sometimes it takes sometime to get the OOM, other times it happens within one minute. One of our users reported to me that it happened within 1 minute (10 seconds and 30 seconds), after he switched to MINA trunk. So it seems it is faster reproducable

Re: JAMES OOM, maybe the cause of MINA ?

2010-04-05 Thread Ashish
Thanks Norman, Will try to see this as soon as I can get some time from paid job :) cheers ashish On Mon, Apr 5, 2010 at 1:27 PM, Norman Maurer norman.mau...@googlemail.com wrote: Hi Ashish, thx for your reply. The problem is sometimes it takes sometime to get the OOM, other times it

Re: JAMES OOM, maybe the cause of MINA ?

2010-04-05 Thread Eric Charles
Hi Ashish, Please also note that we have 2 different exceptions: - http://people.apache.org/~norman/oom/Screenshot.png that occurs with mina 2.0.0-RC1 - http://people.apache.org/~norman/oom/Screenshot-Eclipse_Memory_Analyzer.png that occurs with 2.0.0-RC2-SNAPSHOT It happens on jdk 1.6 (and

Re: JAMES OOM, maybe the cause of MINA ?

2010-04-05 Thread Norman Maurer
I think thats only because mina-trunk is not using CircularQueue anymore. It now seems to use ConcurrentLinkedQueue. Bye, Norman 2010/4/5 Eric Charles eric.char...@u-mangate.com: Hi Ashish, Please also note that we have 2 different exceptions: -

Re: JAMES OOM, maybe the cause of MINA ?

2010-04-05 Thread Eric Charles
Yes, One instance of org.apache.mina.core.session.DefaultIoSessionDataStructureFactory$DefaultWriteRequestQueue contains : - for 2.0.0-RC1 : one instance of org.apache.mina.util.CircularQueue with an array of many org.apache.mina.filter.codec.ProtocolCodecFilter$EncodedWriteRequest - for

Re: JAMES OOM, maybe the cause of MINA ?

2010-04-05 Thread Ashish
On Mon, Apr 5, 2010 at 2:21 PM, Norman Maurer norman.mau...@googlemail.com wrote: I think thats only because mina-trunk is not using CircularQueue anymore. It now seems to use ConcurrentLinkedQueue. Bye, Norman I remember, Emm removed all references to CircularQueue, but this was to make it

JAMES OOM, maybe the cause of MINA ?

2010-04-03 Thread Norman Maurer
Hi all, we at JAMES use MINA since a while now for our socket stuff. After doing some stress tests we encountered some OOM Exceptions. The same was seen by one of our users which use JAMES trunk. After debugging stuff it seems like the cause of the OOM is MINA. We took some heap which shows that

Re: JAMES OOM, maybe the cause of MINA ?

2010-04-03 Thread Norman Maurer
I should think of the attachment stripping on the ml.. You can find the screenshots here: http://people.apache.org/~norman/oom/ Bye, Norman 2010/4/3 Norman Maurer nor...@apache.org: Hi all, we at JAMES use MINA since a while now for our socket stuff. After doing some stress tests we