Hi Eric, I was refering to expunge while using imap...
Bye, Norman 2010/4/6 Eric Charles <eric.char...@u-mangate.com>: > Hi Norman, > > Strange, POP3 is now working correctly. > Regarding expunge, It is disabled by default on thunderbird. > If I enable it, it seems to remove all mails from the folder, even if I > don't delete them. > Remembering the qmail/courrier/... server I was running a few years ago, I > had a transparent behaviour between pop/imap. > But I have to take more time to make some tests. > > Tks, > Eric > > On 04/06/2010 07:09 PM, Norman Maurer wrote: >> >> Hi Eric, >> >> I could only of one change which could maybe improve things and so you >> don't see the OOM anymore. The patch introduced in revision r930727. >> This patch changed the behavoir of pop3servers RETR and TOP handler to >> not load the email in the memory while return it to the client. >> >> Do you know which revision you used before ? >> >> >> Some more info would be interesting. Could you tell me after which >> POP3 command the Mailformed Packets appeared ? >> >> >> Thx, >> Norman >> >> 2010/4/6 Eric Charles<eric.char...@u-mangate.com>: >> >>> >>> Hi Norman, >>> >>> james revision 930727 (mina RC1, jdk 1.6.0_18) up-and-running since 1,5 >>> day >>> in prod. >>> >>> jmap -dump:file=... (this morning and right now) shows >>> org.apache.derby.impl.services.cache.Clock with 15% of Heap, less for the >>> rest. >>> >>> However, I had to redefine all accounts as IMAP in thunderbird to be able >>> to >>> read mails: >>> - the pop3 was working fine, and at a time-being, it hanged. >>> - Since there, all pop3 requests hang (thunderbird gives me a timeout >>> after >>> 2 minutes), simply saying "downloading message 1 of ..." (nothing in the >>> log >>> files) >>> - wireshark shows a correct conversation at the beginning, and afterwards >>> many "Malformed Packets". >>> Maybe I could restart james, but I prefer let him work to have a better >>> view >>> on the OOM. >>> >>> Tks, >>> >>> Eric >>> >>> >>> On 04/02/2010 09:04 PM, Norman Maurer wrote: >>> >>>> >>>> Hi Eric, >>>> >>>> I forgot to mention that one cause of the OOM could be the current >>>> usage of InMemoryMail. At the moment James load the whole MimeMessage >>>> into memory while processing the camel routes. This is really a bad >>>> practice in terms of performance and could maybe cause a OOM if you >>>> processing many big messages which are bigger (the sumary) as the >>>> avaible memory. >>>> Could this be the problem on your side ? >>>> >>>> I'm currently workin on storing the MimeMessage on an external storage >>>> and just stream in on demand.. So only the "meta-data" is stored in >>>> jms and the rest in a db / jcr etc.. I have a proof-of-concept workin >>>> here. But I need a bit more time to see if its the way to go.. >>>> >>>> I'm also evolating the use of BlobMessages with ActiveMQ. >>>> >>>> Bye, >>>> Norman >>>> >>>> >>>> 2010/4/2 Norman Maurer<nor...@apache.org>: >>>> >>>> >>>>> >>>>> Hi Eric >>>>> >>>>> so at least the files not leak anymore, unfortunaly this was the less >>>>> critical problem. >>>>> About the errors in pop3server I will have a look and see what could >>>>> cause this. I'm 100 % sure it's not related to the OOM. >>>>> >>>>> Any more infos related to the OOM are welcome. >>>>> >>>>> Bye >>>>> Norman >>>>> >>>>> 2010/4/2, Eric Charles<eric.char...@u-mangate.com>: >>>>> >>>>> >>>>>> >>>>>> nop. OutOfMemoryError is back. >>>>>> >>>>>> I will double check my server, jdk and environment config, but other >>>>>> java processes are running on the same machine without problem. >>>>>> I will also have a stress scenario on my local dev PC that runs fine >>>>>> and >>>>>> after, I will rerun it on my production server. >>>>>> Dump analysis and full profiling are also to do. >>>>>> Some stuff are difficult to simulate : See for example (occurs >>>>>> regulary) >>>>>> ERROR 16:47:23,508 | james.pop3server | WriteRequest holds not a an >>>>>> valid Object but class java.lang.String >>>>>> >>>>>> There are no more .m64 file in /tmp >>>>>> >>>>>> Tks, >>>>>> Eric >>>>>> >>>>>> >>>>>> ERROR 16:24:53,636 | james.pop3server | WriteRequest holds not a an >>>>>> valid Object but class java.lang.String >>>>>> INFO 16:24:58,061 | james.smtpserver | Successfully spooled mail >>>>>> Mail1270218298023-9820 from jewishnesses...@ecs-solar.com on >>>>>> 68.62.205.100 for [ric.char...@u-mangate.org, >>>>>> ric.char...@u-mangate.com] >>>>>> INFO 16:25:00,097 | org.apache.activemq.store.kahadb.MessageDatabase >>>>>> | >>>>>> Slow KahaDB access: Journal append took: 989 ms, Index update took 0 >>>>>> ms >>>>>> INFO 16:25:00,099 | james.mailetcontext | Storing mail >>>>>> Mail1270218298023-9820-!226071 in file://var/mail/address-error/ >>>>>> INFO 16:25:07,699 | james.smtpserver | Successfully spooled mail >>>>>> Mail1270218307697-9822 from ours...@stahelin.com on 68.62.205.100 for >>>>>> [ric.char...@u-mangate.be] >>>>>> INFO 16:25:08,723 | james.mailetcontext | Storing mail >>>>>> Mail1270218307697-9822-!70969 in file://var/mail/address-error/ >>>>>> INFO 16:25:09,908 | org.apache.activemq.store.kahadb.MessageDatabase >>>>>> | >>>>>> Slow KahaDB access: Journal append took: 1184 ms, Index update took 0 >>>>>> ms >>>>>> ERROR 16:25:35,144 | james.pop3server | WriteRequest holds not a an >>>>>> valid Object but class java.lang.String >>>>>> INFO 16:25:41,212 | org.apache.activemq.store.kahadb.MessageDatabase >>>>>> | >>>>>> Slow KahaDB access: cleanup took 7274 >>>>>> INFO 16:25:55,612 | org.apache.activemq.store.kahadb.MessageDatabase >>>>>> | >>>>>> Slow KahaDB access: cleanup took 1141 >>>>>> ERROR 16:26:42,059 | james.pop3server | WriteRequest holds not a an >>>>>> valid Object but class java.lang.String >>>>>> INFO 16:27:00,003 | org.apache.activemq.store.kahadb.MessageDatabase >>>>>> | >>>>>> Slow KahaDB access: cleanup took 58100 >>>>>> INFO 16:27:00,003 | org.apache.activemq.store.kahadb.MessageDatabase >>>>>> | >>>>>> Slow KahaDB access: Journal append took: 10032 ms, Index update took >>>>>> 52100 ms >>>>>> INFO 16:27:02,183 | james.smtpserver | Successfully spooled mail >>>>>> Mail1270218355606-9824 from inexperienc...@toutgratuit.com on >>>>>> 213.163.116.78 for [angelo.bid...@u-mangate.net] >>>>>> WARN 16:36:27,919 | >>>>>> org.springframework.jms.listener.DefaultMessageListenerContainer | >>>>>> Setup >>>>>> of JMS message listener invoker failed for destination >>>>>> 'processor.local-address-error' - trying to recover. Cause: Java heap >>>>>> space >>>>>> ERROR 16:43:58,281 | james.pop3server | WriteRequest holds not a an >>>>>> valid Object but class java.lang.String >>>>>> WARN 16:44:23,231 | >>>>>> org.springframework.jms.listener.DefaultMessageListenerContainer | >>>>>> Setup >>>>>> of JMS message listener invoker failed for destination >>>>>> 'processor.spam' >>>>>> - trying to recover. Cause: Java heap space >>>>>> ERROR 16:47:23,508 | james.pop3server | WriteRequest holds not a an >>>>>> valid Object but class java.lang.String >>>>>> ERROR 16:48:37,497 | james.pop3server | WriteRequest holds not a an >>>>>> valid Object but class java.lang.String >>>>>> ERROR 16:49:53,684 | james.pop3server | WriteRequest holds not a an >>>>>> valid Object but class java.lang.String >>>>>> ERROR 16:52:17,970 | org.apache.activemq.ActiveMQConnection | Caught >>>>>> an >>>>>> exception trying to create a JMSException for >>>>>> java.lang.OutOfMemoryError: Java heap space >>>>>> java.lang.OutOfMemoryError: Java heap space >>>>>> ERROR 16:58:05,554 | james.pop3server | WriteRequest holds not a an >>>>>> valid Object but class java.lang.String >>>>>> ERROR 17:04:31,427 | james.pop3server | WriteRequest holds not a an >>>>>> valid Object but class java.lang.String >>>>>> WARN 17:09:11,055 | >>>>>> org.springframework.jms.listener.DefaultMessageListenerContainer | >>>>>> Setup >>>>>> of JMS message listener invoker failed for destination >>>>>> 'processor.local-address-error' - trying to recover. Cause: Java heap >>>>>> space >>>>>> ERROR 17:09:35,017 | >>>>>> org.apache.james.transport.camel.JMSSelectorPollingConsumer | Consumer >>>>>> >>>>>> >>>>>> Consumer[pollingjms://queue?consumer.endpointUri=activemq%3Aqueue%3Aoutgoing.retry&delay=30000] >>>>>> could not poll endpoint: >>>>>> >>>>>> >>>>>> pollingjms://queue?consumer.endpointUri=activemq%3Aqueue%3Aoutgoing.retry&delay=30000 >>>>>> caused by: Java heap space >>>>>> java.lang.OutOfMemoryError: Java heap space >>>>>> WARN 17:09:55,361 | >>>>>> org.springframework.jms.listener.DefaultMessageListenerContainer | >>>>>> Setup >>>>>> of JMS message listener invoker failed for destination >>>>>> 'processor.spam' >>>>>> - trying to recover. Cause: Java heap space >>>>>> WARN 17:09:57,557 | >>>>>> org.springframework.jms.listener.DefaultMessageListenerContainer | >>>>>> Setup >>>>>> of JMS message listener invoker failed for destination >>>>>> 'processor.root' >>>>>> - trying to recover. Cause: Java heap space >>>>>> WARN 17:11:00,695 | >>>>>> org.springframework.jms.listener.DefaultMessageListenerContainer | >>>>>> Setup >>>>>> of JMS message listener invoker failed for destination >>>>>> 'processor.local-address-error' - trying to recover. Cause: Java heap >>>>>> space >>>>>> WARN 17:22:19,706 | >>>>>> org.springframework.jms.listener.DefaultMessageListenerContainer | >>>>>> Setup >>>>>> of JMS message listener invoker failed for destination >>>>>> 'processor.bounces' - trying to recover. Cause: Java heap space >>>>>> WARN 17:22:42,422 | >>>>>> org.springframework.jms.listener.DefaultMessageListenerContainer | >>>>>> Setup >>>>>> of JMS message listener invoker failed for destination >>>>>> 'processor.transport' - trying to recover. Cause: Java heap space >>>>>> >>>>>> >>>>>> On 04/02/2010 09:00 AM, Norman Maurer wrote: >>>>>> >>>>>> >>>>>>> >>>>>>> Hi Eric, >>>>>>> >>>>>>> this sounds promising.... Looking forward for your next report. >>>>>>> >>>>>>> Bye >>>>>>> Norman >>>>>>> >>>>>>> 2010/4/2, Eric Charles<eric.char...@u-mangate.com>: >>>>>>> >>>>>>> >>>>>>> >>>>>>>> >>>>>>>> James survived the night. >>>>>>>> 300MB (exactly the same as when it started). >>>>>>>> No m64 file in tmp. >>>>>>>> >>>>>>>> I will post another update this evening. >>>>>>>> >>>>>>>> Tks, >>>>>>>> >>>>>>>> Eric >>>>>>>> >>>>>>>> >>>>>>>> On 04/01/2010 07:06 PM, Norman Maurer wrote: >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>>> >>>>>>>>> Yeah thats right.. just checout server/trunk. >>>>>>>>> >>>>>>>>> I'm looking forward to hear your feedback ;) >>>>>>>>> >>>>>>>>> Bye, >>>>>>>>> Norman >>>>>>>>> >>>>>>>>> >>>>>>>>> 2010/4/1 Eric Charles<eric.char...@u-mangate.com>: >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>>> >>>>>>>>>> Hello Norman, >>>>>>>>>> Done :) >>>>>>>>>> Tks, Eric >>>>>>>>>> >>>>>>>>>> PS1 : I didn't change the config, except for the JDCBDomainList I >>>>>>>>>> am >>>>>>>>>> using. >>>>>>>>>> PS2 : I only checkout >>>>>>>>>> http://svn.apache.org/repos/asf/james/server/trunk >>>>>>>>>> and >>>>>>>>>> not the other projects. From what I read from the poms, snapshot >>>>>>>>>> version >>>>>>>>>> of >>>>>>>>>> imap-mailbox, jsieve,... are taken from the central maven >>>>>>>>>> repository >>>>>>>>>> and >>>>>>>>>> I >>>>>>>>>> don't have to rebuild those projects. >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> Hi Eric, >>>>>>>>>>> >>>>>>>>>>> its me again ;) Could you please svn up to latest trunk and >>>>>>>>>>> try >>>>>>>>>>> again >>>>>>>>>>> ? >>>>>>>>>>> >>>>>>>>>>> Thx, >>>>>>>>>>> Norman >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>> >>>>>>>>>> 2010/4/1 Norman Maurer<norman.mau...@googlemail.com>: >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> Hi Eric, >>>>>>>>>>>> >>>>>>>>>>>> I did a stress test in the last few hours (running in the >>>>>>>>>>>> background >>>>>>>>>>>> here) and I was not able to get a OOM yet with >>>>>>>>>>>> "?broker.useJmx=false&jms.prefetchPolicy.all=1" for the >>>>>>>>>>>> jmsConnectionFactory. I sent 40000 emails with 100k to it and >>>>>>>>>>>> 3000 >>>>>>>>>>>> with 5 mb. >>>>>>>>>>>> >>>>>>>>>>>> I'm using -Xmx512m and memory usage is not getting higher >>>>>>>>>>>> then >>>>>>>>>>>> 700m. >>>>>>>>>>>> >>>>>>>>>>>> Bye. >>>>>>>>>>>> Norman >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> --------------------------------------------------------------------- >>>>>>>>>> To unsubscribe, e-mail: server-dev-unsubscr...@james.apache.org >>>>>>>>>> For additional commands, e-mail: server-dev-h...@james.apache.org >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> --------------------------------------------------------------------- >>>>>>>>> To unsubscribe, e-mail: server-dev-unsubscr...@james.apache.org >>>>>>>>> For additional commands, e-mail: server-dev-h...@james.apache.org >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> --------------------------------------------------------------------- >>>>>>>> To unsubscribe, e-mail: server-dev-unsubscr...@james.apache.org >>>>>>>> For additional commands, e-mail: server-dev-h...@james.apache.org >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>> >>>>>>> --------------------------------------------------------------------- >>>>>>> To unsubscribe, e-mail: server-dev-unsubscr...@james.apache.org >>>>>>> For additional commands, e-mail: server-dev-h...@james.apache.org >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>> >>>>>> --------------------------------------------------------------------- >>>>>> To unsubscribe, e-mail: server-dev-unsubscr...@james.apache.org >>>>>> For additional commands, e-mail: server-dev-h...@james.apache.org >>>>>> >>>>>> >>>>>> >>>>>> >>>>> >>>>> >>>> >>>> --------------------------------------------------------------------- >>>> To unsubscribe, e-mail: server-dev-unsubscr...@james.apache.org >>>> For additional commands, e-mail: server-dev-h...@james.apache.org >>>> >>>> >>>> >>>> >>>> >>> >>> --------------------------------------------------------------------- >>> To unsubscribe, e-mail: server-dev-unsubscr...@james.apache.org >>> For additional commands, e-mail: server-dev-h...@james.apache.org >>> >>> >>> >> >> --------------------------------------------------------------------- >> To unsubscribe, e-mail: server-dev-unsubscr...@james.apache.org >> For additional commands, e-mail: server-dev-h...@james.apache.org >> >> >> >> >> > > > --------------------------------------------------------------------- > To unsubscribe, e-mail: server-dev-unsubscr...@james.apache.org > For additional commands, e-mail: server-dev-h...@james.apache.org > > --------------------------------------------------------------------- To unsubscribe, e-mail: server-dev-unsubscr...@james.apache.org For additional commands, e-mail: server-dev-h...@james.apache.org
