Hi Eric, could you try to downgrade to camel 2.5.0 and see if you can still reproduce it ?
Thanks, Norman 2011/5/27 Eric Charles <[email protected]>: > Hi Nicola, > > Yes, you're right. > > The problem arise in fact when the mail must be delivered to/cc local and > non-local recipients, so when LocalDelivery and RemoteDelivery mailets must > come into the play for the same mail. > > I see cpu rising to 100% and leaking files in blob transfer - I didn't wait > to see oom, but I'm pretty sure it will come. > > I also tested with remotedelivery configured with a gateway and without a > gateway (in my case, non gateway make the mail undeliverable to the remote > recipient). This does not change anything (bug is there for both > configuration). > > Could I ask you to open a JIRA (https://issues.apache.org/jira/browse/JAMES) > for this and define it as blocker. > > Tks very much for the follow, > - Eric > > On 27/05/2011 15:09, Nicola Salvemini wrote: >> >> Hello, >> my mailetcontainer.xml is very standard, I don't use custom servlet or >> something strange. >> Could you please try sending only with James without using your ISP SMTP >> server? >> >> Thanks, >> >> Nicola. >> >> >> >> >> On Fri, 2011-05-27 at 12:42 +0200, Eric Charles wrote: >>> >>> Replicated issue is the begin of the solution :) >>> >>> I sent a mail with my account in cc, and everything happens fine >>> (delivered, no remaining files in blob-transfer). >>> >>> It goes via my ISP SMTP server before reaching James, but that should >>> not change anything. >>> >>> Did you change mailetcontainer.xml or any other configuration ? >>> >>> Tks, >>> - Eric >>> >>> On 27/05/2011 12:34, Nicola Salvemini wrote: >>>> >>>> Hello, >>>> >>>> I have a good news! I'm able to replicate problem: I noticed the problem >>>> appears when James try to delivery email having a local address in CC. >>>> Please consider following example: >>>> James is configured as SMTP server for mydomain.com. If it try to >>>> deliver email to [email protected] having [email protected] in CC, this >>>> is the result: >>>> >>>> DEBUG 12:01:43,641 | james.mailspooler | ==== Begin processing mail >>>> Mail1306490503638-8a087915-aded-4c10-9b37-be8d543652c1==== >>>> DEBUG 12:01:43,641 | james.mailprocessor | Call MailProcessor root >>>> DEBUG 12:01:43,773 | james.mailprocessor | Call MailProcessor transport >>>> >>>> Nothing more! >>>> >>>>> From this point on, James is not able to send any mail. All emails >>>> >>>> remain in "store/activemq/blob-transfer/" until OOM is reached. >>>> To resolve it's necessary delete email in >>>> "store/activemq/blob-transfer/" and restart the server. >>>> >>>> I think the problem is this. What do you think? >>>> >>>> Nicola. >>>> >>>> >>>> >>>> >>>> >>>> >>>> On Thu, 2011-05-26 at 15:28 +0200, Eric Charles wrote: >>>>> >>>>> OK, let's see what the next dump tells us. >>>>> >>>>> Once a OOM occurs, the consecutive exceptions can come from anywhere >>>>> and >>>>> should not be considered too much. Memory becomes short, jvm is >>>>> confused... >>>>> >>>>> Have you still some messages stuck in queue and remaining files in >>>>> "store/activemq/blob-transfer/" ? >>>>> >>>>> I was wondering about your mailetcontainer.xml configuration. Did you >>>>> configure something special? Is it possible that for some reasones, the >>>>> processor route a mails takes is "infinite" ? >>>>> >>>>> Tks, >>>>> - Eric >>>>> >>>>> >>>>> On 26/05/2011 15:18, Nicola Salvemini wrote: >>>>>> >>>>>> Hi, >>>>>> >>>>>> here new exceptions found in the log. I installed Eclipe Memory >>>>>> Analyzer >>>>>> but I was not able to open the dump file because it was corrupt. I'll >>>>>> retry with next OOM. >>>>>> >>>>>> DEBUG 12:21:12,022 | james.mailspooler | ==== Begin processing mail >>>>>> Mail1306405272019-60eec5f7-1b95-49aa-b269-8116f29fda92==== >>>>>> DEBUG 12:21:12,023 | james.mailprocessor | Call MailProcessor root >>>>>> DEBUG 12:21:12,753 | james.mailprocessor | Call MailProcessor >>>>>> transport >>>>>> INFO 12:21:18,821 | james.mailetcontext | Local delivered mail >>>>>> Mail1306405272019-60eec5f7-1b95-49aa-b269-8116f29fda92 sucessfully >>>>>> from >>>>>> [email protected] to [email protected] >>>>>> INFO 12:21:20,157 | james.mailetcontext | Local delivered mail >>>>>> Mail1306405272019-60eec5f7-1b95-49aa-b269-8116f29fda92 sucessfully >>>>>> from >>>>>> [email protected] to [email protected] >>>>>> DEBUG 12:21:20,158 | james.mailprocessor | End of mailetprocessor for >>>>>> state root reached >>>>>> ERROR 12:21:56,674 | james.mailspooler | Exception processing mail >>>>>> while >>>>>> spooling Unable to dequeue next message >>>>>> org.apache.james.queue.api.MailQueue$MailQueueException: Unable to >>>>>> dequeue next message; >>>>>> nested exception is: >>>>>> javax.jms.JMSException: PermGen space >>>>>> at >>>>>> org.apache.james.queue.jms.JMSMailQueue.deQueue(JMSMailQueue.java:171) >>>>>> at >>>>>> >>>>>> org.apache.james.mailetcontainer.lib.JamesMailSpooler.run(JamesMailSpooler.java:135) >>>>>> at java.lang.Thread.run(Thread.java:662) >>>>>> Caused by: javax.jms.JMSException: PermGen space >>>>>> at >>>>>> >>>>>> org.apache.activemq.util.JMSExceptionSupport.create(JMSExceptionSupport.java:49) >>>>>> at >>>>>> >>>>>> org.apache.activemq.ActiveMQConnection.syncSendPacket(ActiveMQConnection.java:1287) >>>>>> at >>>>>> >>>>>> org.apache.activemq.ActiveMQSession.syncSendPacket(ActiveMQSession.java:1874) >>>>>> at >>>>>> >>>>>> org.apache.activemq.ActiveMQMessageConsumer.<init>(ActiveMQMessageConsumer.java:254) >>>>>> at >>>>>> >>>>>> org.apache.activemq.ActiveMQSession.createConsumer(ActiveMQSession.java:1116) >>>>>> at >>>>>> >>>>>> org.apache.activemq.ActiveMQSession.createConsumer(ActiveMQSession.java:1060) >>>>>> at >>>>>> >>>>>> org.apache.activemq.ActiveMQSession.createConsumer(ActiveMQSession.java:973) >>>>>> at sun.reflect.GeneratedMethodAccessor71.invoke(Unknown >>>>>> Source) >>>>>> at >>>>>> >>>>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) >>>>>> at java.lang.reflect.Method.invoke(Method.java:597) >>>>>> at >>>>>> org.springframework.jms.connection.CachingConnectionFactory >>>>>> >>>>>> $CachedSessionInvocationHandler.invoke(CachingConnectionFactory.java:344) >>>>>> at $Proxy61.createConsumer(Unknown Source) >>>>>> at >>>>>> org.apache.james.queue.jms.JMSMailQueue.deQueue(JMSMailQueue.java:109) >>>>>> ... 2 more >>>>>> Caused by: java.lang.OutOfMemoryError: PermGen space >>>>>> at java.lang.String.intern(Native Method) >>>>>> at >>>>>> javax.management.ObjectName.setCanonicalName(ObjectName.java:816) >>>>>> at >>>>>> javax.management.ObjectName.construct(ObjectName.java:644) >>>>>> at javax.management.ObjectName.<init>(ObjectName.java:1403) >>>>>> at >>>>>> >>>>>> org.apache.activemq.broker.jmx.ManagedRegionBroker.registerSubscription(ManagedRegionBroker.java:184) >>>>>> at >>>>>> >>>>>> org.apache.activemq.broker.jmx.ManagedQueueRegion.createSubscription(ManagedQueueRegion.java:45) >>>>>> at >>>>>> >>>>>> org.apache.activemq.broker.region.AbstractRegion.addConsumer(AbstractRegion.java:268) >>>>>> at >>>>>> >>>>>> org.apache.activemq.broker.region.RegionBroker.addConsumer(RegionBroker.java:444) >>>>>> at >>>>>> >>>>>> org.apache.activemq.broker.jmx.ManagedRegionBroker.addConsumer(ManagedRegionBroker.java:240) >>>>>> at >>>>>> >>>>>> org.apache.activemq.broker.BrokerFilter.addConsumer(BrokerFilter.java:89) >>>>>> at >>>>>> >>>>>> org.apache.activemq.advisory.AdvisoryBroker.addConsumer(AdvisoryBroker.java:91) >>>>>> at >>>>>> >>>>>> org.apache.activemq.broker.BrokerFilter.addConsumer(BrokerFilter.java:89) >>>>>> at >>>>>> >>>>>> org.apache.activemq.broker.BrokerFilter.addConsumer(BrokerFilter.java:89) >>>>>> at >>>>>> >>>>>> org.apache.activemq.broker.BrokerFilter.addConsumer(BrokerFilter.java:89) >>>>>> at >>>>>> >>>>>> org.apache.activemq.broker.MutableBrokerFilter.addConsumer(MutableBrokerFilter.java:95) >>>>>> at >>>>>> >>>>>> org.apache.activemq.broker.TransportConnection.processAddConsumer(TransportConnection.java:550) >>>>>> at >>>>>> org.apache.activemq.command.ConsumerInfo.visit(ConsumerInfo.java:349) >>>>>> at >>>>>> >>>>>> org.apache.activemq.broker.TransportConnection.service(TransportConnection.java:311) >>>>>> at org.apache.activemq.broker.TransportConnection >>>>>> $1.onCommand(TransportConnection.java:185) >>>>>> at >>>>>> >>>>>> org.apache.activemq.transport.ResponseCorrelator.onCommand(ResponseCorrelator.java:116) >>>>>> at >>>>>> >>>>>> org.apache.activemq.transport.TransportFilter.onCommand(TransportFilter.java:69) >>>>>> at >>>>>> >>>>>> org.apache.activemq.transport.vm.VMTransport.iterate(VMTransport.java:219) >>>>>> at >>>>>> >>>>>> org.apache.activemq.thread.PooledTaskRunner.runTask(PooledTaskRunner.java:122) >>>>>> at org.apache.activemq.thread.PooledTaskRunner >>>>>> $1.run(PooledTaskRunner.java:43) >>>>>> at java.util.concurrent.ThreadPoolExecutor >>>>>> $Worker.runTask(ThreadPoolExecutor.java:886) >>>>>> at java.util.concurrent.ThreadPoolExecutor >>>>>> $Worker.run(ThreadPoolExecutor.java:908) >>>>>> ... 1 more >>>>>> DEBUG 12:22:06,560 | james.mailspooler | ==== Begin processing mail >>>>>> Mail1306405326556-44ac5713-6208-44a7-985c-86782b4a7db1==== >>>>>> DEBUG 12:22:06,560 | james.mailprocessor | Call MailProcessor root >>>>>> DEBUG 12:22:06,591 | james.mailprocessor | Call MailProcessor >>>>>> transport >>>>>> INFO 12:22:16,207 | james.mailetcontext | Local delivered mail >>>>>> Mail1306405326556-44ac5713-6208-44a7-985c-86782b4a7db1 sucessfully >>>>>> from >>>>>> [email protected] to [email protected] >>>>>> DEBUG 12:22:16,208 | james.mailprocessor | End of mailetprocessor for >>>>>> state root reached >>>>>> DEBUG 12:22:22,101 | james.mailspooler | ==== Begin processing mail >>>>>> Mail1306405342098-6ecf752a-5ec3-4bef-aa54-07b94cb7bbbc==== >>>>>> DEBUG 12:22:22,101 | james.mailprocessor | Call MailProcessor root >>>>>> DEBUG 12:22:22,130 | james.mailprocessor | Call MailProcessor >>>>>> transport >>>>>> INFO 12:22:22,873 | james.mailetcontext | Local delivered mail >>>>>> Mail1306405342098-6ecf752a-5ec3-4bef-aa54-07b94cb7bbbc sucessfully >>>>>> from >>>>>> [email protected] to [email protected] >>>>>> DEBUG 12:22:22,874 | james.mailprocessor | End of mailetprocessor for >>>>>> state root reached >>>>>> ERROR 12:25:41,320 | james.mailspooler | Exception processing mail >>>>>> while >>>>>> spooling Unable to dequeue next message >>>>>> INFO 12:27:26,135 | james.mailetcontext | Exception caught in >>>>>> RemoteDelivery.run() >>>>>> INFO 12:27:38,052 | james.mailetcontext | Exception caught in >>>>>> RemoteDelivery.run() >>>>>> INFO 12:27:38,724 | james.mailetcontext | Exception caught in >>>>>> RemoteDelivery.run() >>>>>> ERROR 12:28:16,597 | james.mailspooler | Exception processing mail >>>>>> while >>>>>> spooling Unable to dequeue next message >>>>>> ERROR 12:28:30,033 | james.mailspooler | Exception processing mail >>>>>> while >>>>>> spooling Unable to dequeue next message >>>>>> ERROR 12:28:32,152 | james.mailspooler | Exception processing mail >>>>>> while >>>>>> spooling Unable to dequeue next message >>>>>> ERROR 12:28:42,077 | james.mailspooler | Exception processing mail >>>>>> while >>>>>> spooling Unable to dequeue next message >>>>>> INFO 12:28:42,830 | james.mailetcontext | Exception caught in >>>>>> RemoteDelivery.run() >>>>>> ERROR 12:28:44,283 | james.mailspooler | Exception processing mail >>>>>> while >>>>>> spooling Unable to dequeue next message >>>>>> ERROR 12:28:47,108 | james.mailspooler | Exception processing mail >>>>>> while >>>>>> spooling Unable to dequeue next message >>>>>> org.apache.james.queue.api.MailQueue$MailQueueException: Unable to >>>>>> dequeue next message; >>>>>> nested exception is: >>>>>> javax.jms.JMSException: PermGen space >>>>>> at >>>>>> org.apache.james.queue.jms.JMSMailQueue.deQueue(JMSMailQueue.java:171) >>>>>> at >>>>>> >>>>>> org.apache.james.mailetcontainer.lib.JamesMailSpooler.run(JamesMailSpooler.java:135) >>>>>> at java.lang.Thread.run(Thread.java:662) >>>>>> Caused by: javax.jms.JMSException: PermGen space >>>>>> at >>>>>> >>>>>> org.apache.activemq.util.JMSExceptionSupport.create(JMSExceptionSupport.java:49) >>>>>> at >>>>>> >>>>>> org.apache.activemq.ActiveMQConnection.syncSendPacket(ActiveMQConnection.java:1287) >>>>>> at >>>>>> >>>>>> org.apache.activemq.ActiveMQSession.syncSendPacket(ActiveMQSession.java:1874) >>>>>> at >>>>>> >>>>>> org.apache.activemq.ActiveMQMessageConsumer.<init>(ActiveMQMessageConsumer.java:254) >>>>>> at >>>>>> >>>>>> org.apache.activemq.ActiveMQSession.createConsumer(ActiveMQSession.java:1116) >>>>>> at >>>>>> >>>>>> org.apache.activemq.ActiveMQSession.createConsumer(ActiveMQSession.java:1060) >>>>>> at >>>>>> >>>>>> org.apache.activemq.ActiveMQSession.createConsumer(ActiveMQSession.java:973) >>>>>> at sun.reflect.GeneratedMethodAccessor71.invoke(Unknown >>>>>> Source) >>>>>> at >>>>>> >>>>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) >>>>>> at java.lang.reflect.Method.invoke(Method.java:597) >>>>>> at >>>>>> org.springframework.jms.connection.CachingConnectionFactory >>>>>> >>>>>> $CachedSessionInvocationHandler.invoke(CachingConnectionFactory.java:344) >>>>>> at $Proxy61.createConsumer(Unknown Source) >>>>>> at >>>>>> org.apache.james.queue.jms.JMSMailQueue.deQueue(JMSMailQueue.java:109) >>>>>> ... 2 more >>>>>> Caused by: java.lang.OutOfMemoryError: PermGen space >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> On Thu, 2011-05-26 at 11:55 +0200, Eric Charles wrote: >>>>>>> >>>>>>> I'm using Eclipe Memory Analyzer (http://www.eclipse.org/mat/). >>>>>>> If you go via the menu and open the dump file, it will propose you >>>>>>> (after a few minutes anlaysis) a report that will present you >>>>>>> "suspects". You can copy/paste this report on ml. >>>>>>> >>>>>>> There's also YourKit (http://www.yourkit.com/) but I never used it >>>>>>> (not >>>>>>> sure it allows to analyze taken dumps, it more for live profiling). >>>>>>> >>>>>>> Tks, >>>>>>> - Eric >>>>>>> >>>>>>> >>>>>>> On 26/05/2011 11:46, Nicola Salvemini wrote: >>>>>>>> >>>>>>>> Hi Eric, >>>>>>>> >>>>>>>> yes, I added "-XX:+HeapDumpOnOutOfMemoryError >>>>>>>> -XX:HeapDumpPath=/your_path" to the command line, but I'm finding a >>>>>>>> easy >>>>>>>> way to analize dump file. Have you any suggestion? >>>>>>>> >>>>>>>> Regards, >>>>>>>> Nicola. >>>>>>>> >>>>>>>> >>>>>>>> On Thu, 2011-05-26 at 11:25 +0200, Eric Charles wrote: >>>>>>>>> >>>>>>>>> Hi, >>>>>>>>> Getting OOM can have various side effects, so I bet the >>>>>>>>> java.lang.VerifyError is a result of the OOM. >>>>>>>>> >>>>>>>>> Did you add "-XX:+HeapDumpOnOutOfMemoryError >>>>>>>>> -XX:HeapDumpPath=/your_path" and got a dump? >>>>>>>>> >>>>>>>>> If you have now time to analyze to find the "suspect", we can do it >>>>>>>>> for >>>>>>>>> you if you transfer us the dump (be aware that some objects can >>>>>>>>> contain >>>>>>>>> some user credentials, but that really doesn't interest us). >>>>>>>>> >>>>>>>>> Tks, >>>>>>>>> - Eric >>>>>>>>> >>>>>>>>> >>>>>>>>> On 26/05/2011 09:20, Nicola Salvemini wrote: >>>>>>>>>> >>>>>>>>>> Hello, >>>>>>>>>> >>>>>>>>>> the problem persists, but now I have a new exception in the log: >>>>>>>>>> >>>>>>>>>> ERROR 09:12:38,851 | james.mailspooler | Exception processing mail >>>>>>>>>> while >>>>>>>>>> spooling PermGen space >>>>>>>>>> java.lang.OutOfMemoryError: PermGen space >>>>>>>>>> ERROR 09:12:49,757 | james.mailspooler | Exception processing mail >>>>>>>>>> while >>>>>>>>>> spooling (class: javax/mail/MessagingException, method: >>>>>>>>>> superToString >>>>>>>>>> signature: ()Ljava/lang/String;) Illegal constant pool index >>>>>>>>>> java.lang.VerifyError: (class: javax/mail/MessagingException, >>>>>>>>>> method: >>>>>>>>>> superToString signature: ()Ljava/lang/String;) Illegal constant >>>>>>>>>> pool >>>>>>>>>> index >>>>>>>>>> at >>>>>>>>>> >>>>>>>>>> org.apache.james.mailetcontainer.camel.CamelMailetProcessor.service(CamelMailetProcessor.java:68) >>>>>>>>>> at >>>>>>>>>> >>>>>>>>>> org.apache.james.mailetcontainer.lib.AbstractStateCompositeProcessor.service(AbstractStateCompositeProcessor.java:104) >>>>>>>>>> at >>>>>>>>>> >>>>>>>>>> org.apache.james.mailetcontainer.lib.JamesMailSpooler.run(JamesMailSpooler.java:147) >>>>>>>>>> at java.lang.Thread.run(Thread.java:662) >>>>>>>>>> ERROR 09:13:21,452 | james.mailspooler | Exception processing mail >>>>>>>>>> while >>>>>>>>>> spooling PermGen space >>>>>>>>>> java.lang.OutOfMemoryError: PermGen space >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> Nicola. >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> On Wed, 2011-05-25 at 16:09 +0200, Eric Charles wrote: >>>>>>>>>>> >>>>>>>>>>> OK, so you're using a snapshot of 23/5, just after the commit [1] >>>>>>>>>>> of 20/5. >>>>>>>>>>> >>>>>>>>>>> If you still encounter the OOM, you can further analyse with >>>>>>>>>>> "eclipse >>>>>>>>>>> mat" [2] for example and ask for "suspects". It will show you the >>>>>>>>>>> objects that eat memory (launch james with standard Mx 512M, >>>>>>>>>>> otherwise >>>>>>>>>>> you will need more time to laod dump). >>>>>>>>>>> >>>>>>>>>>> Optionally, you can try to re-add the prefetchPolicy in >>>>>>>>>>> james-server-context.xml (see changes from [1]). >>>>>>>>>>> >>>>>>>>>>> Tks, >>>>>>>>>>> - Eric >>>>>>>>>>> >>>>>>>>>>> [1] http://s.apache.org/30x >>>>>>>>>>> [2] http://www.eclipse.org/mat/ >>>>>>>>>>> >>>>>>>>>>> On 25/05/2011 15:56, Nicola Salvemini wrote: >>>>>>>>>>>> >>>>>>>>>>>> Hi Eric, >>>>>>>>>>>> I'm using snapshot >>>>>>>>>>>> james-server-container-spring-3.0-M3-20110523.043027-400 >>>>>>>>>>>> >>>>>>>>>>>> now I'll try to install snapshot >>>>>>>>>>>> james-server-container-spring-3.0-M3-20110525.115759-407 >>>>>>>>>>>> and I'll add the complete debugging path in the startup script >>>>>>>>>>>> to >>>>>>>>>>>> further analyse the dump. >>>>>>>>>>>> >>>>>>>>>>>> Many thanks for your help, >>>>>>>>>>>> >>>>>>>>>>>> Nicola. >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> On Wed, 2011-05-25 at 15:17 +0200, Eric Charles wrote: >>>>>>>>>>>>> >>>>>>>>>>>>> Tks for your answer. >>>>>>>>>>>>> >>>>>>>>>>>>> So you've got more or less the same configuration as I have: >>>>>>>>>>>>> - maildir >>>>>>>>>>>>> - low smtp traffic (I've got a bit more with the spam, but it's >>>>>>>>>>>>> rejected >>>>>>>>>>>>> via smtp hook) >>>>>>>>>>>>> - not many users >>>>>>>>>>>>> >>>>>>>>>>>>> My config here is really stable after weeks (no oom, low memory >>>>>>>>>>>>> consumption when i monitor via JMX, blob-transfer folders >>>>>>>>>>>>> empty). >>>>>>>>>>>>> >>>>>>>>>>>>> Yes, it sounds like some mails remains blocked, giving the >>>>>>>>>>>>> issue. >>>>>>>>>>>>> Would it come from recent >>>>>>>>>>>>> https://issues.apache.org/jira/browse/JAMES-1253 ? >>>>>>>>>>>>> btw, which version are you using ? >>>>>>>>>>>>> >>>>>>>>>>>>> The complete debugging path would be to add >>>>>>>>>>>>> -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=/your_path in >>>>>>>>>>>>> the >>>>>>>>>>>>> startup script and to further analyse the dump: it will give >>>>>>>>>>>>> for sure >>>>>>>>>>>>> the cause of oom. >>>>>>>>>>>>> >>>>>>>>>>>>> - Eric >>>>>>>>>>>>> >>>>>>>>>>>>> On 25/05/2011 15:02, Nicola Salvemini wrote: >>>>>>>>>>>>>> >>>>>>>>>>>>>> On Wed, 2011-05-25 at 12:16 +0200, Eric Charles wrote: >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Hi, >>>>>>>>>>>>>> >>>>>>>>>>>>>> Hi, >>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Can you give us a rough estimate of the traffic you have? >>>>>>>>>>>>>>> Mainly SMTP? How many incoming mails/requests per second? >>>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> at this time we don't have very high mail traffic, more or >>>>>>>>>>>>>> less 500/600 >>>>>>>>>>>>>> mail per day. >>>>>>>>>>>>>> James is configured as smtp and imap servers and I think that >>>>>>>>>>>>>> most of >>>>>>>>>>>>>> the traffic is IMAP (currently there are about ten active >>>>>>>>>>>>>> clients at the >>>>>>>>>>>>>> same time making requests to the James IMAP server). >>>>>>>>>>>>>> >>>>>>>>>>>>>>> This will allow us to better define the place to look at. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Also, is the CPU always remain 100% when you have that issue? >>>>>>>>>>>>>> >>>>>>>>>>>>>> Yes, it is in most cases. >>>>>>>>>>>>>> >>>>>>>>>>>>>>> Is the log still showing messages after the OOM? >>>>>>>>>>>>>> >>>>>>>>>>>>>> Yes, please consider following log sequence: >>>>>>>>>>>>>> >>>>>>>>>>>>>> DEBUG 04:51:13,425 | james.smtpserver | Unable to process >>>>>>>>>>>>>> request >>>>>>>>>>>>>> java.lang.OutOfMemoryError: PermGen space >>>>>>>>>>>>>> DEBUG 04:51:16,731 | james.imapserver | ID=2085331846 >>>>>>>>>>>>>> Got<tag>: 1125 >>>>>>>>>>>>>> DEBUG 04:51:16,731 | james.imapserver | ID=2085331846 >>>>>>>>>>>>>> Got<command>: >>>>>>>>>>>>>> noop >>>>>>>>>>>>>> DEBUG 04:51:16,731 | james.smtpserver | ID=1200023601 >>>>>>>>>>>>>> executing hook >>>>>>>>>>>>>> >>>>>>>>>>>>>> org.apache.james.protocols.smtp.core.log.HookResultLogger@5ae99cbb >>>>>>>>>>>>>> DEBUG 04:51:16,731 | james.smtpserver | Unable to process >>>>>>>>>>>>>> request >>>>>>>>>>>>>> java.lang.OutOfMemoryError: PermGen space >>>>>>>>>>>>>> DEBUG 04:51:16,731 | james.smtpserver | Unable to process >>>>>>>>>>>>>> request >>>>>>>>>>>>>> java.lang.OutOfMemoryError: PermGen space >>>>>>>>>>>>>> DEBUG 04:51:13,425 | james.smtpserver | Unable to process >>>>>>>>>>>>>> request >>>>>>>>>>>>>> java.lang.OutOfMemoryError: PermGen space >>>>>>>>>>>>>> DEBUG 04:51:13,425 | james.imapserver | ID=2140019352 >>>>>>>>>>>>>> Got<tag>: A05974 >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> I noticed that sometimes the problem occurs for a "blockage" >>>>>>>>>>>>>> of >>>>>>>>>>>>>> ActiveMQ. For some reason it happens that not all spooled mail >>>>>>>>>>>>>> will be >>>>>>>>>>>>>> processed and they remain in "store/activemq/blob-transfer/" >>>>>>>>>>>>>> folder. If >>>>>>>>>>>>>> the number of mail rises too high then the problem occurs. >>>>>>>>>>>>>> >>>>>>>>>>>>>> Last thing: James is installed on a server machine whit >>>>>>>>>>>>>> following >>>>>>>>>>>>>> characteristics: >>>>>>>>>>>>>> OS: Opensuse 11.4 64bit Linux kernel >>>>>>>>>>>>>> 2.6.37.6-0.5 >>>>>>>>>>>>>> java version: 1.6.0_25 >>>>>>>>>>>>>> RAM: 8Gb >>>>>>>>>>>>>> Architecture: x86_64 >>>>>>>>>>>>>> CPU(s): 6 >>>>>>>>>>>>>> On-line CPU(s) list: 0-5 >>>>>>>>>>>>>> Thread(s) per core: 1 >>>>>>>>>>>>>> Core(s) per socket: 6 >>>>>>>>>>>>>> CPU socket(s): 1 >>>>>>>>>>>>>> Vendor ID: AuthenticAMD >>>>>>>>>>>>>> CPU family: 16 >>>>>>>>>>>>>> Model: 8 >>>>>>>>>>>>>> Stepping: 1 >>>>>>>>>>>>>> CPU MHz: 2599.966 >>>>>>>>>>>>>> BogoMIPS: 5200.12 >>>>>>>>>>>>>> Virtualization: AMD-V >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> Thanks for your help, >>>>>>>>>>>>>> >>>>>>>>>>>>>> Nicola. >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Tks, >>>>>>>>>>>>>>> - Eric >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> On 25/05/2011 11:54, Nicola Salvemini wrote: >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Hello, >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> I'm using James 3.0-M3 with maildir. Everything works fine >>>>>>>>>>>>>>>> for one or >>>>>>>>>>>>>>>> two days until I start having memory problems and I need to >>>>>>>>>>>>>>>> restart >>>>>>>>>>>>>>>> James. >>>>>>>>>>>>>>>> The problems begin with a long series of messages such as >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> WARN 04:46:27,928 | >>>>>>>>>>>>>>>> org.apache.activemq.broker.jmx.ManagedRegionBroker >>>>>>>>>>>>>>>> | Failed to register MBean: >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> org.apache.activemq:BrokerName=james,Type=Subscription,persistentMode=Non-Durable,destinationType=Queue,destinationName=spool,clientId=ID_alice-58445-1306246275420-4_1,consumerId=ID_alice-58445-1306246275420-3_1_22_4524 >>>>>>>>>>>>>>>> WARN 04:46:27,928 | >>>>>>>>>>>>>>>> org.apache.activemq.broker.jmx.ManagedRegionBroker >>>>>>>>>>>>>>>> | Failed to register MBean: >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> org.apache.activemq:BrokerName=james,Type=Subscription,persistentMode=Non-Durable,destinationType=Queue,destinationName=spool,clientId=ID_alice-58445-1306246275420-4_1,consumerId=ID_alice-58445-1306246275420-3_1_22_4524 >>>>>>>>>>>>>>>> WARN 04:46:27,928 | >>>>>>>>>>>>>>>> org.apache.activemq.broker.jmx.ManagedRegionBroker >>>>>>>>>>>>>>>> | Failed to register MBean: >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> org.apache.activemq:BrokerName=james,Type=Subscription,persistentMode=Non-Durable,destinationType=Queue,destinationName=spool,clientId=ID_alice-58445-1306246275420-4_1,consumerId=ID_alice-58445-1306246275420-3_1_22_4524 >>>>>>>>>>>>>>>> WARN 04:46:27,928 | >>>>>>>>>>>>>>>> org.apache.activemq.broker.jmx.ManagedRegionBroker >>>>>>>>>>>>>>>> | Failed to register MBean: >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> org.apache.activemq:BrokerName=james,Type=Subscription,persistentMode=Non-Durable,destinationType=Queue,destinationName=spool,clientId=ID_alice-58445-1306246275420-4_1,consumerId=ID_alice-58445-1306246275420-3_1_22_4524 >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> and then >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> DEBUG 04:51:11,445 | james.smtpserver | Unable to process >>>>>>>>>>>>>>>> request >>>>>>>>>>>>>>>> java.lang.OutOfMemoryError: PermGen space >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> The CPU usage is over 100% and my only alternative is to >>>>>>>>>>>>>>>> restart James. >>>>>>>>>>>>>>>> I tried to increase VM memory (at this time I have -Xms512m >>>>>>>>>>>>>>>> -Xmx1024m) >>>>>>>>>>>>>>>> but the problem keeps coming back. >>>>>>>>>>>>>>>> Does anyone have a suggestion to solve the problem? >>>>>>>>>>>>>>>> Thanks in advance, >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Nicola. >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> --------------------------------------------------------------------- >>>>>>>>>>>>>>>> To unsubscribe, e-mail: >>>>>>>>>>>>>>>> [email protected] >>>>>>>>>>>>>>>> For additional commands, e-mail: >>>>>>>>>>>>>>>> [email protected] >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> --------------------------------------------------------------------- >>>>>>>>>>>>>>> To unsubscribe, e-mail: >>>>>>>>>>>>>>> [email protected] >>>>>>>>>>>>>>> For additional commands, e-mail: >>>>>>>>>>>>>>> [email protected] >>>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> --------------------------------------------------------------------- >>>>>>>>>>>>>> To unsubscribe, e-mail: >>>>>>>>>>>>>> [email protected] >>>>>>>>>>>>>> For additional commands, e-mail: >>>>>>>>>>>>>> [email protected] >>>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> --------------------------------------------------------------------- >>>>>>>>>>>>> To unsubscribe, e-mail: >>>>>>>>>>>>> [email protected] >>>>>>>>>>>>> For additional commands, e-mail: >>>>>>>>>>>>> [email protected] >>>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> --------------------------------------------------------------------- >>>>>>>>>>>> To unsubscribe, e-mail: [email protected] >>>>>>>>>>>> For additional commands, e-mail: >>>>>>>>>>>> [email protected] >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> --------------------------------------------------------------------- >>>>>>>>>>> To unsubscribe, e-mail: [email protected] >>>>>>>>>>> For additional commands, e-mail: >>>>>>>>>>> [email protected] >>>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> --------------------------------------------------------------------- >>>>>>>>>> To unsubscribe, e-mail: [email protected] >>>>>>>>>> For additional commands, e-mail: [email protected] >>>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> --------------------------------------------------------------------- >>>>>>>>> To unsubscribe, e-mail: [email protected] >>>>>>>>> For additional commands, e-mail: [email protected] >>>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> --------------------------------------------------------------------- >>>>>>>> To unsubscribe, e-mail: [email protected] >>>>>>>> For additional commands, e-mail: [email protected] >>>>>>>> >>>>>>> >>>>>>> >>>>>>> --------------------------------------------------------------------- >>>>>>> To unsubscribe, e-mail: [email protected] >>>>>>> For additional commands, e-mail: [email protected] >>>>>>> >>>>>> >>>>>> >>>>>> >>>>>> --------------------------------------------------------------------- >>>>>> To unsubscribe, e-mail: [email protected] >>>>>> For additional commands, e-mail: [email protected] >>>>>> >>>>> >>>>> >>>>> --------------------------------------------------------------------- >>>>> To unsubscribe, e-mail: [email protected] >>>>> For additional commands, e-mail: [email protected] >>>>> >>>> >>>> >>>> >>>> --------------------------------------------------------------------- >>>> To unsubscribe, e-mail: [email protected] >>>> For additional commands, e-mail: [email protected] >>>> >>> >>> >>> --------------------------------------------------------------------- >>> To unsubscribe, e-mail: [email protected] >>> For additional commands, e-mail: [email protected] >>> >> >> >> >> --------------------------------------------------------------------- >> To unsubscribe, e-mail: [email protected] >> For additional commands, e-mail: [email protected] >> > > > --------------------------------------------------------------------- > To unsubscribe, e-mail: [email protected] > For additional commands, e-mail: [email protected] > > --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
