Hmm.. never saw this problem. But could you try to move the activemq dir out of the way.
# mv /path/to/james/var/store/activemq /path/to/james/var/store/activemq.crc After that try to start again.. Bye, Norman 2011/9/15 Zach Cox <[email protected]>: > And now every time I start James, after it completes startup, it just > shuts down: > > > 2011-09-15 17:56:26 INFO smtpserver:? - Init SMTP Service done > 2011-09-15 17:56:26 DEBUG ConfigurationUtils:? - > ConfigurationUtils.locate(): base is null, name is null > 2011-09-15 17:56:26 INFO lmtpserver:? - LMTP Service disabled by > configuration > 2011-09-15 17:56:26 DEBUG ConfigurationUtils:? - > ConfigurationUtils.locate(): base is null, name is null > 2011-09-15 17:56:26 INFO pop3server:? - POP3 Service disabled by > configuration > 2011-09-15 17:56:27 DEBUG ConfigurationUtils:? - > ConfigurationUtils.locate(): base is null, name is null > 2011-09-15 17:56:27 INFO imapserver:? - IMAP Service disabled by > configuration > 2011-09-15 17:56:27 DEBUG ConfigurationUtils:? - > ConfigurationUtils.locate(): base is null, name is null > 2011-09-15 17:56:27 INFO usersrepository23:? - Init > org.apache.james.repository.file.FilePersistentObjectRepository Store > 2011-09-15 17:56:27 INFO usersrepository23:? - > org.apache.james.repository.file.FilePersistentObjectRepository opened > in ../var/users > 2011-09-15 17:56:27 DEBUG usersrepository23:? - > org.apache.james.user.file.UsersFileRepository created in > file://var/users/ > 2011-09-15 17:56:27 INFO imapserver:? - Dispose IMAP Service > 2011-09-15 17:56:27 INFO imapserver:? - Dispose IMAP Service done > 2011-09-15 17:56:27 INFO pop3server:? - Dispose POP3 Service > 2011-09-15 17:56:27 INFO pop3server:? - Dispose POP3 Service done > 2011-09-15 17:56:27 INFO lmtpserver:? - Dispose LMTP Service > 2011-09-15 17:56:27 INFO lmtpserver:? - Dispose LMTP Service done > 2011-09-15 17:56:27 INFO smtpserver:? - Dispose SMTP Service > 2011-09-15 17:56:27 INFO smtpserver:? - Dispose SMTP Service done > 2011-09-15 17:56:27 INFO mailspooler:? - > org.apache.james.mailetcontainer.impl.JamesMailSpooler dispose... > 2011-09-15 17:56:27 INFO mailspooler:? - Stop > org.apache.james.mailetcontainer.impl.JamesMailSpooler: dequeuer-1 > 2011-09-15 17:56:27 INFO mailspooler:? - Stop > org.apache.james.mailetcontainer.impl.JamesMailSpooler: dequeuer-2 > 2011-09-15 17:56:27 INFO mailspooler:? - > org.apache.james.mailetcontainer.impl.JamesMailSpooler thread shutdown > completed. > 2011-09-15 17:56:30 INFO SocketNode:94 - Caught java.io.EOFException > closing conneciton. > > > Any advice would be greatly appreciated. > > > > On Thu, Sep 15, 2011 at 12:54 PM, Zach Cox <[email protected]> wrote: >> OK now several minutes after I restarted James, everything was working >> fine, then all of a sudden I see this while tailing james-server.log: >> >> 2011-09-15 17:51:24 WARN DefaultShutdownStrategy:? - Timeout >> occurred. Now forcing the routes to be shutdown now. >> 2011-09-15 17:51:24 WARN DefaultShutdownStrategy:? - Interrupted >> while waiting during graceful shutdown, will force shutdown now. >> 2011-09-15 17:51:24 WARN DefaultInflightRepository:? - Shutting down >> while there are still 8 in flight exchanges. >> 2011-09-15 17:51:26 INFO SocketNode:94 - Caught java.io.EOFException >> closing conneciton. >> >> >> And James just shuts down. This is incredibly bad to have a mail >> server just shut down out of the blue... >> >> >> >> On Thu, Sep 15, 2011 at 12:46 PM, Zach Cox <[email protected]> wrote: >>> Also when James is starting up I see this in logs: >>> >>> 2011-09-15 17:45:05 WARN AMQPersistenceAdapter:? - The ReferenceStore >>> is not valid - recovering ... >>> >>> >>> I'm guessing that's not a good thing? >>> >>> >>> On Thu, Sep 15, 2011 at 11:54 AM, Zach Cox <[email protected]> wrote: >>>> I don't think we're seeing the problem again yet, but I just had to >>>> restart James and noticed this on shutdown: >>>> >>>> 2011-09-15 16:48:31 INFO JamesServerApplicationContext:? - Closing >>>> org.apache.james.container.spring.context.JamesServerApplicationContext@86fe26: >>>> startup date [Thu Sep 15 12:20:21 UTC 2011]; root of context hierarchy >>>> 2011-09-15 16:48:31 INFO imapserver:? - Dispose IMAP Service >>>> 2011-09-15 16:48:31 INFO imapserver:? - Dispose IMAP Service done >>>> 2011-09-15 16:48:31 INFO pop3server:? - Dispose POP3 Service >>>> 2011-09-15 16:48:31 INFO pop3server:? - Dispose POP3 Service done >>>> 2011-09-15 16:48:31 INFO lmtpserver:? - Dispose LMTP Service >>>> 2011-09-15 16:48:31 INFO lmtpserver:? - Dispose LMTP Service done >>>> 2011-09-15 16:48:31 INFO smtpserver:? - Dispose SMTP Service >>>> 2011-09-15 16:48:31 INFO smtpserver:? - Dispose SMTP Service done >>>> 2011-09-15 16:48:31 INFO mailspooler:? - >>>> org.apache.james.mailetcontainer.impl.JamesMailSpooler dispose... >>>> 2011-09-15 16:48:32 INFO mailspooler:? - Stop >>>> org.apache.james.mailetcontainer.impl.JamesMailSpooler: dequeuer-2 >>>> 2011-09-15 16:48:32 INFO mailspooler:? - Stop >>>> org.apache.james.mailetcontainer.impl.JamesMailSpooler: dequeuer-1 >>>> 2011-09-15 16:48:51 INFO SocketNode:94 - Caught java.io.EOFException >>>> closing conneciton. >>>> >>>> >>>> Is that java.io.EOFException normal and expected? Or is it a sign of >>>> some problems? >>>> >>>> Thanks, >>>> Zach >>>> >>>> >>>> >>>> >>>> >>>> On Thu, Sep 15, 2011 at 8:35 AM, Norman Maurer >>>> <[email protected]> wrote: >>>>> Using: >>>>> >>>>> # jstack <pid> >>>>> >>>>> >>>>> Please take 3 within on minute ... >>>>> >>>>> Bye, >>>>> Norman >>>>> >>>>> 2011/9/15 Zach Cox <[email protected]>: >>>>>> I had to restart James several times this morning so nothing is >>>>>> blocked at the moment. >>>>>> >>>>>> There is nothing indicated in the log files. Things look normal, and >>>>>> then they look like this: >>>>>> >>>>>> 2011-09-15 01:25:26 INFO smtpserver:? - ID=8443803 Connection >>>>>> established from x >>>>>> 2011-09-15 01:25:31 INFO smtpserver:? - ID=8443803 Successfully >>>>>> spooled mail Mail1316049931087-05b8199e-e01f-4d14-84d5-3d7ac2443bb3 >>>>>> from x on x for [x] >>>>>> 2011-09-15 01:25:36 INFO smtpserver:? - ID=8443803 Connection closed >>>>>> for x >>>>>> 2011-09-15 01:25:47 INFO smtpserver:? - ID=11521429 Connection >>>>>> established from x >>>>>> 2011-09-15 01:25:48 INFO smtpserver:? - ID=11521429 Successfully >>>>>> spooled mail Mail1316049948254-abd67e8f-bafe-41ab-8eef-c551420fd56e >>>>>> from x on x for [x] >>>>>> 2011-09-15 01:25:48 INFO smtpserver:? - ID=11521429 Connection closed >>>>>> for x >>>>>> 2011-09-15 01:26:41 INFO smtpserver:? - ID=5675333 Connection >>>>>> established from x >>>>>> 2011-09-15 01:26:52 INFO smtpserver:? - ID=5675333 Successfully >>>>>> spooled mail Mail1316050012621-522b1f2a-2851-46fb-940c-711591f637cc >>>>>> from x on x for [x] >>>>>> 2011-09-15 01:26:57 INFO smtpserver:? - ID=5675333 Connection closed >>>>>> for x >>>>>> 2011-09-15 01:27:52 INFO smtpserver:? - ID=33404412 Connection >>>>>> established from localhost (127.0.0.1) >>>>>> 2011-09-15 01:27:52 INFO smtpserver:? - ID=33404412 Connection closed >>>>>> for localhost (127.0.0.1) >>>>>> 2011-09-15 01:28:12 INFO smtpserver:? - ID=11673762 Connection >>>>>> established from x >>>>>> 2011-09-15 01:28:22 INFO smtpserver:? - ID=11673762 Successfully >>>>>> spooled mail Mail1316050102393-6fcc21aa-8f32-4935-b0eb-df892d54a5e8 >>>>>> from x on x for [x] >>>>>> 2011-09-15 01:28:27 INFO smtpserver:? - ID=11673762 Connection closed >>>>>> for x >>>>>> >>>>>> So every mail that comes in just gets spooled and not processed. >>>>>> >>>>>> When this starts happening again, what's the best way to take a thread >>>>>> dump from James? >>>>>> >>>>>> Thanks, >>>>>> Zach >>>>>> >>>>>> >>>>>> On Thu, Sep 15, 2011 at 8:13 AM, Norman Maurer >>>>>> <[email protected]> wrote: >>>>>>> Hi there, >>>>>>> >>>>>>> please take a thread-dump to see why its blocked... >>>>>>> >>>>>>> Anything in the logs ? >>>>>>> >>>>>>> Bye, >>>>>>> Norman >>>>>>> >>>>>>> >>>>>>> 2011/9/15 Zach Cox <[email protected]>: >>>>>>>> We upgraded to James 3.0-beta3 on Tuesday night. Things worked fine >>>>>>>> until around 9pm last night, where exactly the same problem started >>>>>>>> occurring: mail would get spooled, but not processed. After >>>>>>>> restarting James, some of the mail was then processed successfully, >>>>>>>> but most was lost. >>>>>>>> >>>>>>>> We consider this an emergency situation for our company and need to >>>>>>>> find a resolution ASAP. What kind of information can we provide to >>>>>>>> help find a resolution? If financial resources need to be involved to >>>>>>>> find a resolution or move things along, that is an option. >>>>>>>> >>>>>>>> Any help is most greatly appreciated. >>>>>>>> >>>>>>>> Thanks, >>>>>>>> Zach >>>>>>>> >>>>>>>> >>>>>>>> On Tue, Sep 13, 2011 at 12:41 PM, Norman Maurer >>>>>>>> <[email protected]> wrote: >>>>>>>>> No problems... >>>>>>>>> >>>>>>>>> If you still can replicate then please open a issue with a stack trace >>>>>>>>> attached.. >>>>>>>>> >>>>>>>>> Bye, >>>>>>>>> Norman >>>>>>>>> >>>>>>>>> >>>>>>>>> 2011/9/13 Raju Buchi <[email protected]>: >>>>>>>>>> James 3.0-M2. I will try to stand up a new server with beta3 and see >>>>>>>>>> if I >>>>>>>>>> can replicate the issue. >>>>>>>>>> >>>>>>>>>> Thanks >>>>>>>>>> >>>>>>>>>> - Raju >>>>>>>>>> >>>>>>>>>> On Tue, Sep 13, 2011 at 12:37 PM, Norman Maurer < >>>>>>>>>> [email protected]> wrote: >>>>>>>>>> >>>>>>>>>>> Do you use beta3 ? >>>>>>>>>>> >>>>>>>>>>> Bye, >>>>>>>>>>> Norman >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> 2011/9/13 Raju Buchi <[email protected]>: >>>>>>>>>>> > Hello >>>>>>>>>>> > >>>>>>>>>>> > We are also facing exactly the same issues. Tried to debug my >>>>>>>>>>> > custom >>>>>>>>>>> mailets >>>>>>>>>>> > and seems to be everything is working fine. Logs did not show any >>>>>>>>>>> exception >>>>>>>>>>> > messages except it will always spool a mail and will never >>>>>>>>>>> > attempt to >>>>>>>>>>> send >>>>>>>>>>> > it. i.e. all mails are stuck in the spool. >>>>>>>>>>> > >>>>>>>>>>> > - Raju >>>>>>>>>>> > >>>>>>>>>>> > On Tue, Sep 13, 2011 at 12:20 PM, Zach Cox <[email protected]> >>>>>>>>>>> > wrote: >>>>>>>>>>> > >>>>>>>>>>> >> We are having a severe problem with James 3.0-M2 where it will >>>>>>>>>>> >> operate >>>>>>>>>>> >> correctly for hours or days, and then all of a sudden it will not >>>>>>>>>>> >> process a lot of the mail it spools. Sometimes it will not >>>>>>>>>>> >> process >>>>>>>>>>> >> *any* mail that it spools, and other times it will spool a bunch >>>>>>>>>>> >> of >>>>>>>>>>> >> mail, and then process others seemingly at random. I have to >>>>>>>>>>> >> restart >>>>>>>>>>> >> James to get that spooled mail to be processed. But even then, >>>>>>>>>>> >> sometimes the spooled mail is completely lost after a restart. >>>>>>>>>>> >> I have >>>>>>>>>>> >> seen several other threads lately with similar problems, but they >>>>>>>>>>> >> haven't helped our situation. >>>>>>>>>>> >> >>>>>>>>>>> >> We do not store mail in James for personal accounts and we have >>>>>>>>>>> >> IMAP & >>>>>>>>>>> >> POP3 disabled. Every inbound email has one of two things done >>>>>>>>>>> >> to it: >>>>>>>>>>> >> - it is processed locally by custom mailets & matchers >>>>>>>>>>> >> - or it is relayed to our corporate Google Apps account >>>>>>>>>>> >> >>>>>>>>>>> >> A previous thread recommended checking all custom mailets and >>>>>>>>>>> >> matchers. As far as I can tell, they are all working properly. >>>>>>>>>>> >> There >>>>>>>>>>> >> are no exceptions or errors in log/james-server.log, and in the >>>>>>>>>>> >> scenario where James spools some mail & processes others, the >>>>>>>>>>> >> mail >>>>>>>>>>> >> that is processed is handled properly by our custom mailets & >>>>>>>>>>> >> matchers. >>>>>>>>>>> >> >>>>>>>>>>> >> I've tried connecting to James JMX remotely via JConsole but >>>>>>>>>>> >> have not >>>>>>>>>>> >> been able to connect. Which port does James run JMX on? Any >>>>>>>>>>> >> help >>>>>>>>>>> >> with JMX would be appreciated. >>>>>>>>>>> >> >>>>>>>>>>> >> Does anyone have any other ideas about what we should look for? >>>>>>>>>>> >> I >>>>>>>>>>> >> noticed 3.0-beta3 is out now, were there fixes in beta3 related >>>>>>>>>>> >> to >>>>>>>>>>> >> mail being spooled but not processed? >>>>>>>>>>> >> >>>>>>>>>>> >> Thanks, >>>>>>>>>>> >> Zach >>>>>>>>>>> >> >>>>>>>>>>> >> --------------------------------------------------------------------- >>>>>>>>>>> >> To unsubscribe, e-mail: [email protected] >>>>>>>>>>> >> For additional commands, e-mail: >>>>>>>>>>> >> [email protected] >>>>>>>>>>> >> >>>>>>>>>>> >> >>>>>>>>>>> > >>>>>>>>>>> >>>>>>>>>>> --------------------------------------------------------------------- >>>>>>>>>>> To unsubscribe, e-mail: [email protected] >>>>>>>>>>> For additional commands, e-mail: [email protected] >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>> >>>>>>>>> >>>>>>>>> --------------------------------------------------------------------- >>>>>>>>> To unsubscribe, e-mail: [email protected] >>>>>>>>> For additional commands, e-mail: [email protected] >>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>>> --------------------------------------------------------------------- >>>>>>>> To unsubscribe, e-mail: [email protected] >>>>>>>> For additional commands, e-mail: [email protected] >>>>>>>> >>>>>>>> >>>>>>> >>>>>>> --------------------------------------------------------------------- >>>>>>> To unsubscribe, e-mail: [email protected] >>>>>>> For additional commands, e-mail: [email protected] >>>>>>> >>>>>>> >>>>>> >>>>>> --------------------------------------------------------------------- >>>>>> To unsubscribe, e-mail: [email protected] >>>>>> For additional commands, e-mail: [email protected] >>>>>> >>>>>> >>>>> >>>>> --------------------------------------------------------------------- >>>>> To unsubscribe, e-mail: [email protected] >>>>> For additional commands, e-mail: [email protected] >>>>> >>>>> >>>> >>> >> > > --------------------------------------------------------------------- > To unsubscribe, e-mail: [email protected] > For additional commands, e-mail: [email protected] > > --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
