7.6.04? sp2? by chance.. solaris ? On Wed, Feb 15, 2012 at 12:27 AM, Axton <axton.gr...@gmail.com> wrote:
> API, SQL, and filter. Make a copy of them before you restart arserver > so they are not overwritten or otherwise lost. See what the last > entries in the logs are at the time it was hung. This will most > likely point you to the cause. These log entries are just symptomatic > of the underlying issue: > - AssignEng : Timeout during database update > - SLMCS : Timeout during database update > - BRIE : Timeout during database query > - CMDB Dispatcher : Timeout during database query > With the underlying issue being that your app server and/or db server > are unresponsive. When it says unresponsive, it usually means a > rather long time, I think 5 minutes, if my memory serves me correctly. > > If you inspect the SQL logs, you should be able to get a list of the > last sql statements issued to the database server. Look for the > one(s) that have not returned. If all the threads are blocked, you > will get this error. Take a look at the server statistics and see if > you have any queue blocked items. > > Check the Cache-Mode of your server and make sure it is in production > mode. Escalations can cause the server to hang in this way if the > server is in Dev cache mode. > > Not sure what OS you are using, but gathering a stack trace for all > the threads for arserver can be helpful in seeing what arserver is > doing at the time it is hung. > > If possible, generate a core dump from the arserver process before > restarting. This will let you dig into the state of the process at a > later time, after restoring services. > > Axton Grams > > On Tue, Feb 14, 2012 at 2:26 PM, Avy Mann <avym...@yahoo.ca> wrote: > > ** > > Ron, > > > > I wanted to suggest looking at any actions that trigger at 6 am. > > Escalations come to mind because I noticed a line in your log: > > > > Fri Feb 10 06:30:05 2012 BRIE : Timeout during database query -- > consider > > using more specific search criteria to narrow the results, and retry the > > operation (OUR_AR_SERVER) ARERR - 94 > > > > I see this repeated. I am thinking there maybe be something in the > > background running a bad query. Just a thought. > > > > > > ________________________________ > > From: "Peters, Ron" <rpet...@columbia.com> > > To: arslist@ARSLIST.ORG > > Sent: Tuesday, February 14, 2012 3:12:25 PM > > > > Subject: Re: AR Server or DB issue > > > > ** > > Hmm, that KB article is interesting. It sure does sound plausible. If the > > server is stuck in a loop, resetting its brain seems like the right > > solution. Since the server is unresponsive during our issues, what > should I > > look for on the DB to see what is causing the issue? > > > > I’m working with the DBA on this and since it happened last week > mon/wed/fri > > and again yesterday, I can reasonably assume it will happen again > tomorrow > > morning. I’d like to set up some type of process to catch this in the act > > and I’d like to gather all the appropriate data. Thougths? > > > > Thanks. > > > > From: Action Request System discussion list(ARSList) > > [mailto:arslist@ARSLIST.ORG] On Behalf Of Jonas Stumph Stevnsvig > > Sent: Monday, February 13, 2012 2:47 PM > > To: arslist@ARSLIST.ORG > > Subject: Re: AR Server or DB issue > > > > ** > > Hi Ron > > > > I noticed the arAPPnote - indicating an approval notice. I've recently > > hotfxed our servers which were presenting us with ARERR 93 (server busy) > or > > the same error you are getting (ARERR 92). Our servers would hang upon > > handling some approval actions. The symtom turned out to be unbelievable > > numbers of notifications being triggered - by a single Approval on a > change. > > We got 130000 entries in NTE:SYS-NT Proces Control for a single change. > > hotfix was SW00369891 Knowledgebase article is KA 354923 - just a hunch - > > but check your NTE:SYS-NT Proces Control form (or select from the > Database > > if your server is unresponsive as ours was). > > > > br > > > > Jonas > > > > Den 10-02-2012 16:34, Peters, Ron skrev: > > ** > > Hello all, > > > > Having some AR server issues. Something has been happening in the > mornings > > about 6am in that our service desk can’t access Remedy. This started in > > Decemeber but only once. Then twice in January and now a couple times > this > > week. The first few times it resolved itself after about 30 minutes but > this > > week we had to restart the AR service on the server to get it back. The > > following is the arerror log from this morning. I’ve contacted our DBA’s > to > > see if they notice anything but since an AR restart resolves it, I’m not > > convinced it’s a DB issue. Any thoughts? > > > > Thanks. > > > > Fri Feb 10 06:25:30 2012 AssignEng : Timeout during database update -- > the > > operation has been accepted by the server and will usually complete > > successfully (OUR_AR_SERVER) ARERR - 92 > > Fri Feb 10 06:25:30 2012 AssignEng : AR System Application server > > terminated -- fatal error encountered (ARAPPNOTE 4501) > > Fri Feb 10 06:25:31 2012 SLMCS : Timeout during database update -- the > > operation has been accepted by the server and will usually complete > > successfully (OUR_AR_SERVER) ARERR - 92 > > Fri Feb 10 06:25:31 2012 SLMCS : AR System Application server > terminated -- > > fatal error encountered (ARAPPNOTE 4501) > > Fri Feb 10 06:27:32 2012 AssignEng : Timeout during database update -- > the > > operation has been accepted by the server and will usually complete > > successfully (OUR_AR_SERVER) ARERR - 92 > > Fri Feb 10 06:27:32 2012 AssignEng : AR System Application server > > terminated -- fatal error encountered (ARAPPNOTE 4501) > > Fri Feb 10 06:27:32 2012 SLMCS : Timeout during database update -- the > > operation has been accepted by the server and will usually complete > > successfully (OUR_AR_SERVER) ARERR - 92 > > Fri Feb 10 06:27:32 2012 SLMCS : AR System Application server > terminated -- > > fatal error encountered (ARAPPNOTE 4501) > > Fri Feb 10 06:29:32 2012 AssignEng : Timeout during database update -- > the > > operation has been accepted by the server and will usually complete > > successfully (OUR_AR_SERVER) ARERR - 92 > > Fri Feb 10 06:29:32 2012 AssignEng : AR System Application server > > terminated -- fatal error encountered (ARAPPNOTE 4501) > > Fri Feb 10 06:29:32 2012 SLMCS : Timeout during database update -- the > > operation has been accepted by the server and will usually complete > > successfully (OUR_AR_SERVER) ARERR - 92 > > Fri Feb 10 06:29:32 2012 SLMCS : AR System Application server > terminated -- > > fatal error encountered (ARAPPNOTE 4501) > > Fri Feb 10 06:30:05 2012 BRIE : Timeout during database query -- > consider > > using more specific search criteria to narrow the results, and retry the > > operation (OUR_AR_SERVER) ARERR - 94 > > Fri Feb 10 06:31:32 2012 AssignEng : Timeout during database update -- > the > > operation has been accepted by the server and will usually complete > > successfully (OUR_AR_SERVER) ARERR - 92 > > Fri Feb 10 06:31:32 2012 AssignEng : AR System Application server > > terminated -- fatal error encountered (ARAPPNOTE 4501) > > Fri Feb 10 06:31:32 2012 SLMCS : Timeout during database update -- the > > operation has been accepted by the server and will usually complete > > successfully (OUR_AR_SERVER) ARERR - 92 > > Fri Feb 10 06:31:32 2012 SLMCS : AR System Application server > terminated -- > > fatal error encountered (ARAPPNOTE 4501) > > Fri Feb 10 06:33:32 2012 AssignEng : Timeout during database update -- > the > > operation has been accepted by the server and will usually complete > > successfully (OUR_AR_SERVER) ARERR - 92 > > Fri Feb 10 06:33:32 2012 AssignEng : AR System Application server > > terminated -- fatal error encountered (ARAPPNOTE 4501) > > Fri Feb 10 06:33:32 2012 SLMCS : Timeout during database update -- the > > operation has been accepted by the server and will usually complete > > successfully (OUR_AR_SERVER) ARERR - 92 > > Fri Feb 10 06:33:32 2012 SLMCS : AR System Application server > terminated -- > > fatal error encountered (ARAPPNOTE 4501) > > Fri Feb 10 06:35:02 2012 CMDB Dispatcher : Timeout during database > query -- > > consider using more specific search criteria to narrow the results, and > > retry the operation (OUR_AR_SERVER) ARERR - 94 > > Fri Feb 10 06:35:05 2012 BRIE : Timeout during database query -- > consider > > using more specific search criteria to narrow the results, and retry the > > operation (OUR_AR_SERVER) ARERR - 94 > > Fri Feb 10 06:35:32 2012 AssignEng : Timeout during database update -- > the > > operation has been accepted by the server and will usually complete > > successfully (OUR_AR_SERVER) ARERR - 92 > > Fri Feb 10 06:35:32 2012 AssignEng : AR System Application server > > terminated -- fatal error encountered (ARAPPNOTE 4501) > > Fri Feb 10 06:35:32 2012 SLMCS : Timeout during database update -- the > > operation has been accepted by the server and will usually complete > > successfully (OUR_AR_SERVER) ARERR - 92 > > Fri Feb 10 06:35:32 2012 SLMCS : AR System Application server > terminated -- > > fatal error encountered (ARAPPNOTE 4501) > > Fri Feb 10 06:37:32 2012 AssignEng : Timeout during database update -- > the > > operation has been accepted by the server and will usually complete > > successfully (OUR_AR_SERVER) ARERR - 92 > > Fri Feb 10 06:37:32 2012 AssignEng : AR System Application server > > terminated -- fatal error encountered (ARAPPNOTE 4501) > > Fri Feb 10 06:37:32 2012 SLMCS : Timeout during database update -- the > > operation has been accepted by the server and will usually complete > > successfully (OUR_AR_SERVER) ARERR - 92 > > Fri Feb 10 06:37:32 2012 SLMCS : AR System Application server > terminated -- > > fatal error encountered (ARAPPNOTE 4501) > > _attend WWRUG12 www.wwrug.com ARSlist: "Where the Answers Are"_ > > _attend WWRUG12 www.wwrug.com ARSlist: "Where the Answers Are"_ > > _attend WWRUG12 www.wwrug.com ARSlist: "Where the Answers Are"_ > > > > > > _attend WWRUG12 www.wwrug.com ARSlist: "Where the Answers Are"_ > > > _______________________________________________________________________________ > UNSUBSCRIBE or access ARSlist Archives at www.arslist.org > attend wwrug12 www.wwrug12.com ARSList: "Where the Answers Are" > -- Patrick Zandi _______________________________________________________________________________ UNSUBSCRIBE or access ARSlist Archives at www.arslist.org attend wwrug12 www.wwrug12.com ARSList: "Where the Answers Are"
