Thanks Ben and all others who wrote to help The problem if you are interested in the solution was two filters which someone had written getting into a loop!! Disabling one of the filters fixed the problem. I would have thought the ARS Server would/should be able to handle this type of problem and not just crash. The problem was, and this has still not been identified, nothing was getting written to ANY log files!! so the analysis of the log files did not show anything. Many thanks once again Shafqat
--- On Thu, 9/25/08, Ben Chernys <[EMAIL PROTECTED]> wrote: From: Ben Chernys <[EMAIL PROTECTED]> Subject: Re: Server problems To: [email protected] Date: Thursday, September 25, 2008, 10:13 PM ** It cannot be the assignment engine. The assignment engine got a 90 which means he was the first to issue an api call (unsuccessfully, so in fact having done nothing on the server at all) and the server was not listening on a port. A Segmentation Violation is the result of a bad pointer in the arserverd c code. This can happen for a number of reasons: a stray store of a byte onto the wrong place in the stack can go unnoticed for a long time and in many different installations. While adjusting the callers / clients CAN make such an event go away (by avoiding a specific code path in arserverd) ANY Segmentation Violation is a result of bad code in arserverd. On behalf of the arserverd coders, I would venture to say ALL non-trivial software will have some of these lurking around. In any event, I would turn on pretty much all logging (api, sql, filter), non-buffered, into a single file. I would perhaps monitor the growth of the file deleting it every now and then. I would also look at any binary patches issued that may have corrected the problem. Binary patches are relatively easy to install and back out though of-course, if the problem is very hard to reproduce, testing a version is always difficult. Cheers Ben Chernys Software Tool House Inc. Canada / Deutschland / Germany Mobile: +49 162 175 0956 GMT + 1 + [ DST ] Email: mailto:[EMAIL PROTECTED] Web: http://www.softwaretoolhouse.com A free notepad for Diary fields: http://www.softwaretoolhouse.com/downloads/DiaryFieldEditor.htm An ARS API scripting tool used for migrations, integrations, imports, reports, extracts, batch jobs: http://www.softwaretoolhouse.com/products/SthMupd From: Action Request System discussion list(ARSList) [mailto:[EMAIL PROTECTED] On Behalf Of Robert Molenda Sent: September 25, 2008 10:49 PM To: [email protected] Subject: Re: Server problems ** if it is happening frequently enough - enable server logging (API) to see what the last thing was... Do not think necessarily it is the Assignment Engine - it might be the one that just "tried it" last so to say - could be email / dso / approval / .... Yes Segmentation Fault (as mentioned above) check memory usage, see if trends happen, etc. HTH Robert On Thu, Sep 25, 2008 at 11:54 AM, Shafqat Ayaz <[EMAIL PROTECTED]> wrote: ** Sorry patch 5. I am looking at the other logs, will let you know what I find thanks shafqat --- On Thu, 9/25/08, Grooms, Frederick W <[EMAIL PROTECTED]> wrote: From: Grooms, Frederick W <[EMAIL PROTECTED]> Subject: Re: Server problems To: [email protected] Date: Thursday, September 25, 2008, 7:21 PM ** A signal 11 is a Segmentation fault error. Which patch level(s) are you on, and what do the other logs show is the last thing executed before the crash? Fred From: Action Request System discussion list(ARSList) [mailto:[EMAIL PROTECTED] On Behalf Of Shafqat Ayaz Sent: Thursday, September 25, 2008 1:07 PM To: [email protected] Subject: Server problems ** Hi All We are having problems with our production server where the assignment engine is dying frequently, arfork is dying frequently, and thus causing the arserver to restart. While this is going on, the users are unable to connect. This is a urgent issue, so I would appreciate any help anyone can provide. I am putting in a bit of excerpt from the arerror.log file. The environment is ARS 7.0.1 Solaris 5.10 ITSM 7 Oracle 10.2.0.3 xxx denotes the server name. Shafqat Wed Sep 24 10:36:14 2008 AssignEng : Cannot establish a network connection to the AR System server (xxx : RPC: Miscellaneous tli error - An event requires attentionNo message of desired type) ARERR - 90 Wed Sep 24 10:36:14 2008 AssignEng : AR System Application server terminated when a signal/exception was received by the server (ARAPPNOTE 4500) Wed Sep 24 10:36:14 2008 11 Wed Sep 24 10:36:14 2008 AssignEng : AR System Application server terminated -- fatal error encountered (ARAPPNOTE 4501) Wed Sep 24 10:37:14 2008 Dispatch : Cannot establish a network connection to the AR System server (xxx : RPC: Miscellaneous tli error - An event requires attentionNo message of desired type) ARERR - 90 Wed Sep 24 10:38:14 2008 Dispatch : Cannot establish a network connection to the AR System server (xxx : RPC: Miscellaneous tli error - An event requires attentionNo message of desired type) ARERR - 90 Wed Sep 24 10:39:14 2008 Dispatch : Cannot establish a network connection to the AR System server (xxx : RPC: Miscellaneous tli error - An event requires attentionNo message of desired type) ARERR - 90 Wed Sep 24 10:40:14 2008 Dispatch : Cannot establish a network connection to the AR System server (xxx : RPC: Miscellaneous tli error - An event requires attentionNo message of desired type) ARERR - 90 Wed Sep 24 10:41:14 2008 Dispatch : Cannot establish a network connection to the AR System server (xxx : RPC: Miscellaneous tli error - An event requires attentionNo message of desired type) ARERR - 90 Wed Sep 24 10:43:42 2008 390603 : __Platinum Sponsor: www.rmsportal.com ARSlist: "Where the Answers Are" html___ __Platinum Sponsor: www.rmsportal.com ARSlist: "Where the Answers Are" html___ __Platinum Sponsor: www.rmsportal.com ARSlist: "Where the Answers Are" html___ -- __Platinum Sponsor: www.rmsportal.com ARSlist: "Where the Answers Are" html___ __Platinum Sponsor: www.rmsportal.com ARSlist: "Where the Answers Are" html___ _______________________________________________________________________________ UNSUBSCRIBE or access ARSlist Archives at www.arslist.org Platinum Sponsor: www.rmsportal.com ARSlist: "Where the Answers Are"
