Scott - thanks, in pouring over the logs, I have not found anything certain, but have turned up a ton of mesages about my sysadmins se-linux security and php/pg (don't know if they're my app or script kiddies...). I will keep looking for things pertianing to these crashes. The message relative to php and pgsql: Sep 20 11:37:32 deq1 setroubleshoot: SELinux is preventing php (httpd_sys_script_t) "setopt" to <Unknown> (httpd_sys_script_t). For complete SELinux messages. run sealert -l 199268c4-b84d-4a33-a073-29bd4461f875
Joe - it appears that it ALWAYS involves pLR - even a simple median call has caused it, though I must say it is something that is calculating the median of somewhere around 10-20,000 pieces of data if that makes any difference. I would be delighted to run any kind of debugging necessary and share the info. I have an identical system that can reproduce the errors (I am pretty certain that they HAVE previously). What I DON'T have is any knowledge of the stack-trace/debugger things, but I'm willing to learn, and I have a sysadmin who may be able to lend a hand. Thanks a bunch gents - I have been nibbling around the edges of this problem for quite some time and I am ready to take a bite, r.b. -----Original Message----- From: Scott Marlowe [mailto:[email protected]] Sent: Fri 9/23/2011 3:42 PM To: Burgholzer, Robert (DEQ) Cc: Joe Conway; [email protected] Subject: Re: [ADMIN] diagnosing a db crash - server exit code 2 On Fri, Sep 23, 2011 at 1:38 PM, Burgholzer, Robert (DEQ) <[email protected]> wrote: > Joe, > Thanks - I will try to check into this - however, we have done some tuning > on the memory over the last 2 years and gotten it such that it is seldom if > every having to dip into its swap too substantially -- according to "top", > we remain under 1% swap usage during most times. Generally speaking, I run > 3-5 of these large PHP processes simultaneously with no issue, however, when > I issue even a "median" function call, after several calls (no consistent > pattern that I can discern), the backend crashes. > > If this were the OOM killer - any way I would diagnose it? If it's the OOM killer it'll be in your /var/log/messages log.
