Re: [HACKERS] Can't find thread on Linux memory overcommit

2003-08-21 Thread Josh Berkus
Folks, > > Well, sadly, the reason I posted is that I (apparently) had a client's > > database fatally corrupted by this problem. OK, diagnosis progresses, it's not the Linux OOM problem, it's something else. -- -Josh Berkus Aglio Database Solutions San Francisco --

Re: [HACKERS] Can't find thread on Linux memory overcommit

2003-08-21 Thread Andrew Dunstan
(er, that's "Andrew" :-) It depends how important your data is to you. A modest server probably costs a few thousand dollars. How much does an expert to install a custom kernel cost? Probably about the same. I believe paranoid mode is supposed to prevent any overcommiting that can't later be h

Re: [HACKERS] Can't find thread on Linux memory overcommit

2003-08-21 Thread Tom Lane
Kurt Roeckx <[EMAIL PROTECTED]> writes: > On Thu, Aug 21, 2003 at 01:04:13PM -0400, Tom Lane wrote: >> "Fatally corrupted"? That should not happen --- at worst, an OOM kill >> should lose your current sessions. We need more details. > Even if it kills the postmaster? Then you'd have to start a

Re: [HACKERS] Can't find thread on Linux memory overcommit

2003-08-21 Thread Josh Berkus
Alan, > You need to be careful using Alan's patch. The reason RH stopped using > this part of it in their errata kernels is that it had conflicts with > other stuff, specifically the rmap stuff (he told me that himself in > email). Hmmm ... that leaves us without a workaround for this problem, th

Re: [HACKERS] Can't find thread on Linux memory overcommit

2003-08-21 Thread Kurt Roeckx
On Thu, Aug 21, 2003 at 01:04:13PM -0400, Tom Lane wrote: > Josh Berkus <[EMAIL PROTECTED]> writes: > > Well, sadly, the reason I posted is that I (apparently) had a client's > > database fatally corrupted by this problem. > > "Fatally corrupted"? That should not happen --- at worst, an OOM ki

Re: [HACKERS] Can't find thread on Linux memory overcommit

2003-08-21 Thread Josh Berkus
Tom, > "Fatally corrupted"? That should not happen --- at worst, an OOM kill > should lose your current sessions. We need more details. Joe and I are diagnosing. Likely the files will come to you before the end of the day. -- Josh Berkus Aglio Database Solutions San Francisco

Re: [HACKERS] Can't find thread on Linux memory overcommit

2003-08-21 Thread Andrew Dunstan
I replied to Josh thus: --- You need to be careful using Alan's patch. The reason RH stopped using this part of it in their errata kernels is that it had conflicts with other stuff, specifically the rmap stuff (he told me that himself in email). I am very wary of advising

Re: [HACKERS] Can't find thread on Linux memory overcommit

2003-08-21 Thread Tom Lane
Josh Berkus <[EMAIL PROTECTED]> writes: > Well, sadly, the reason I posted is that I (apparently) had a client's > database fatally corrupted by this problem. "Fatally corrupted"? That should not happen --- at worst, an OOM kill should lose your current sessions. We need more details.

Re: [HACKERS] Can't find thread on Linux memory overcommit

2003-08-21 Thread Josh Berkus
Andrew, > I see btw that no change has been made to the docs. That's bad IMNSHO. > The situation with RH is unchanged with today's kernel errata patch, > too. I propose to submit a doc patch with the following wording, unless > someone objects or improves it: First, off, I'm crossing this to PGSQ

Re: [HACKERS] Can't find thread on Linux memory overcommit

2003-08-21 Thread Josh Berkus
Guys, > So there's no need to wonder whether that's a source of trouble for your > PostgreSQL processes or not; just check the logs. I've had the OOM killer > go after large Perl processes and X, but never (yet) PostgreSQL, I'm happy > to say. Well, sadly, the reason I posted is that I (apparentl

Re: [HACKERS] Can't find thread on Linux memory overcommit

2003-08-21 Thread Jon Jensen
On Thu, 21 Aug 2003, Andrew Dunstan wrote: > Linux kernel version 2.4.* has poor default memory overcommit behavior, > which can result in the postmaster being killed by the kernel due to > memory demands by another process if the system runs out of memory. To > avoid this situation, run postgr

Re: [HACKERS] Can't find thread on Linux memory overcommit

2003-08-21 Thread Andrew Dunstan
I see btw that no change has been made to the docs. That's bad IMNSHO. The situation with RH is unchanged with today's kernel errata patch, too. I propose to submit a doc patch with the following wording, unless someone objects or improves it: --- Linux kernel version 2.4.*

Re: [HACKERS] Can't find thread on Linux memory overcommit

2003-08-21 Thread Shridhar Daithankar
On Wednesday 20 August 2003 23:57, Andrew Dunstan wrote: > http://archives.postgresql.org/pgsql-hackers/2003-07/msg00608.php > > Subject is "reprise on Linux overcommit handling" - is that too > deceptive? :-) I did little searching on this and found.. http://www.ussg.iu.edu/hypermail/linux/kern

Re: [HACKERS] Can't find thread on Linux memory overcommit

2003-08-20 Thread Thomas Swan
On 8/20/2003 1:02 PM, Josh Berkus wrote: >Hackers, > >I've been searching the archives, but I can't find the thread from last month >where we discussed the problem with Linux memory overcommits in kernel 2.4.x. > >Can someone point me to the right thread? I think maybe the subject line was >so

Re: [HACKERS] Can't find thread on Linux memory overcommit

2003-08-20 Thread Andrew Dunstan
http://archives.postgresql.org/pgsql-hackers/2003-07/msg00608.php Subject is "reprise on Linux overcommit handling" - is that too deceptive? :-) andrew Josh Berkus wrote: Hackers, I've been searching the archives, but I can't find the thread from last month where we discussed the problem wi

[HACKERS] Can't find thread on Linux memory overcommit

2003-08-20 Thread Josh Berkus
Hackers, I've been searching the archives, but I can't find the thread from last month where we discussed the problem with Linux memory overcommits in kernel 2.4.x. Can someone point me to the right thread? I think maybe the subject line was something deceptive -- -Josh Berkus Aglio D