Re: [HACKERS] POSIX shared memory redux

2011-04-14 Thread Robert Haas
On Thu, Apr 14, 2011 at 7:26 AM, A.M. wrote: > From what I understood, the primary purpose of the SysV check was to protect > the shared memory from multiple stompers. The interlock was a neat > side-effect. Not really - the purpose of the interlock is to protect the underlying data files. The

Re: [HACKERS] POSIX shared memory redux

2011-04-14 Thread Martijn van Oosterhout
On Thu, Apr 14, 2011 at 10:26:33AM -0400, A.M. wrote: > 1) the SysV nattch method's primary purpose is to protect the shmem > region. This is no longer necessary in my patch because the shared > memory in unlinked immediately after creation, so only the initial > postmaster and its children have ac

Re: [HACKERS] POSIX shared memory redux

2011-04-14 Thread A.M.
On Apr 13, 2011, at 11:37 PM, Tom Lane wrote: > "A.M." writes: >> 1. As long one keeps SysV shared memory around, the postgresql project >> has to maintain the annoying platform-specific document on how to >> configure the poorly named kernel parameters. > > No, if it's just a small area, I don

Re: [HACKERS] POSIX shared memory redux

2011-04-14 Thread A.M.
On Apr 14, 2011, at 8:22 AM, Florian Weimer wrote: > * Tom Lane: > >> Well, the fundamental point is that "ignoring NFS" is not the real >> world. We can't tell people not to put data directories on NFS, >> and even if we did tell them not to, they'd still do it. And NFS >> locking is not trus

Re: [HACKERS] POSIX shared memory redux

2011-04-14 Thread A.M.
On Apr 13, 2011, at 9:30 PM, Robert Haas wrote: > On Wed, Apr 13, 2011 at 6:11 PM, A.M. wrote: >>> I don't see why we need to get rid of SysV shared memory; needing less >>> of it seems just as good. >> >> 1. As long one keeps SysV shared memory around, the postgresql project has >> to maintai

Re: [HACKERS] POSIX shared memory redux

2011-04-14 Thread Florian Weimer
* Tom Lane: > Well, the fundamental point is that "ignoring NFS" is not the real > world. We can't tell people not to put data directories on NFS, > and even if we did tell them not to, they'd still do it. And NFS > locking is not trustworthy, because the remote lock daemon can crash > and resta

Re: [HACKERS] POSIX shared memory redux

2011-04-13 Thread Tom Lane
"A.M." writes: > 1. As long one keeps SysV shared memory around, the postgresql project > has to maintain the annoying platform-specific document on how to > configure the poorly named kernel parameters. No, if it's just a small area, I don't see that that's an issue. You're going to max out on o

Re: [HACKERS] POSIX shared memory redux

2011-04-13 Thread Robert Haas
On Wed, Apr 13, 2011 at 6:11 PM, A.M. wrote: >> I don't see why we need to get rid of SysV shared memory; needing less >> of it seems just as good. > > 1. As long one keeps SysV shared memory around, the postgresql project has to > maintain the annoying platform-specific document on how to config

Re: [HACKERS] POSIX shared memory redux

2011-04-13 Thread A.M.
On Apr 13, 2011, at 8:36 PM, Robert Haas wrote: > > I don't see why we need to get rid of SysV shared memory; needing less > of it seems just as good. 1. As long one keeps SysV shared memory around, the postgresql project has to maintain the annoying platform-specific document on how to configu

Re: [HACKERS] POSIX shared memory redux

2011-04-13 Thread Tom Lane
Robert Haas writes: > In answer to your off-list question, one of the principle ways I've > seen fcntl() locking fall over and die is when someone removes the > lock file. You might think that this could be avoided by picking > something important like pg_control as the log file, but it turns out

Re: [HACKERS] POSIX shared memory redux

2011-04-13 Thread Robert Haas
On Wed, Apr 13, 2011 at 7:20 AM, A.M. wrote: > The goal of this patch is to eliminate SysV shared memory, not to implement > NFS-capable locking which, as you point out, is virtually impossible. > > As far as I can tell, in the worst case, my patch does not change how > postgresql handles the NF

Re: [HACKERS] POSIX shared memory redux

2011-04-13 Thread A . M .
On Apr 13, 2011, at 2:06 AM, Tom Lane wrote: > "A.M." writes: >> On Apr 11, 2011, at 7:13 PM, Tom Lane wrote: >>> Robert Haas writes: I mean I'm not convinced that fcntl() locking will be as reliable. > >>> I'm not either. Particularly not on NFS. > >> Is there an example of a recent sy

Re: [HACKERS] POSIX shared memory redux

2011-04-12 Thread Tom Lane
"A.M." writes: > On Apr 11, 2011, at 7:13 PM, Tom Lane wrote: >> Robert Haas writes: >>> I mean I'm not convinced that fcntl() locking will be as reliable. >> I'm not either. Particularly not on NFS. > Is there an example of a recent system where fcntl is broken (ignoring NFS)? Well, the fund

Re: [HACKERS] POSIX shared memory redux

2011-04-11 Thread A.M.
On Apr 11, 2011, at 7:13 PM, Tom Lane wrote: > Robert Haas writes: >> On Mon, Apr 11, 2011 at 3:11 PM, A.M. wrote: >>> What do you mean by "leakier"? The goal here is to extinguish SysV shared >>> memory for portability and convenience benefits. The mini-SysV proposal was >>> implemented and

Re: [HACKERS] POSIX shared memory redux

2011-04-11 Thread A.M.
On Apr 11, 2011, at 7:25 PM, Tom Lane wrote: > Robert Haas writes: >> On Sun, Apr 10, 2011 at 5:03 PM, A.M. wrote: >>> To ensure that no two postmasters can startup in the same data directory, I >>> use fcntl range locking on the data directory lock file, which also works >>> properly on (pro

Re: [HACKERS] POSIX shared memory redux

2011-04-11 Thread Tom Lane
Robert Haas writes: > On Sun, Apr 10, 2011 at 5:03 PM, A.M. wrote: >> To ensure that no two postmasters can startup in the same data directory, I >> use fcntl range locking on the data directory lock file, which also works >> properly on (properly configured) NFS volumes. Whenever a postmaster

Re: [HACKERS] POSIX shared memory redux

2011-04-11 Thread Tom Lane
Robert Haas writes: > On Mon, Apr 11, 2011 at 3:11 PM, A.M. wrote: >> What do you mean by "leakier"? The goal here is to extinguish SysV shared >> memory for portability and convenience benefits. The mini-SysV proposal was >> implemented and shot down by Tom Lane. > I mean I'm not convinced th

Re: [HACKERS] POSIX shared memory redux

2011-04-11 Thread Robert Haas
On Mon, Apr 11, 2011 at 3:11 PM, A.M. wrote: > > On Apr 11, 2011, at 6:06 PM, Robert Haas wrote: > >> On Sun, Apr 10, 2011 at 5:03 PM, A.M. wrote: >>> To ensure that no two postmasters can startup in the same data directory, I >>> use fcntl range locking on the data directory lock file, which al

Re: [HACKERS] POSIX shared memory redux

2011-04-11 Thread A.M.
On Apr 11, 2011, at 6:06 PM, Robert Haas wrote: > On Sun, Apr 10, 2011 at 5:03 PM, A.M. wrote: >> To ensure that no two postmasters can startup in the same data directory, I >> use fcntl range locking on the data directory lock file, which also works >> properly on (properly configured) NFS vo

Re: [HACKERS] POSIX shared memory redux

2011-04-11 Thread Robert Haas
On Sun, Apr 10, 2011 at 5:03 PM, A.M. wrote: > To ensure that no two postmasters can startup in the same data directory, I > use fcntl range locking on the data directory lock file, which also works > properly on (properly configured) NFS volumes. Whenever a postmaster or > postmaster child sta

Re: [HACKERS] POSIX shared memory redux

2010-11-14 Thread Robert Haas
On Sun, Nov 14, 2010 at 11:06 AM, Tom Lane wrote: > Martijn van Oosterhout writes: >> The only real solution seems to me to be to keep a small SysV shared >> memory segment for the locking and allocate the rest of the shared >> memory some other way. > > Yeah, that's been discussed.  It throws al

Re: [HACKERS] POSIX shared memory redux

2010-11-14 Thread Tom Lane
Martijn van Oosterhout writes: > The only real solution seems to me to be to keep a small SysV shared > memory segment for the locking and allocate the rest of the shared > memory some other way. Yeah, that's been discussed. It throws all the portability gains out the window. It might get you o

Re: [HACKERS] POSIX shared memory redux

2010-11-14 Thread Martijn van Oosterhout
On Sat, Nov 13, 2010 at 08:07:52PM -0500, Tom Lane wrote: > "A.M." writes: > > The goal of this work is to address all of the shortcomings of previous > > POSIX shared memory patches as pointed out mostly by Tom Lane. > > It seems like you've failed to understand the main shortcoming of this > w

Re: [HACKERS] POSIX shared memory redux

2010-11-13 Thread Tom Lane
"A.M." writes: > The goal of this work is to address all of the shortcomings of previous POSIX > shared memory patches as pointed out mostly by Tom Lane. It seems like you've failed to understand the main shortcoming of this whole idea, which is the loss of ability to detect pre-existing backend