Re: [PATCHES] Feature: POSIX Shared memory support, round 2

2007-02-09 Thread Chris Marcellino
That is strange, because the majority of the comments are new. Much of the code and comments are reused from the SysV code because, you know, this is an enhancement. The comments that are left serve a purpose. In PGSharedMemoryCreate, this implementation avoids the need to tell if live back

Re: [PATCHES] Feature: POSIX Shared memory support, round 2

2007-02-09 Thread Tom Lane
Chris Marcellino <[EMAIL PROTECTED]> writes: > Here is a new patch that uses the POSIX api's. It encodes the > canonical path (see 'man realpath') of the database's data directory > into the shared memory segment name using an strong hash function to > make it fit in the shared memory segment

[PATCHES] Feature: POSIX Shared memory support, round 2

2007-02-08 Thread Chris Marcellino
As discussed earlier, using POSIX shared memory can solve a few issues, On Mac OS X and other BSD's, the default System V shared memory limits are often very low and require adjustment for acceptable performance. Particularly, when Postgres is included as part of larger end-user friendly sof

Re: [PATCHES] Feature: POSIX Shared memory support

2007-02-08 Thread Magnus Hagander
Tom Lane wrote: > Magnus Hagander <[EMAIL PROTECTED]> writes: >> Tom Lane wrote: >>> Magnus Hagander <[EMAIL PROTECTED]> writes: If we just didn't add the serial number at the end, then it would be impossible to create a shared memory segment for the same port again. That protects th

Re: [PATCHES] Feature: POSIX Shared memory support

2007-02-08 Thread Tom Lane
Magnus Hagander <[EMAIL PROTECTED]> writes: > Tom Lane wrote: >> Magnus Hagander <[EMAIL PROTECTED]> writes: >>> If we just didn't add the serial number at the end, then it would be >>> impossible to create a shared memory segment for the same port again. >>> That protects the port and not the data

Re: [PATCHES] Feature: POSIX Shared memory support

2007-02-08 Thread Magnus Hagander
Tom Lane wrote: > Magnus Hagander <[EMAIL PROTECTED]> writes: >> If we just didn't add the serial number at the end, then it would be >> impossible to create a shared memory segment for the same port again. >> That protects the port and not the datadir. But what if we change the >> name of the shar

Re: [PATCHES] Feature: POSIX Shared memory support

2007-02-08 Thread Tom Lane
Magnus Hagander <[EMAIL PROTECTED]> writes: > If we just didn't add the serial number at the end, then it would be > impossible to create a shared memory segment for the same port again. > That protects the port and not the datadir. But what if we change the > name of the shared memory segment to b

Re: [PATCHES] Feature: POSIX Shared memory support

2007-02-08 Thread Magnus Hagander
On Wed, Feb 07, 2007 at 09:40:16AM -0500, Tom Lane wrote: > Magnus Hagander <[EMAIL PROTECTED]> writes: > > On Tue, Feb 06, 2007 at 11:08:51PM -0500, Tom Lane wrote: > >> AFAIK the Windows port is simply wrong/insecure on this point --- it's > >> one of the reasons you'll never see me recommending

Re: [PATCHES] Feature: POSIX Shared memory support

2007-02-07 Thread Alvaro Herrera
Andrew Dunstan wrote: > Perl provides a moderately portable flock(), which we use in fact in > buildfarm to stop it from running more than one at a time on a given repo > copy. > [...] > Maybe we can borrow some code. Probably not, because it's GPL/Artistic; but we could borrow some ideas inste

Re: [PATCHES] Feature: POSIX Shared memory support

2007-02-07 Thread Andrew Dunstan
Alvaro Herrera wrote: Andrew Dunstan wrote: Maybe we should look some more at that. Use of file locking was one thought I had today after I saw Tom's earlier comments. Perl provides a moderately portable flock(), which we use in fact in buildfarm to stop it from running more than one at a t

Re: [PATCHES] Feature: POSIX Shared memory support

2007-02-07 Thread Alvaro Herrera
Andrew Dunstan wrote: > Maybe we should look some more at that. Use of file locking was one > thought I had today after I saw Tom's earlier comments. > > Perl provides a moderately portable flock(), which we use in fact in > buildfarm to stop it from running more than one at a time on a given rep

Re: [PATCHES] Feature: POSIX Shared memory support

2007-02-07 Thread Tom Lane
Magnus Hagander <[EMAIL PROTECTED]> writes: > On Tue, Feb 06, 2007 at 11:08:51PM -0500, Tom Lane wrote: >> AFAIK the Windows port is simply wrong/insecure on this point --- it's >> one of the reasons you'll never see me recommending Windows as the OS >> for a production Postgres server. > What exa

Re: [PATCHES] Feature: POSIX Shared memory support

2007-02-07 Thread Magnus Hagander
On Tue, Feb 06, 2007 at 11:08:51PM -0500, Tom Lane wrote: > "Takayuki Tsunakawa" <[EMAIL PROTECTED]> writes: > > From: "Tom Lane" <[EMAIL PROTECTED]> > >> the POSIX API provides no way to detect whether anyone else is > >> attached to the segment. Not being able to tell that is a tremendous > >> r

Re: [PATCHES] Feature: POSIX Shared memory support

2007-02-06 Thread Andrew Dunstan
Tom Lane wrote: > > We've speculated on occasion about using file locking in some form as a > substitute mechanism for detecting this, but that seems to just bring > its own set of not-too-portable assumptions. > Maybe we should look some more at that. Use of file locking was one thought I had to

Re: [PATCHES] Feature: POSIX Shared memory support

2007-02-06 Thread Chris Marcellino
Attached is a beta of the POSIX shared memory layer. It is 75% the original sysv_shmem.c code. I'm looking for ways to refactor it down a bit, while changing as little of the tried-and-tested code as possible. I though I'd put it out there for comments. Of course, unfortunately it is more c

Re: [PATCHES] Feature: POSIX Shared memory support

2007-02-06 Thread Chris Marcellino
Yes, as Tom pointed out. Sorry, I misread the autoconf file. I've gotten quite used to Darwin == BSD. I've added a note to my todo list to look into the posix semaphore performance on the Darwin side. --Chris On Feb 6, 2007, at 8:32 PM, Takayuki Tsunakawa wrote: Then, how about semaphores?

Re: [PATCHES] Feature: POSIX Shared memory support

2007-02-06 Thread Takayuki Tsunakawa
>> Then, how about semaphores? When I just do configure, PostgreSQL >> seems to use SysV semaphores. But POSIX semaphore implementation is >> prepared in src/backend/port/posix_sema.c. Why isn't it used by >> default? Does it have any problem? >> > > Either way, Essentially, no one is running o

Re: [PATCHES] Feature: POSIX Shared memory support

2007-02-06 Thread Takayuki Tsunakawa
ep ---(end of broadcast)--- TIP 1: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly

Re: [PATCHES] Feature: POSIX Shared memory support

2007-02-06 Thread Tom Lane
Chris Marcellino <[EMAIL PROTECTED]> writes: > As Tom pointed out, the code I posted yesterday is not robust enough > for general consumption. I'm working on a better solution, which will > likely involve using a very small SysV shmem segment as a mutex of > sorts (as Michael Paesold suggeste

Re: [PATCHES] Feature: POSIX Shared memory support

2007-02-06 Thread Tom Lane
"Takayuki Tsunakawa" <[EMAIL PROTECTED]> writes: > From: "Tom Lane" <[EMAIL PROTECTED]> >> the POSIX API provides no way to detect whether anyone else is >> attached to the segment. Not being able to tell that is a tremendous >> robustness hit for us. > How is this done on Windows? Is it possibl

Re: [PATCHES] Feature: POSIX Shared memory support

2007-02-06 Thread Chris Marcellino
Responses inline. On Feb 6, 2007, at 7:05 PM, Takayuki Tsunakawa wrote: From: "Chris Marcellino" <[EMAIL PROTECTED]> To this end, I have "ported" the svsv_shmem.c layer to use the POSIX calls (which are some ways more robust w.r.t reducing collision by using strings as shared memory id's, inst

Re: [PATCHES] Feature: POSIX Shared memory support

2007-02-06 Thread Takayuki Tsunakawa
From: "Chris Marcellino" <[EMAIL PROTECTED]> > To this end, I have "ported" the svsv_shmem.c layer to use the POSIX > calls (which are some ways more robust w.r.t reducing collision by > using strings as shared memory id's, instead of ints). I hope your work will be accepted. Setting IPC paramete

Re: [PATCHES] Feature: POSIX Shared memory support

2007-02-06 Thread Tom Lane
Chris Marcellino <[EMAIL PROTECTED]> writes: > I was actually referring to the check that the current SysV code does > on the pid that is stored in the shmem header. I presume that if the > backend is dead, the kill(hdr->creatorPID, 0) returning zero would > suffice for confirming the existen

Re: [PATCHES] Feature: POSIX Shared memory support

2007-02-06 Thread Chris Marcellino
To my knowledge there is unfortunately not a portable call that does that. I was actually referring to the check that the current SysV code does on the pid that is stored in the shmem header. I presume that if the backend is dead, the kill(hdr->creatorPID, 0) returning zero would suffice fo

Re: [PATCHES] Feature: POSIX Shared memory support

2007-02-06 Thread Alvaro Herrera
Chris Marcellino wrote: > Tom, that is a definitely valid point and thanks for the feedback. I > assume that the 'more modern' string segment naming gave the POSIX > methods an edge in avoiding collision between other apps. > As far as detecting a) whether anyone else is currently attached to

Re: [PATCHES] Feature: POSIX Shared memory support

2007-02-06 Thread Chris Marcellino
Tom, that is a definitely valid point and thanks for the feedback. I assume that the 'more modern' string segment naming gave the POSIX methods an edge in avoiding collision between other apps. As far as detecting a) whether anyone else is currently attached to that segment and b) whether an

Re: [PATCHES] Feature: POSIX Shared memory support

2007-02-06 Thread Michael Paesold
Tom Lane wrote: Chris Marcellino <[EMAIL PROTECTED]> writes: To this end, I have "ported" the svsv_shmem.c layer to use the POSIX calls (which are some ways more robust w.r.t reducing collision by using strings as shared memory id's, instead of ints). This has been suggested before, and rej

Re: [PATCHES] Feature: POSIX Shared memory support

2007-02-06 Thread Tom Lane
Chris Marcellino <[EMAIL PROTECTED]> writes: > To this end, I have "ported" the svsv_shmem.c layer to use the POSIX > calls (which are some ways more robust w.r.t reducing collision by > using strings as shared memory id's, instead of ints). This has been suggested before, and rejected before,

[PATCHES] Feature: POSIX Shared memory support

2007-02-06 Thread Chris Marcellino
On Mac OS X and other BSD's, the default System V shared memory limits are often very low and require adjustment for acceptable performance. Particularly, when Postgres is included as part of larger end-user friendly software products, these kernel settings are often difficult to change for