Re: [PATCHES] Feature: POSIX Shared memory support, round 2

2007-02-09 Thread Tom Lane
Chris Marcellino [EMAIL PROTECTED] writes: Here is a new patch that uses the POSIX api's. It encodes the canonical path (see 'man realpath') of the database's data directory into the shared memory segment name using an strong hash function to make it fit in the shared memory segment name

Re: [PATCHES] Feature: POSIX Shared memory support

2007-02-08 Thread Magnus Hagander
On Wed, Feb 07, 2007 at 09:40:16AM -0500, Tom Lane wrote: Magnus Hagander [EMAIL PROTECTED] writes: On Tue, Feb 06, 2007 at 11:08:51PM -0500, Tom Lane wrote: AFAIK the Windows port is simply wrong/insecure on this point --- it's one of the reasons you'll never see me recommending Windows as

Re: [PATCHES] Feature: POSIX Shared memory support

2007-02-08 Thread Tom Lane
Magnus Hagander [EMAIL PROTECTED] writes: If we just didn't add the serial number at the end, then it would be impossible to create a shared memory segment for the same port again. That protects the port and not the datadir. But what if we change the name of the shared memory segment to be

Re: [PATCHES] Feature: POSIX Shared memory support

2007-02-08 Thread Magnus Hagander
Tom Lane wrote: Magnus Hagander [EMAIL PROTECTED] writes: If we just didn't add the serial number at the end, then it would be impossible to create a shared memory segment for the same port again. That protects the port and not the datadir. But what if we change the name of the shared memory

Re: [PATCHES] Feature: POSIX Shared memory support

2007-02-08 Thread Tom Lane
Magnus Hagander [EMAIL PROTECTED] writes: Tom Lane wrote: Magnus Hagander [EMAIL PROTECTED] writes: If we just didn't add the serial number at the end, then it would be impossible to create a shared memory segment for the same port again. That protects the port and not the datadir. But what

Re: [PATCHES] Feature: POSIX Shared memory support

2007-02-08 Thread Magnus Hagander
Tom Lane wrote: Magnus Hagander [EMAIL PROTECTED] writes: Tom Lane wrote: Magnus Hagander [EMAIL PROTECTED] writes: If we just didn't add the serial number at the end, then it would be impossible to create a shared memory segment for the same port again. That protects the port and not the

Re: [PATCHES] Feature: POSIX Shared memory support

2007-02-07 Thread Magnus Hagander
On Tue, Feb 06, 2007 at 11:08:51PM -0500, Tom Lane wrote: Takayuki Tsunakawa [EMAIL PROTECTED] writes: From: Tom Lane [EMAIL PROTECTED] the POSIX API provides no way to detect whether anyone else is attached to the segment. Not being able to tell that is a tremendous robustness hit for

Re: [PATCHES] Feature: POSIX Shared memory support

2007-02-07 Thread Tom Lane
Magnus Hagander [EMAIL PROTECTED] writes: On Tue, Feb 06, 2007 at 11:08:51PM -0500, Tom Lane wrote: AFAIK the Windows port is simply wrong/insecure on this point --- it's one of the reasons you'll never see me recommending Windows as the OS for a production Postgres server. What exactly is

Re: [PATCHES] Feature: POSIX Shared memory support

2007-02-07 Thread Alvaro Herrera
Andrew Dunstan wrote: Maybe we should look some more at that. Use of file locking was one thought I had today after I saw Tom's earlier comments. Perl provides a moderately portable flock(), which we use in fact in buildfarm to stop it from running more than one at a time on a given repo

Re: [PATCHES] Feature: POSIX Shared memory support

2007-02-07 Thread Andrew Dunstan
Alvaro Herrera wrote: Andrew Dunstan wrote: Maybe we should look some more at that. Use of file locking was one thought I had today after I saw Tom's earlier comments. Perl provides a moderately portable flock(), which we use in fact in buildfarm to stop it from running more than one at a

Re: [PATCHES] Feature: POSIX Shared memory support

2007-02-07 Thread Alvaro Herrera
Andrew Dunstan wrote: Perl provides a moderately portable flock(), which we use in fact in buildfarm to stop it from running more than one at a time on a given repo copy. [...] Maybe we can borrow some code. Probably not, because it's GPL/Artistic; but we could borrow some ideas instead.

Re: [PATCHES] Feature: POSIX Shared memory support

2007-02-06 Thread Tom Lane
Chris Marcellino [EMAIL PROTECTED] writes: To this end, I have ported the svsv_shmem.c layer to use the POSIX calls (which are some ways more robust w.r.t reducing collision by using strings as shared memory id's, instead of ints). This has been suggested before, and rejected before, on

Re: [PATCHES] Feature: POSIX Shared memory support

2007-02-06 Thread Michael Paesold
Tom Lane wrote: Chris Marcellino [EMAIL PROTECTED] writes: To this end, I have ported the svsv_shmem.c layer to use the POSIX calls (which are some ways more robust w.r.t reducing collision by using strings as shared memory id's, instead of ints). This has been suggested before, and

Re: [PATCHES] Feature: POSIX Shared memory support

2007-02-06 Thread Chris Marcellino
Tom, that is a definitely valid point and thanks for the feedback. I assume that the 'more modern' string segment naming gave the POSIX methods an edge in avoiding collision between other apps. As far as detecting a) whether anyone else is currently attached to that segment and b) whether an

Re: [PATCHES] Feature: POSIX Shared memory support

2007-02-06 Thread Alvaro Herrera
Chris Marcellino wrote: Tom, that is a definitely valid point and thanks for the feedback. I assume that the 'more modern' string segment naming gave the POSIX methods an edge in avoiding collision between other apps. As far as detecting a) whether anyone else is currently attached to

Re: [PATCHES] Feature: POSIX Shared memory support

2007-02-06 Thread Chris Marcellino
To my knowledge there is unfortunately not a portable call that does that. I was actually referring to the check that the current SysV code does on the pid that is stored in the shmem header. I presume that if the backend is dead, the kill(hdr-creatorPID, 0) returning zero would suffice

Re: [PATCHES] Feature: POSIX Shared memory support

2007-02-06 Thread Tom Lane
Chris Marcellino [EMAIL PROTECTED] writes: I was actually referring to the check that the current SysV code does on the pid that is stored in the shmem header. I presume that if the backend is dead, the kill(hdr-creatorPID, 0) returning zero would suffice for confirming the existence of

Re: [PATCHES] Feature: POSIX Shared memory support

2007-02-06 Thread Takayuki Tsunakawa
From: Chris Marcellino [EMAIL PROTECTED] To this end, I have ported the svsv_shmem.c layer to use the POSIX calls (which are some ways more robust w.r.t reducing collision by using strings as shared memory id's, instead of ints). I hope your work will be accepted. Setting IPC parameters is

Re: [PATCHES] Feature: POSIX Shared memory support

2007-02-06 Thread Chris Marcellino
Responses inline. On Feb 6, 2007, at 7:05 PM, Takayuki Tsunakawa wrote: From: Chris Marcellino [EMAIL PROTECTED] To this end, I have ported the svsv_shmem.c layer to use the POSIX calls (which are some ways more robust w.r.t reducing collision by using strings as shared memory id's, instead

Re: [PATCHES] Feature: POSIX Shared memory support

2007-02-06 Thread Tom Lane
Takayuki Tsunakawa [EMAIL PROTECTED] writes: From: Tom Lane [EMAIL PROTECTED] the POSIX API provides no way to detect whether anyone else is attached to the segment. Not being able to tell that is a tremendous robustness hit for us. How is this done on Windows? Is it possible to count the

Re: [PATCHES] Feature: POSIX Shared memory support

2007-02-06 Thread Tom Lane
Chris Marcellino [EMAIL PROTECTED] writes: As Tom pointed out, the code I posted yesterday is not robust enough for general consumption. I'm working on a better solution, which will likely involve using a very small SysV shmem segment as a mutex of sorts (as Michael Paesold suggested).

Re: [PATCHES] Feature: POSIX Shared memory support

2007-02-06 Thread Takayuki Tsunakawa
ep ---(end of broadcast)--- TIP 1: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly

Re: [PATCHES] Feature: POSIX Shared memory support

2007-02-06 Thread Takayuki Tsunakawa
Then, how about semaphores? When I just do configure, PostgreSQL seems to use SysV semaphores. But POSIX semaphore implementation is prepared in src/backend/port/posix_sema.c. Why isn't it used by default? Does it have any problem? Either way, Essentially, no one is running out of

Re: [PATCHES] Feature: POSIX Shared memory support

2007-02-06 Thread Chris Marcellino
Yes, as Tom pointed out. Sorry, I misread the autoconf file. I've gotten quite used to Darwin == BSD. I've added a note to my todo list to look into the posix semaphore performance on the Darwin side. --Chris On Feb 6, 2007, at 8:32 PM, Takayuki Tsunakawa wrote: Then, how about

Re: [PATCHES] Feature: POSIX Shared memory support

2007-02-06 Thread Chris Marcellino
Attached is a beta of the POSIX shared memory layer. It is 75% the original sysv_shmem.c code. I'm looking for ways to refactor it down a bit, while changing as little of the tried-and-tested code as possible. I though I'd put it out there for comments. Of course, unfortunately it is more

Re: [PATCHES] Feature: POSIX Shared memory support

2007-02-06 Thread Andrew Dunstan
Tom Lane wrote: We've speculated on occasion about using file locking in some form as a substitute mechanism for detecting this, but that seems to just bring its own set of not-too-portable assumptions. Maybe we should look some more at that. Use of file locking was one thought I had today