Re: [HACKERS] Proposal to add a QNX 6.5 port to PostgreSQL

2014-08-22 Thread Andres Freund
On 2014-08-22 10:41:55 -0400, Alvaro Herrera wrote: > Andres Freund wrote: > > Hi, > > > > On 2014-08-21 15:25:44 +, Baker, Keith [OCDUS Non-J&J] wrote: > > > About SA_RESTART: > > > > > > I would like to offer you a different perspective which may alter your > > > cu

Re: [HACKERS] Proposal to add a QNX 6.5 port to PostgreSQL

2014-08-22 Thread
on-J&J]; Robert Haas; Tom Lane; pgsql- > hack...@postgresql.org > Subject: Re: [HACKERS] Proposal to add a QNX 6.5 port to PostgreSQL > > Andres Freund wrote: > > Hi, > > > > On 2014-08-21 15:25:44 +, Baker, Keith [OCDUS Non-J&J] wrote: > > > About SA

Re: [HACKERS] Proposal to add a QNX 6.5 port to PostgreSQL

2014-08-22 Thread Alvaro Herrera
Andres Freund wrote: > Hi, > > On 2014-08-21 15:25:44 +, Baker, Keith [OCDUS Non-J&J] wrote: > > About SA_RESTART: > > > > I would like to offer you a different perspective which may alter your > > current opinion. > > I believe the port.h QNX macro replacement for SA

Re: [HACKERS] Proposal to add a QNX 6.5 port to PostgreSQL

2014-08-22 Thread Andres Freund
Hi, On 2014-08-21 15:25:44 +, Baker, Keith [OCDUS Non-J&J] wrote: > About SA_RESTART: > > I would like to offer you a different perspective which may alter your > current opinion. > I believe the port.h QNX macro replacement for SA_RESTART is still a > reasonable sol

Re: [HACKERS] Proposal to add a QNX 6.5 port to PostgreSQL

2014-08-22 Thread Noah Misch
On Fri, Aug 22, 2014 at 09:34:42AM +0200, Andres Freund wrote: > On 2014-08-22 01:36:37 -0400, Noah Misch wrote: > > On Thu, Aug 21, 2014 at 01:33:38AM +0200, Andres Freund wrote: > > > On 2014-07-25 18:29:53 -0400, Tom Lane wrote: > > > > > * QNX lacks sigaction SA_RESTART: I modified > >

Re: [HACKERS] Proposal to add a QNX 6.5 port to PostgreSQL

2014-08-22 Thread Andres Freund
On 2014-08-22 01:36:37 -0400, Noah Misch wrote: > On Thu, Aug 21, 2014 at 01:33:38AM +0200, Andres Freund wrote: > > On 2014-07-25 18:29:53 -0400, Tom Lane wrote: > > > > * QNX lacks sigaction SA_RESTART: I modified > > > > "src/include/port.h" to define macros to retry system calls upon E

Re: [HACKERS] Proposal to add a QNX 6.5 port to PostgreSQL

2014-08-21 Thread Noah Misch
On Thu, Aug 21, 2014 at 01:33:38AM +0200, Andres Freund wrote: > On 2014-07-25 18:29:53 -0400, Tom Lane wrote: > > > * QNX lacks sigaction SA_RESTART: I modified "src/include/port.h" > > > to define macros to retry system calls upon EINTR (open,read,write,...) > > > when compiled on QNX >

Re: [HACKERS] Proposal to add a QNX 6.5 port to PostgreSQL

2014-08-21 Thread Alvaro Herrera
Baker, Keith [OCDUS Non-J&J] wrote: > About configure: > > "./configure" barked at 2 things on QNX, and it advised using both > "--without-readline --disable-thread-safety". > I can investigate further, but I have been focusing on the bigger issues > first. I don't think th

Re: [HACKERS] Proposal to add a QNX 6.5 port to PostgreSQL

2014-08-21 Thread
sday, August 20, 2014 7:25 PM > To: Baker, Keith [OCDUS Non-J&J] > Cc: Alvaro Herrera; Robert Haas; Tom Lane; pgsql-hackers@postgresql.org > Subject: Re: [HACKERS] Proposal to add a QNX 6.5 port to PostgreSQL > > Hi, > > On 2014-08-20 21:21:41 +, Baker, Keith [OCDUS No

Re: [HACKERS] Proposal to add a QNX 6.5 port to PostgreSQL

2014-08-20 Thread Andres Freund
On 2014-07-25 18:29:53 -0400, Tom Lane wrote: > > * QNX lacks sigaction SA_RESTART: I modified "src/include/port.h" > > to define macros to retry system calls upon EINTR (open,read,write,...) > > when compiled on QNX > > That's pretty scary too. For one thing, such macros would affect e

Re: [HACKERS] Proposal to add a QNX 6.5 port to PostgreSQL

2014-08-20 Thread Andres Freund
Hi, On 2014-08-20 21:21:41 +, Baker, Keith [OCDUS Non-J&J] wrote: > To work around lack of SA_RESTART, I added QNX-specific retry macros to port.h > With these macros in place "make check" runs cleanly (fails in many place > without them). > > +#if defined(__QNX__) > +/* QNX does not

Re: [HACKERS] Proposal to add a QNX 6.5 port to PostgreSQL

2014-08-20 Thread
pect I can solve that. Keith Baker > -Original Message- > From: Alvaro Herrera [mailto:alvhe...@2ndquadrant.com] > Sent: Wednesday, August 20, 2014 4:16 PM > To: Baker, Keith [OCDUS Non-J&J] > Cc: Robert Haas; Tom Lane; pgsql-hackers@postgresql.org > Subject:

Re: [HACKERS] Proposal to add a QNX 6.5 port to PostgreSQL

2014-08-20 Thread Alvaro Herrera
Baker, Keith [OCDUS Non-J&J] wrote: > Please let me know if more discussion is required, or if it would be > reasonable for me (or someone else of your choosing) to work on the > coding effort (perhaps targeted for 9.5?) > If on the other hand it has been decided that a QNX port is not in the > ca

Re: [HACKERS] Proposal to add a QNX 6.5 port to PostgreSQL

2014-08-20 Thread
Keith Baker > -Original Message- > From: Robert Haas [mailto:robertmh...@gmail.com] > Sent: Wednesday, August 20, 2014 12:26 PM > To: Baker, Keith [OCDUS Non-J&J] > Cc: Tom Lane; pgsql-hackers@postgresql.org > Subject: Re: [HACKERS] Proposal to add a QNX 6.5 port to

Re: [HACKERS] Proposal to add a QNX 6.5 port to PostgreSQL

2014-08-20 Thread Robert Haas
On Mon, Aug 18, 2014 at 11:02 AM, Baker, Keith [OCDUS Non-J&J] wrote: > My proof of concept code (steps a though e below) avoided any reading or > writing to the pipe (and associated handling of SIGPIPE), it just relied on > postmaster open of PIPE with ENXIO to indicate all is clear. I'm not f

Re: [HACKERS] Proposal to add a QNX 6.5 port to PostgreSQL

2014-08-18 Thread
Lane > Cc: Baker, Keith [OCDUS Non-J&J]; pgsql-hackers@postgresql.org > Subject: Re: [HACKERS] Proposal to add a QNX 6.5 port to PostgreSQL > > On Fri, Aug 15, 2014 at 12:02 PM, Tom Lane wrote: > > * I think 5..8 are overly complex: we can just set SIGPIPE to SIG_IGN >

Re: [HACKERS] Proposal to add a QNX 6.5 port to PostgreSQL

2014-08-18 Thread Noah Misch
On Mon, Aug 18, 2014 at 09:01:20AM -0400, Robert Haas wrote: > On Sat, Aug 16, 2014 at 3:28 AM, Noah Misch wrote: > >> I'd be afraid that a secondary mechanism that mostly-but-not-really > >> works could do more harm by allowing us to miss bugs in the primary, > >> pipe-based locking mechanism tha

Re: [HACKERS] Proposal to add a QNX 6.5 port to PostgreSQL

2014-08-18 Thread Robert Haas
On Sat, Aug 16, 2014 at 3:28 AM, Noah Misch wrote: > Nice algorithm. Thanks. >> I'd be afraid that a secondary mechanism that mostly-but-not-really >> works could do more harm by allowing us to miss bugs in the primary, >> pipe-based locking mechanism than the good it would accomplish. > > Users

Re: [HACKERS] Proposal to add a QNX 6.5 port to PostgreSQL

2014-08-16 Thread Noah Misch
Nice algorithm. On Fri, Aug 15, 2014 at 02:16:08PM -0400, Robert Haas wrote: > On Fri, Aug 15, 2014 at 12:02 PM, Tom Lane wrote: > > * We talked about combining this technique with a plain file lock > > so that we would have belt-and-suspenders protection, in particular > > something that would h

Re: [HACKERS] Proposal to add a QNX 6.5 port to PostgreSQL

2014-08-15 Thread Robert Haas
On Fri, Aug 15, 2014 at 12:02 PM, Tom Lane wrote: > * I think 5..8 are overly complex: we can just set SIGPIPE to SIG_IGN > (which is its usual setting in the postmaster already) and check for > EPIPE from the write(). wfm. > * There might be some benefit to swapping steps 9 and 10; at the > ver

Re: [HACKERS] Proposal to add a QNX 6.5 port to PostgreSQL

2014-08-15 Thread Tom Lane
Robert Haas writes: > How about this locking protocol: > Postmaster: > 1. Acquire an exclusive lock on some file in the data directory, maybe > the control file, using fcntl(). > 2. Open the named pipe for read. > 3. Open the named pipe for write. > 4. Close the named pipe for read. > 5. Install

Re: [HACKERS] Proposal to add a QNX 6.5 port to PostgreSQL

2014-08-15 Thread Robert Haas
On Thu, Aug 14, 2014 at 12:08 PM, Baker, Keith [OCDUS Non-J&J] wrote: > I tried a combination of PIPE lock and file lock (fcntl) as Tom had suggested. > Attached experimental patch has this logic... > > Postmaster : > - get exclusive fcntl lock (to guard against race condition in PIPE-based > loc

Re: [HACKERS] Proposal to add a QNX 6.5 port to PostgreSQL

2014-08-14 Thread
014 11:02 AM > To: Robert Haas > Cc: Baker, Keith [OCDUS Non-J&J]; pgsql-hackers@postgresql.org > Subject: Re: [HACKERS] Proposal to add a QNX 6.5 port to PostgreSQL > > Robert Haas writes: > > On Tue, Jul 29, 2014 at 7:06 PM, Tom Lane wrote: > >> Hm. That

Re: [HACKERS] Proposal to add a QNX 6.5 port to PostgreSQL

2014-08-13 Thread
gain for your feedback, I really do appreciate it. -Keith Baker > -Original Message- > From: Tom Lane [mailto:t...@sss.pgh.pa.us] > Sent: Wednesday, August 13, 2014 7:05 PM > To: Baker, Keith [OCDUS Non-J&J] > Cc: Robert Haas; pgsql-hackers@postgresql.org > Subject:

Re: [HACKERS] Proposal to add a QNX 6.5 port to PostgreSQL

2014-08-13 Thread Tom Lane
"Baker, Keith [OCDUS Non-J&J]" writes: > I assume you guys are working on other priorities, so I did some locking > experiments on QNX. > I know fcntl() locking has downsides, but I think it deserves a second look: > - it is POSIX, so should be fairly consistent across platforms (at least more

Re: [HACKERS] Proposal to add a QNX 6.5 port to PostgreSQL

2014-08-13 Thread
: Baker, Keith [OCDUS Non-J&J]; pgsql-hackers@postgresql.org > Subject: Re: [HACKERS] Proposal to add a QNX 6.5 port to PostgreSQL > > On Wed, Jul 30, 2014 at 11:02 AM, Tom Lane wrote: > > So it seems like we could possibly go this route, assuming we can > > think of a variant of

Re: [HACKERS] Proposal to add a QNX 6.5 port to PostgreSQL

2014-08-11 Thread Robert Haas
On Sat, Aug 9, 2014 at 2:00 PM, Tom Lane wrote: >> +1. I think the current behaviour is a seriously bad idea. > > I don't think it's anywhere near as black-and-white as you guys claim. > What it comes down to is whether allowing existing transactions/sessions > to finish is more important than all

Re: [HACKERS] Proposal to add a QNX 6.5 port to PostgreSQL

2014-08-11 Thread Andres Freund
On 2014-08-10 18:36:18 -0400, Noah Misch wrote: > [Due for a new subject line?] > > On Sat, Aug 09, 2014 at 08:16:01PM +0200, Andres Freund wrote: > > On 2014-08-09 14:09:36 -0400, Tom Lane wrote: > > > Andres Freund writes: > > > > On 2014-08-09 14:00:49 -0400, Tom Lane wrote: > > > >> I don't t

Re: [HACKERS] Proposal to add a QNX 6.5 port to PostgreSQL

2014-08-11 Thread Kevin Grittner
Stephen Frost wrote: >> Our grace period for active backends after unclean exit of one >> of their peers is low, milliseconds to seconds.  Our grace >> period for active backends after unclean exit of the postmaster >> is unconstrained.  At least one of those policies has to be >> wrong. Like And

Re: [HACKERS] Proposal to add a QNX 6.5 port to PostgreSQL

2014-08-10 Thread Stephen Frost
* Noah Misch (n...@leadboat.com) wrote: > [Due for a new subject line?] Probably. > Our grace period for active backends after unclean exit of one of their peers > is low, milliseconds to seconds. Our grace period for active backends after > unclean exit of the postmaster is unconstrained. At l

Re: [HACKERS] Proposal to add a QNX 6.5 port to PostgreSQL

2014-08-10 Thread Noah Misch
[Due for a new subject line?] On Sat, Aug 09, 2014 at 08:16:01PM +0200, Andres Freund wrote: > On 2014-08-09 14:09:36 -0400, Tom Lane wrote: > > Andres Freund writes: > > > On 2014-08-09 14:00:49 -0400, Tom Lane wrote: > > >> I don't think it's anywhere near as black-and-white as you guys claim.

Re: [HACKERS] Proposal to add a QNX 6.5 port to PostgreSQL

2014-08-09 Thread Andres Freund
On 2014-08-09 14:09:36 -0400, Tom Lane wrote: > Andres Freund writes: > > On 2014-08-09 14:00:49 -0400, Tom Lane wrote: > >> I don't think it's anywhere near as black-and-white as you guys claim. > >> What it comes down to is whether allowing existing transactions/sessions > >> to finish is more i

Re: [HACKERS] Proposal to add a QNX 6.5 port to PostgreSQL

2014-08-09 Thread Tom Lane
Andres Freund writes: > On 2014-08-09 14:00:49 -0400, Tom Lane wrote: >> I don't think it's anywhere near as black-and-white as you guys claim. >> What it comes down to is whether allowing existing transactions/sessions >> to finish is more important than allowing new sessions to start. >> Dependi

Re: [HACKERS] Proposal to add a QNX 6.5 port to PostgreSQL

2014-08-09 Thread Tom Lane
Andres Freund writes: > On 2014-08-04 10:54:25 -0400, Robert Haas wrote: >> I believe that multiple people have said multiple times that we should >> change the behavior so that orphaned backends exit immediately; I >> think you are the only one defending the current behavior. There are >> severa

Re: [HACKERS] Proposal to add a QNX 6.5 port to PostgreSQL

2014-08-09 Thread Andres Freund
On 2014-08-09 14:00:49 -0400, Tom Lane wrote: > Andres Freund writes: > > On 2014-08-04 10:54:25 -0400, Robert Haas wrote: > >> I believe that multiple people have said multiple times that we should > >> change the behavior so that orphaned backends exit immediately; I > >> think you are the only

Re: [HACKERS] Proposal to add a QNX 6.5 port to PostgreSQL

2014-08-09 Thread Andres Freund
On 2014-08-04 10:54:25 -0400, Robert Haas wrote: > On Thu, Jul 31, 2014 at 9:51 PM, Tom Lane wrote: > > Even without that issue, there's no consensus that forcibly making > > orphan backends exit would be a good thing. (Some people would > > like to have such an option, but the key word in that s

Re: [HACKERS] Proposal to add a QNX 6.5 port to PostgreSQL

2014-08-04 Thread Josh Berkus
On 08/04/2014 07:54 AM, Robert Haas wrote: > 1. Most seriously, once the postmaster is gone, there's nobody to > SIGQUIT remaining backends if somebody exits uncleanly. This means > that a backend running without a postmaster could be running in a > corrupt shared memory segment, which could lead

Re: [HACKERS] Proposal to add a QNX 6.5 port to PostgreSQL

2014-08-04 Thread Robert Haas
On Thu, Jul 31, 2014 at 9:51 PM, Tom Lane wrote: > "Baker, Keith [OCDUS Non-J&J]" writes: >> Since ensuring there are not orphaned back-end processes is vital, could we >> add a check for getppid() == 1 ? > > No. Or yeah, we could, but that patch would add no security worth > mentioning. For e

Re: [HACKERS] Proposal to add a QNX 6.5 port to PostgreSQL

2014-07-31 Thread Tom Lane
"Baker, Keith [OCDUS Non-J&J]" writes: > Since ensuring there are not orphaned back-end processes is vital, could we > add a check for getppid() == 1 ? No. Or yeah, we could, but that patch would add no security worth mentioning. For example, someone could launch a query that runs for many min

Re: [HACKERS] Proposal to add a QNX 6.5 port to PostgreSQL

2014-07-31 Thread
: Baker, Keith [OCDUS Non-J&J]; pgsql-hackers@postgresql.org > Subject: Re: [HACKERS] Proposal to add a QNX 6.5 port to PostgreSQL > > On Wed, Jul 30, 2014 at 11:02 AM, Tom Lane wrote: > > So it seems like we could possibly go this route, assuming we can > > think of a v

Re: [HACKERS] Proposal to add a QNX 6.5 port to PostgreSQL

2014-07-31 Thread Robert Haas
On Wed, Jul 30, 2014 at 11:02 AM, Tom Lane wrote: > So it seems like we could possibly go this route, assuming we can think > of a variant of your proposal that's race-condition-free. A disadvantage > compared to a true file lock is that it would not protect against people > trying to start postm

Re: [HACKERS] Proposal to add a QNX 6.5 port to PostgreSQL

2014-07-30 Thread Tom Lane
"Baker, Keith [OCDUS Non-J&J]" writes: > Please let me know if either of you are ready to experiment with the "named > pipe" idea anytime soon. > If not, I would be happy to take a crack at it, but would appreciate your > expert advice to start me down the right path (files/functions to update,

Re: [HACKERS] Proposal to add a QNX 6.5 port to PostgreSQL

2014-07-30 Thread
esql.org > Subject: Re: [HACKERS] Proposal to add a QNX 6.5 port to PostgreSQL > > Robert Haas writes: > > On Tue, Jul 29, 2014 at 7:06 PM, Tom Lane wrote: > >> Hm. That particular protocol is broken: two postmasters doing it at > >> the same time would both pass (be

Re: [HACKERS] Proposal to add a QNX 6.5 port to PostgreSQL

2014-07-30 Thread Tom Lane
Robert Haas writes: > On Tue, Jul 29, 2014 at 7:06 PM, Tom Lane wrote: >> Hm. That particular protocol is broken: two postmasters doing it at the >> same time would both pass (because neither has it open for read at the >> instant where they try to write). But we could possibly frob the idea >>

Re: [HACKERS] Proposal to add a QNX 6.5 port to PostgreSQL

2014-07-30 Thread Robert Haas
On Tue, Jul 29, 2014 at 7:06 PM, Tom Lane wrote: >> I think it would be good to spend some energy figuring out what to do >> about this. > > Well, we've been around on this multiple times before, but if we have > any new ideas, sure ... Well, I tried to compile a more comprehensive list of possib

Re: [HACKERS] Proposal to add a QNX 6.5 port to PostgreSQL

2014-07-29 Thread Tom Lane
"Baker, Keith [OCDUS Non-J&J]" writes: > If there are existing tests I can run to ensure the QNX port meets your > criteria for robust failure handling in this area I would be happy to run > them. > If not, perhaps someone can provide a quick list of failure modes to consider. > As-is: > - start

Re: [HACKERS] Proposal to add a QNX 6.5 port to PostgreSQL

2014-07-29 Thread
as Cc: Baker, Keith [OCDUS Non-J&J]; pgsql-hackers@postgresql.org Subject: Re: [HACKERS] Proposal to add a QNX 6.5 port to PostgreSQL Robert Haas writes: > On Fri, Jul 25, 2014 at 6:29 PM, Tom Lane wrote: >> This isn't really acceptable for production usage; if it were, we'

Re: [HACKERS] Proposal to add a QNX 6.5 port to PostgreSQL

2014-07-29 Thread Tom Lane
Robert Haas writes: > On Fri, Jul 25, 2014 at 6:29 PM, Tom Lane wrote: >> This isn't really acceptable for production usage; if it were, we'd have >> done it already. The POSIX APIs lack any way to tell how many processes >> are attached to a shmem segment, which is *necessary* functionality for

Re: [HACKERS] Proposal to add a QNX 6.5 port to PostgreSQL

2014-07-28 Thread Peter Geoghegan
On Mon, Jul 28, 2014 at 9:41 AM, Merlin Moncure wrote: >> I don't think a buildfarm animal that doesn't run the actual upstream >> code is a good idea. That'll make it a lot harder to understand what's >> going on when something breaks after a commit. It'd also require the >> custom patches being

Re: [HACKERS] Proposal to add a QNX 6.5 port to PostgreSQL

2014-07-28 Thread Merlin Moncure
On Mon, Jul 28, 2014 at 11:22 AM, Andres Freund wrote: > On 2014-07-28 11:19:48 -0500, Merlin Moncure wrote: >> Maybe step #1 is to get a buildfarm member set up. Is there any >> policy against unsupported environments in the buildfarm? (I hope not) >> >> You're going to have to run it against a

Re: [HACKERS] Proposal to add a QNX 6.5 port to PostgreSQL

2014-07-28 Thread Andres Freund
On 2014-07-28 11:19:48 -0500, Merlin Moncure wrote: > Maybe step #1 is to get a buildfarm member set up. Is there any > policy against unsupported environments in the buildfarm? (I hope not) > > You're going to have to run it against a git repository containing > your custom patches. It's a long

Re: [HACKERS] Proposal to add a QNX 6.5 port to PostgreSQL

2014-07-28 Thread Merlin Moncure
On Fri, Jul 25, 2014 at 3:16 PM, Baker, Keith [OCDUS Non-J&J] wrote: > I propose that a QNX 6.5 port be introduced to PostgreSQL. > > I am new to PostgreSQL development, so please bear with me. > > > > I have made good progress (with 1 outstanding issue, details below): > > · I created a Q

Re: [HACKERS] Proposal to add a QNX 6.5 port to PostgreSQL

2014-07-25 Thread Tom Lane
"Baker, Keith [OCDUS Non-J&J]" writes: > I propose that a QNX 6.5 port be introduced to PostgreSQL. Hmm ... you're aware that there used to be a QNX port? We removed it back in 2006 for lack of interest and maintainers, and AFAIR you're the first person to show any interest in reintroducing it s

[HACKERS] Proposal to add a QNX 6.5 port to PostgreSQL

2014-07-25 Thread
I propose that a QNX 6.5 port be introduced to PostgreSQL. I am new to PostgreSQL development, so please bear with me. I have made good progress (with 1 outstanding issue, details below): * I created a QNX 6.5 port of PostgreSQL 9.3.4 which passes regression tests. * I merged m