Re: [HACKERS] Proof of concept: standalone backend with full FE/BE protocol

2012-09-05 Thread Bruce Momjian
On Wed, Sep 5, 2012 at 01:50:06PM -0700, Josh Berkus wrote: > Tom, > > > However, there are some additional things > > we'd need to think about before advertising it as a fit solution for that. > > Notably, while the lack of any background processes is just what you want > > for pg_upgrade and di

Re: [HACKERS] Proof of concept: standalone backend with full FE/BE protocol

2012-09-05 Thread Tom Lane
Josh Berkus writes: >> However, there are some additional things >> we'd need to think about before advertising it as a fit solution for that. >> Notably, while the lack of any background processes is just what you want >> for pg_upgrade and disaster recovery, an ordinary application is probably >

Re: [HACKERS] Proof of concept: standalone backend with full FE/BE protocol

2012-09-05 Thread Josh Berkus
Tom, > However, there are some additional things > we'd need to think about before advertising it as a fit solution for that. > Notably, while the lack of any background processes is just what you want > for pg_upgrade and disaster recovery, an ordinary application is probably > going to want to r

Re: [HACKERS] Proof of concept: standalone backend with full FE/BE protocol

2012-09-05 Thread anara...@anarazel.de
Tom Lane schrieb: >Andres Freund writes: >> I don't find that a convincing comparison. Normally don't need to >shutdown the >> server between two pg_dump commands. Which very well might be >scripted. > >> Especially as for now, without a background writer/checkpointer >writing stuff >> befor

Re: [HACKERS] Proof of concept: standalone backend with full FE/BE protocol

2012-09-05 Thread Tom Lane
"anara...@anarazel.de" writes: > I am not saying its bad that it is slower, that's absolutely OK. Just that it > will take a variable amount of time till you can run pgdump again and its not > easily detectable without looping and trying again. Well, that's why the proposed libpq code is writte

Re: [HACKERS] Proof of concept: standalone backend with full FE/BE protocol

2012-09-05 Thread Tom Lane
Andres Freund writes: > I don't find that a convincing comparison. Normally don't need to shutdown > the > server between two pg_dump commands. Which very well might be scripted. > Especially as for now, without a background writer/checkpointer writing stuff > beforehand, the shutdown checkpoi

Re: [HACKERS] Proof of concept: standalone backend with full FE/BE protocol

2012-09-05 Thread Andres Freund
On Tuesday, September 04, 2012 12:11:28 PM Amit Kapila wrote: > On Tuesday, September 04, 2012 11:00 AM Andres Freund wrote: > > On Tuesday, September 04, 2012 06:20:59 AM Tom Lane wrote: > > Andres Freund writes: > >> > I can see why that would be nice, but is it really realistic? Don't we > >>

Re: [HACKERS] Proof of concept: standalone backend with full FE/BE protocol

2012-09-05 Thread Amit Kapila
On Wednesday, September 05, 2012 3:58 PM Magnus Hagander wrote: On Wed, Sep 5, 2012 at 7:31 AM, Amit Kapila wrote: > On Tuesday, September 04, 2012 12:40 AM Tom Lane wrote: > Magnus Hagander writes: >> On Mon, Sep 3, 2012 at 8:51 PM, Tom Lane wrote: > I have another question after thinking a

Re: [HACKERS] Proof of concept: standalone backend with full FE/BE protocol

2012-09-05 Thread Magnus Hagander
On Wed, Sep 5, 2012 at 7:31 AM, Amit Kapila wrote: > On Tuesday, September 04, 2012 12:40 AM Tom Lane wrote: > Magnus Hagander writes: >> On Mon, Sep 3, 2012 at 8:51 PM, Tom Lane wrote: I have another question after thinking about that for awhile: is there any security concern there?

Re: [HACKERS] Proof of concept: standalone backend with full FE/BE protocol

2012-09-04 Thread Amit Kapila
On Tuesday, September 04, 2012 12:40 AM Tom Lane wrote: Magnus Hagander writes: > On Mon, Sep 3, 2012 at 8:51 PM, Tom Lane wrote: >>> I have another question after thinking about that for awhile: is there >>> any security concern there? On Unix-oid systems, we expect the kernel >>> to restrict w

Re: [HACKERS] Proof of concept: standalone backend with full FE/BE protocol

2012-09-04 Thread Amit Kapila
On Tuesday, September 04, 2012 11:00 AM Andres Freund wrote: On Tuesday, September 04, 2012 06:20:59 AM Tom Lane wrote: > Andres Freund writes: >> > I can see why that would be nice, but is it really realistic? Don't we >> > expect some more diligence in applications using this against letting >>

Re: [HACKERS] Proof of concept: standalone backend with full FE/BE protocol

2012-09-03 Thread Andres Freund
On Tuesday, September 04, 2012 06:20:59 AM Tom Lane wrote: > Andres Freund writes: > > I can see why that would be nice, but is it really realistic? Don't we > > expect some more diligence in applications using this against letting > > such a child continue to run after ctrl-c/SIGTERMing e.g. pg_d

Re: [HACKERS] Proof of concept: standalone backend with full FE/BE protocol

2012-09-03 Thread Noah Misch
On Tue, Sep 04, 2012 at 12:01:17AM -0400, Robert Haas wrote: > Another idea here might be to stick with Tom's > original idea of making it a connection parameter, but have it be > turned off by default. In other words, if an application wants to > allow those parameters to be used, it would need t

Re: [HACKERS] Proof of concept: standalone backend with full FE/BE protocol

2012-09-03 Thread Tom Lane
Andres Freund writes: > I can see why that would be nice, but is it really realistic? Don't we > expect some more diligence in applications using this against letting > such a child continue to run after ctrl-c/SIGTERMing e.g. pg_dump in > comparison to closing a normal database connection? Er, w

Re: [HACKERS] Proof of concept: standalone backend with full FE/BE protocol

2012-09-03 Thread Tom Lane
Robert Haas writes: > I tend to agree. Another idea here might be to stick with Tom's > original idea of making it a connection parameter, but have it be > turned off by default. In other words, if an application wants to > allow those parameters to be used, it would need to do > PQenableStartSe

Re: [HACKERS] Proof of concept: standalone backend with full FE/BE protocol

2012-09-03 Thread Robert Haas
On Mon, Sep 3, 2012 at 4:38 PM, Andres Freund wrote: > On Monday, September 03, 2012 10:23:52 PM Tom Lane wrote: >> I'm reluctantly coming to the conclusion that we can't pass these >> parameters through the regular libpq connection string mechanism, and >> will have to invent something else. Tha

Re: [HACKERS] Proof of concept: standalone backend with full FE/BE protocol

2012-09-03 Thread Amit Kapila
On Monday, September 03, 2012 10:37 PM Tom Lane wrote: Amit Kapila writes: >> I think part of the code for windows can be written by referring function >> internal_forkexec(), >> If you are okay, I can take up this. Please confirm. > Nobody else volunteered, so have at it. Note that I'm plannin

Re: [HACKERS] Proof of concept: standalone backend with full FE/BE protocol

2012-09-03 Thread Andres Freund
On Monday, September 03, 2012 10:54:23 PM Tom Lane wrote: > Andres Freund writes: > > On Monday, September 03, 2012 10:23:52 PM Tom Lane wrote: > >> I'm reluctantly coming to the conclusion that we can't pass these > >> parameters through the regular libpq connection string mechanism, and > >> wil

Re: [HACKERS] Proof of concept: standalone backend with full FE/BE protocol

2012-09-03 Thread Tom Lane
Andres Freund writes: > On Monday, September 03, 2012 10:23:52 PM Tom Lane wrote: >> I'm reluctantly coming to the conclusion that we can't pass these >> parameters through the regular libpq connection string mechanism, and >> will have to invent something else. That's awfully nasty though; >> it

Re: [HACKERS] Proof of concept: standalone backend with full FE/BE protocol

2012-09-03 Thread Tom Lane
Andrew Dunstan writes: > On 09/03/2012 04:23 PM, Tom Lane wrote: >> I'm reluctantly coming to the conclusion that we can't pass these >> parameters through the regular libpq connection string mechanism, and >> will have to invent something else. That's awfully nasty though; >> it will pretty much

Re: [HACKERS] Proof of concept: standalone backend with full FE/BE protocol

2012-09-03 Thread Andres Freund
Hi, On Monday, September 03, 2012 10:23:52 PM Tom Lane wrote: > I'm reluctantly coming to the conclusion that we can't pass these > parameters through the regular libpq connection string mechanism, and > will have to invent something else. That's awfully nasty though; > it will pretty much crippl

Re: [HACKERS] Proof of concept: standalone backend with full FE/BE protocol

2012-09-03 Thread Andrew Dunstan
On 09/03/2012 04:23 PM, Tom Lane wrote: I'm reluctantly coming to the conclusion that we can't pass these parameters through the regular libpq connection string mechanism, and will have to invent something else. That's awfully nasty though; it will pretty much cripple the idea that this would b

Re: [HACKERS] Proof of concept: standalone backend with full FE/BE protocol

2012-09-03 Thread Tom Lane
Noah Misch writes: > On Mon, Sep 03, 2012 at 12:11:20AM -0400, Tom Lane wrote: >> One easy thing we could do that would at least narrow the risks is to >> only allow the executable's *directory* to be specified, hardwiring the >> executable file name to "postgres" (or "postgres.exe" I guess). > I

Re: [HACKERS] Proof of concept: standalone backend with full FE/BE protocol

2012-09-03 Thread Tom Lane
Magnus Hagander writes: > On Mon, Sep 3, 2012 at 8:51 PM, Tom Lane wrote: >> I have another question after thinking about that for awhile: is there >> any security concern there? On Unix-oid systems, we expect the kernel >> to restrict who can do a kill() on a postgres process. If there's any >

Re: [HACKERS] Proof of concept: standalone backend with full FE/BE protocol

2012-09-03 Thread Magnus Hagander
On Mon, Sep 3, 2012 at 8:51 PM, Tom Lane wrote: > Magnus Hagander writes: >> On Mon, Sep 3, 2012 at 7:07 PM, Tom Lane wrote: >>> Hmm, after looking at src/port/kill.c it doesn't seem like there's much >>> of a problem with doing that. I had had the idea that our kill >>> emulation only worked w

Re: [HACKERS] Proof of concept: standalone backend with full FE/BE protocol

2012-09-03 Thread Tom Lane
Magnus Hagander writes: > On Mon, Sep 3, 2012 at 7:07 PM, Tom Lane wrote: >> Hmm, after looking at src/port/kill.c it doesn't seem like there's much >> of a problem with doing that. I had had the idea that our kill >> emulation only worked within the backend environment, but of course >> pg_ctl

Re: [HACKERS] Proof of concept: standalone backend with full FE/BE protocol

2012-09-03 Thread Magnus Hagander
On Mon, Sep 3, 2012 at 7:07 PM, Tom Lane wrote: > Amit Kapila writes: >>> 8. PQcancel needs some work - it can't do what it does now, but it could >>> do kill(conn->postgres_pid, SIGINT) instead. At least in Unix. I have no >>> idea what we'd do in Windows. This doesn't matter for pg_upgrade o

Re: [HACKERS] Proof of concept: standalone backend with full FE/BE protocol

2012-09-03 Thread Tom Lane
Amit Kapila writes: > I think part of the code for windows can be written by referring function > internal_forkexec(), > If you are okay, I can take up this. Please confirm. Nobody else volunteered, so have at it. Note that I'm planning to redo that code to use socketpair(), so possibly you wan

Re: [HACKERS] Proof of concept: standalone backend with full FE/BE protocol

2012-09-03 Thread Amit Kapila
> 5. The fork/exec code is pretty primitive with respect to error handling. > I didn't put much time into it since I'm afraid we may need to refactor it entirely before a Windows equivalent can be > written. (And I need somebody to write/test the Windows equivalent - any volunteers?) I think pa

Re: [HACKERS] Proof of concept: standalone backend with full FE/BE protocol

2012-09-03 Thread Magnus Hagander
On Mon, Sep 3, 2012 at 6:42 AM, Tom Lane wrote: > Noah Misch writes: >> Windows does not have socketpair(), nor a strict pipe() equivalent. I expect >> switching to socketpair() makes the Windows side trickier in some ways and >> simpler in others. +1 for exploring that direction first. > > A b

Re: [HACKERS] Proof of concept: standalone backend with full FE/BE protocol

2012-09-02 Thread Noah Misch
On Mon, Sep 03, 2012 at 12:11:20AM -0400, Tom Lane wrote: > Noah Misch writes: > > I don't think it's libpq's job to enforce security policy this way. In any > > event, it has no reason to presume an environment variable is a safer > > source. > > One easy thing we could do that would at least

Re: [HACKERS] Proof of concept: standalone backend with full FE/BE protocol

2012-09-02 Thread Tom Lane
Noah Misch writes: > Windows does not have socketpair(), nor a strict pipe() equivalent. I expect > switching to socketpair() makes the Windows side trickier in some ways and > simpler in others. +1 for exploring that direction first. A bit of googling suggests that emulating socketpair() on Wi

Re: [HACKERS] Proof of concept: standalone backend with full FE/BE protocol

2012-09-02 Thread Tom Lane
Noah Misch writes: > On Sun, Sep 02, 2012 at 11:34:42PM -0400, Tom Lane wrote: >> Heikki Linnakangas writes: >>> Are there security issues with this? If a user can specify libpq >>> connection options, he can now execute any file he wants by passing it >>> as standalone_backend. >> Hmm, that's

Re: [HACKERS] Proof of concept: standalone backend with full FE/BE protocol

2012-09-02 Thread Noah Misch
On Sun, Sep 02, 2012 at 11:34:42PM -0400, Tom Lane wrote: > Heikki Linnakangas writes: > > On 03.09.2012 03:23, Tom Lane wrote: > >> 1. As you can see above, the feature is triggered by specifying the new > >> connection option "standalone_datadir", whose value must be the location > >> of the dat

Re: [HACKERS] Proof of concept: standalone backend with full FE/BE protocol

2012-09-02 Thread Tom Lane
Heikki Linnakangas writes: > On 03.09.2012 03:23, Tom Lane wrote: >> 1. As you can see above, the feature is triggered by specifying the new >> connection option "standalone_datadir", whose value must be the location >> of the data directory. I also invented an option "standalone_backend", >> whi

Re: [HACKERS] Proof of concept: standalone backend with full FE/BE protocol

2012-09-02 Thread Heikki Linnakangas
On 03.09.2012 03:23, Tom Lane wrote: 1. As you can see above, the feature is triggered by specifying the new connection option "standalone_datadir", whose value must be the location of the data directory. I also invented an option "standalone_backend", which can be set to specify which postgres

<    1   2