Re: [HACKERS] Proof of concept: standalone backend with full FE/BE protocol

2013-12-09 Thread Bruce Momjian
On Fri, Dec 6, 2013 at 04:04:36PM +0100, Andres Freund wrote: On 2013-12-05 23:01:28 +0200, Heikki Linnakangas wrote: On 12/05/2013 10:37 PM, Robert Haas wrote: On Thu, Dec 5, 2013 at 3:05 PM, Tom Lane t...@sss.pgh.pa.us wrote: It might be unpleasant to use in some cases, though. Why

Re: [HACKERS] Proof of concept: standalone backend with full FE/BE protocol

2013-12-06 Thread Andres Freund
On 2013-12-05 23:01:28 +0200, Heikki Linnakangas wrote: On 12/05/2013 10:37 PM, Robert Haas wrote: On Thu, Dec 5, 2013 at 3:05 PM, Tom Lane t...@sss.pgh.pa.us wrote: It might be unpleasant to use in some cases, though. Why would there be more than a few cases in the first place? Who is

Re: [HACKERS] Proof of concept: standalone backend with full FE/BE protocol

2013-12-06 Thread Tom Lane
Andres Freund and...@2ndquadrant.com writes: On 2013-12-05 23:01:28 +0200, Heikki Linnakangas wrote: Right. Not all of the parameters will make sense for a stand-alone backend though, like the hostname and port number. And I think you need need a new parameter to pass the path to the

Re: [HACKERS] Proof of concept: standalone backend with full FE/BE protocol

2013-12-06 Thread Andres Freund
On 2013-12-06 11:02:48 -0500, Tom Lane wrote: Andres Freund and...@2ndquadrant.com writes: My feeling is that we should just treat the executable name and data directory path as new connection parameters, which'd be ignored in normal-connection mode, just as some other parameters will be

Re: [HACKERS] Proof of concept: standalone backend with full FE/BE protocol

2013-12-06 Thread Tom Lane
Andres Freund and...@2ndquadrant.com writes: On 2013-12-06 11:02:48 -0500, Tom Lane wrote: I think the special-purpose command line switches you mention can be passed through PGOPTIONS, rather than inventing a new parameter -- do you have an objection to that? I am not sure if they currently

Re: [HACKERS] Proof of concept: standalone backend with full FE/BE protocol

2013-12-05 Thread Simon Riggs
On 5 December 2013 01:55, Peter Eisentraut pete...@gmx.net wrote: On Thu, 2013-11-14 at 12:11 +0530, Amit Kapila wrote: If an application wants to allow these connection parameters to be used, it would need to do PQenableStartServer() first. If it doesn't, those connection parameters will

Re: [HACKERS] Proof of concept: standalone backend with full FE/BE protocol

2013-12-05 Thread Andres Freund
On 2013-12-04 20:55:08 -0500, Peter Eisentraut wrote: On Thu, 2013-11-14 at 12:11 +0530, Amit Kapila wrote: If an application wants to allow these connection parameters to be used, it would need to do PQenableStartServer() first. If it doesn't, those connection parameters will be

Re: [HACKERS] Proof of concept: standalone backend with full FE/BE protocol

2013-12-05 Thread Peter Eisentraut
On 12/5/13, 6:07 AM, Simon Riggs wrote: On 5 December 2013 01:55, Peter Eisentraut pete...@gmx.net wrote: On Thu, 2013-11-14 at 12:11 +0530, Amit Kapila wrote: If an application wants to allow these connection parameters to be used, it would need to do PQenableStartServer() first. If it

Re: [HACKERS] Proof of concept: standalone backend with full FE/BE protocol

2013-12-05 Thread Peter Eisentraut
I think this proposal is a bit deadlocked now. - There are technical concerns about launching a server executable from within a client. - There are conceptual concerns about promoting an embedded database mode. On the other hand: - Everyone would like to have a way to use psql (and other basic

Re: [HACKERS] Proof of concept: standalone backend with full FE/BE protocol

2013-12-05 Thread Andres Freund
On 2013-12-05 11:39:29 -0500, Peter Eisentraut wrote: I think this proposal is a bit deadlocked now. - There are technical concerns about launching a server executable from within a client. - There are conceptual concerns about promoting an embedded database mode. On the other hand:

Re: [HACKERS] Proof of concept: standalone backend with full FE/BE protocol

2013-12-05 Thread Robert Haas
On Thu, Dec 5, 2013 at 11:52 AM, Andres Freund and...@2ndquadrant.com wrote: On 2013-12-05 11:39:29 -0500, Peter Eisentraut wrote: I think this proposal is a bit deadlocked now. - There are technical concerns about launching a server executable from within a client. - There are conceptual

Re: [HACKERS] Proof of concept: standalone backend with full FE/BE protocol

2013-12-05 Thread Tom Lane
Robert Haas robertmh...@gmail.com writes: Yeah, seriously. I don't understand what the big deal is here. The right design here is 99.44% clear here, and the committer (presumably Tom) can handle the other 0.56% however he'd like. Let's do this and move on. Yeah, but the remaining 0.56% is

Re: [HACKERS] Proof of concept: standalone backend with full FE/BE protocol

2013-12-05 Thread Robert Haas
On Thu, Dec 5, 2013 at 3:05 PM, Tom Lane t...@sss.pgh.pa.us wrote: I'm pretty much persuaded by Andres' point that we should not allow a child process to be launched under a client app without clear permission from the code of the app (and *not* just some environment variable that might have

Re: [HACKERS] Proof of concept: standalone backend with full FE/BE protocol

2013-12-05 Thread Heikki Linnakangas
On 12/05/2013 10:37 PM, Robert Haas wrote: On Thu, Dec 5, 2013 at 3:05 PM, Tom Lane t...@sss.pgh.pa.us wrote: It might be unpleasant to use in some cases, though. Why would there be more than a few cases in the first place? Who is going to use this beyond psql, pg_dump(all), and pg_upgrade,

Re: [HACKERS] Proof of concept: standalone backend with full FE/BE protocol

2013-12-04 Thread Peter Eisentraut
On Thu, 2013-11-14 at 12:11 +0530, Amit Kapila wrote: If an application wants to allow these connection parameters to be used, it would need to do PQenableStartServer() first. If it doesn't, those connection parameters will be rejected. Stupid idea: Would it work that we require an

Re: [HACKERS] Proof of concept: standalone backend with full FE/BE protocol

2013-12-04 Thread Amit Kapila
On Thu, Dec 5, 2013 at 7:25 AM, Peter Eisentraut pete...@gmx.net wrote: On Thu, 2013-11-14 at 12:11 +0530, Amit Kapila wrote: If an application wants to allow these connection parameters to be used, it would need to do PQenableStartServer() first. If it doesn't, those connection parameters

Re: [HACKERS] Proof of concept: standalone backend with full FE/BE protocol

2013-12-04 Thread Peter Eisentraut
On Thu, 2013-12-05 at 09:02 +0530, Amit Kapila wrote: This is certainly not a stupid idea, rather something on similar lines has been discussed previously in this thread. Tom has suggested something similar, but I am not sure if there was a conclusion on that point. Please see the relavant

Re: [HACKERS] Proof of concept: standalone backend with full FE/BE protocol

2013-11-22 Thread Amit Kapila
On Thu, Nov 21, 2013 at 9:54 PM, Amit Kapila amit.kapil...@gmail.com wrote: On Thu, Nov 21, 2013 at 8:14 PM, Tom Lane t...@sss.pgh.pa.us wrote: Amit Kapila amit.kapil...@gmail.com writes: Here what I have in mind is that: Why would you make psql behave differently from our other command-line

Re: [HACKERS] Proof of concept: standalone backend with full FE/BE protocol

2013-11-21 Thread Amit Kapila
On Thu, Nov 21, 2013 at 8:14 PM, Tom Lane t...@sss.pgh.pa.us wrote: Amit Kapila amit.kapil...@gmail.com writes: Here what I have in mind is that: a. In pg_dump or other internal utilities where we want to use this feature, they should call PQenableStart() or some other API before calling

Re: [HACKERS] Proof of concept: standalone backend with full FE/BE protocol

2013-11-21 Thread Tom Lane
Amit Kapila amit.kapil...@gmail.com writes: Here what I have in mind is that: a. In pg_dump or other internal utilities where we want to use this feature, they should call PQenableStart() or some other API before calling PQConnect() which will indicate that it wants to operate as a

Re: [HACKERS] Proof of concept: standalone backend with full FE/BE protocol

2013-11-21 Thread Amit Kapila
On Thu, Nov 21, 2013 at 9:11 AM, Amit Kapila amit.kapil...@gmail.com wrote: On Thu, Nov 21, 2013 at 2:14 AM, Tom Lane t...@sss.pgh.pa.us wrote: Peter Eisentraut pete...@gmx.net writes: The argument elsewhere in this thread was that the reason for putting this in the connection options was so

Re: [HACKERS] Proof of concept: standalone backend with full FE/BE protocol

2013-11-20 Thread Peter Eisentraut
On 11/14/13, 1:41 AM, Amit Kapila wrote: Security Concern - If a user can specify libpq connection options, he can now execute any file he wants by passing it as standalone_backend. Method to resolve Security concern

Re: [HACKERS] Proof of concept: standalone backend with full FE/BE protocol

2013-11-20 Thread Tom Lane
Peter Eisentraut pete...@gmx.net writes: I would consider sidestepping this entire issue by having the stand-alone backend create a Unix-domain socket and have a client connect to that in the normal way. Hmm. But that requires the stand-alone backend to take on at least some properties of a

Re: [HACKERS] Proof of concept: standalone backend with full FE/BE protocol

2013-11-20 Thread Andres Freund
On 2013-11-20 10:48:20 -0500, Tom Lane wrote: constraining what can be executed as a standalone backend. Would it work to insist that psql/pg_dump launch the program named postgres from the same bin directory they're in, rather than accepting a path from the connection string? But why do we

Re: [HACKERS] Proof of concept: standalone backend with full FE/BE protocol

2013-11-20 Thread Tom Lane
Andres Freund and...@2ndquadrant.com writes: On 2013-11-20 10:48:20 -0500, Tom Lane wrote: constraining what can be executed as a standalone backend. Would it work to insist that psql/pg_dump launch the program named postgres from the same bin directory they're in, rather than accepting a

Re: [HACKERS] Proof of concept: standalone backend with full FE/BE protocol

2013-11-20 Thread Andres Freund
On 2013-11-20 11:08:33 -0500, Tom Lane wrote: Andres Freund and...@2ndquadrant.com writes: On 2013-11-20 10:48:20 -0500, Tom Lane wrote: constraining what can be executed as a standalone backend. Would it work to insist that psql/pg_dump launch the program named postgres from the same

Re: [HACKERS] Proof of concept: standalone backend with full FE/BE protocol

2013-11-20 Thread Stephen Frost
* Tom Lane (t...@sss.pgh.pa.us) wrote: I think we'd be better off trying to fix the security issue by constraining what can be executed as a standalone backend. Would it work to insist that psql/pg_dump launch the program named postgres from the same bin directory they're in, rather than

Re: [HACKERS] Proof of concept: standalone backend with full FE/BE protocol

2013-11-20 Thread Andres Freund
On 2013-11-20 17:19:42 +0100, Andres Freund wrote: That just pushes the problem up a level --- how are you going to tell psql, pg_dump, or other programs that they should do that? An explicit parameter. A program imo explicitly needs to be aware that a PQconnect() suddenly starts forking

Re: [HACKERS] Proof of concept: standalone backend with full FE/BE protocol

2013-11-20 Thread Tom Lane
Andres Freund and...@2ndquadrant.com writes: On 2013-11-20 11:08:33 -0500, Tom Lane wrote: Andres Freund and...@2ndquadrant.com writes: Something like PQstartSingleUser(dsn) returning a established connection seems better to me. That just pushes the problem up a level --- how are you going

Re: [HACKERS] Proof of concept: standalone backend with full FE/BE protocol

2013-11-20 Thread Peter Eisentraut
On 11/20/13, 11:31 AM, Stephen Frost wrote: Couldn't that be an issue for people who have multiple major versions of binaries installed? In particular, the default on the system for psql might be 9.3 while the cluster you're trying to recover may be 9.2. Of course, in that case you might say

Re: [HACKERS] Proof of concept: standalone backend with full FE/BE protocol

2013-11-20 Thread Peter Eisentraut
On 11/20/13, 10:48 AM, Tom Lane wrote: Perhaps more to the point, I think this approach actually breaks one of the principal good-thing-in-emergencies attributes of standalone mode, namely being sure that nobody but you can connect. With this, you're right back to having a race condition as

Re: [HACKERS] Proof of concept: standalone backend with full FE/BE protocol

2013-11-20 Thread Robert Haas
On Wed, Nov 20, 2013 at 10:13 AM, Peter Eisentraut pete...@gmx.net wrote: On 11/14/13, 1:41 AM, Amit Kapila wrote: Security Concern - If a user can specify libpq connection options, he can now execute any file he wants by passing it as standalone_backend.

Re: [HACKERS] Proof of concept: standalone backend with full FE/BE protocol

2013-11-20 Thread Peter Eisentraut
On 11/20/13, 3:24 PM, Robert Haas wrote: The point is that client applications should expose whether or not to set this function as a command-line switch separate from whatever they accept in terms of connection strings. So pg_dump should have a flag called --standalone-server or something

Re: [HACKERS] Proof of concept: standalone backend with full FE/BE protocol

2013-11-20 Thread Robert Haas
On Wed, Nov 20, 2013 at 3:32 PM, Peter Eisentraut pete...@gmx.net wrote: On 11/20/13, 3:24 PM, Robert Haas wrote: The point is that client applications should expose whether or not to set this function as a command-line switch separate from whatever they accept in terms of connection strings.

Re: [HACKERS] Proof of concept: standalone backend with full FE/BE protocol

2013-11-20 Thread Tom Lane
Peter Eisentraut pete...@gmx.net writes: The argument elsewhere in this thread was that the reason for putting this in the connection options was so that you do *not* have to patch up every client to be able to use this functionality. If you have to add separate options everywhere, then you

Re: [HACKERS] Proof of concept: standalone backend with full FE/BE protocol

2013-11-20 Thread Gurjeet Singh
On Wed, Nov 20, 2013 at 3:44 PM, Tom Lane t...@sss.pgh.pa.us wrote: To my mind, the create a socket and hope nobody else can get to it approach is exactly one of the main things we're trying to avoid here. If you'll recall, awhile back we had a big discussion about how pg_upgrade could

Re: [HACKERS] Proof of concept: standalone backend with full FE/BE protocol

2013-11-20 Thread Andres Freund
On 2013-11-20 15:44:03 -0500, Tom Lane wrote: In practice, as long as psql and pg_dump and pg_upgrade can do it, I think we've covered most of the interesting bases. I'd say vacuumdb/reindexdb should be added to that list. In my experience xid wraparound and corrupted system indexes are the

Re: [HACKERS] Proof of concept: standalone backend with full FE/BE protocol

2013-11-20 Thread Bruce Momjian
On Wed, Nov 20, 2013 at 05:38:14PM -0500, Gurjeet Singh wrote: On Wed, Nov 20, 2013 at 3:44 PM, Tom Lane t...@sss.pgh.pa.us wrote: To my mind, the create a socket and hope nobody else can get to it approach is exactly one of the main things we're trying to avoid here. If you'll

Re: [HACKERS] Proof of concept: standalone backend with full FE/BE protocol

2013-11-20 Thread Amit Kapila
On Thu, Nov 21, 2013 at 2:14 AM, Tom Lane t...@sss.pgh.pa.us wrote: Peter Eisentraut pete...@gmx.net writes: The argument elsewhere in this thread was that the reason for putting this in the connection options was so that you do *not* have to patch up every client to be able to use this

Re: [HACKERS] Proof of concept: standalone backend with full FE/BE protocol

2013-11-19 Thread Robert Haas
On Fri, Nov 15, 2013 at 6:51 AM, Simon Riggs si...@2ndquadrant.com wrote: Not enough. This feature is clearly being suggested as a way to offer Postgres in embedded mode for users by a back door. Doing that forces us to turn off many of the server's features and we will take a huge step

Re: [HACKERS] Proof of concept: standalone backend with full FE/BE protocol

2013-11-19 Thread Tom Lane
Robert Haas robertmh...@gmail.com writes: On Fri, Nov 15, 2013 at 6:51 AM, Simon Riggs si...@2ndquadrant.com wrote: Not enough. This feature is clearly being suggested as a way to offer Postgres in embedded mode for users by a back door. Doing that forces us to turn off many of the server's

Re: [HACKERS] Proof of concept: standalone backend with full FE/BE protocol

2013-11-16 Thread Amit Kapila
On Fri, Nov 15, 2013 at 5:21 PM, Simon Riggs si...@2ndquadrant.com wrote: On 14 November 2013 03:41, Amit Kapila amit.kapil...@gmail.com wrote: I have gone through the mail chain of this thread and tried to find the different concerns or open ends for this patch. Not enough. This feature is

Re: [HACKERS] Proof of concept: standalone backend with full FE/BE protocol

2013-11-15 Thread Simon Riggs
On 14 November 2013 03:41, Amit Kapila amit.kapil...@gmail.com wrote: I have gone through the mail chain of this thread and tried to find the different concerns or open ends for this patch. Not enough. This feature is clearly being suggested as a way to offer Postgres in embedded mode for

Re: [HACKERS] Proof of concept: standalone backend with full FE/BE protocol

2013-11-15 Thread Andres Freund
On 2013-11-15 09:51:28 -0200, Simon Riggs wrote: On 14 November 2013 03:41, Amit Kapila amit.kapil...@gmail.com wrote: I have gone through the mail chain of this thread and tried to find the different concerns or open ends for this patch. Not enough. This feature is clearly being

Re: [HACKERS] Proof of concept: standalone backend with full FE/BE protocol

2013-11-15 Thread Dimitri Fontaine
Andres Freund and...@2ndquadrant.com writes: I think fixing single user mode to work halfway reasonable is enough justification for the feature. Having to deal with that when solving critical issues is just embarassing. +1 But: I very, very much agree with the other concerns around this.

Re: [HACKERS] Proof of concept: standalone backend with full FE/BE protocol

2013-11-15 Thread Simon Riggs
On 15 November 2013 09:00, Andres Freund and...@2ndquadrant.com wrote: This should be a patch to fix single user mode, not one to make postgres into a single process database. +1 -- Simon Riggs http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training

Re: [HACKERS] Proof of concept: standalone backend with full FE/BE protocol

2013-11-15 Thread Merlin Moncure
On Fri, Nov 15, 2013 at 6:06 AM, Dimitri Fontaine dimi...@2ndquadrant.fr wrote: But: I very, very much agree with the other concerns around this. This should be a patch to fix single user mode, not one to make postgres into a single process database. It's not, and trying to make it by using

Re: [HACKERS] Proof of concept: standalone backend with full FE/BE protocol

2013-11-13 Thread Amit Kapila
I have gone through the mail chain of this thread and tried to find the different concerns or open ends for this patch. Summarisation of the discussion and concerns for this patch: 1. Security concern in interface 2. Security concern in Windows implementation 3. Handling of Ctrl-C/SIGTERM 4.

Re: [HACKERS] Proof of concept: standalone backend with full FE/BE protocol

2013-11-13 Thread Tom Lane
Amit Kapila amit.kapil...@gmail.com writes: Any objections for adding this idea/patch to CF? You should certainly add it to the CF. You've listed lots of matters for review, but that's no reason to not get it in the queue to be reviewed. regards, tom lane -- Sent via

Re: [HACKERS] Proof of concept: standalone backend with full FE/BE protocol

2012-11-13 Thread Andres Freund
On 2012-11-12 19:21:28 +, Simon Riggs wrote: On 10 September 2012 17:50, Tom Lane t...@sss.pgh.pa.us wrote: The point of the proposal that I am making is to have a simple, low-maintenance solution for people who need a single-application database. A compromise somewhere in the middle

Re: [HACKERS] Proof of concept: standalone backend with full FE/BE protocol

2012-11-13 Thread Simon Riggs
On 13 November 2012 06:14, Amit kapila amit.kap...@huawei.com wrote: I get the installability thang, very very much, I just don't see the single process thing as the only solution. At very least an open minded analysis of the actual problem and ways of solving it is called for, not just reach for

Re: [HACKERS] Proof of concept: standalone backend with full FE/BE protocol

2012-11-13 Thread Alvaro Herrera
Simon Riggs escribió: So even if this solution doesn't meet all requirements of single process solution (and neither I think it is written to address all) but can't we think of it as first version and then based on requirements extend it to have other capabilities: a. to have a mechnism

Re: [HACKERS] Proof of concept: standalone backend with full FE/BE protocol

2012-11-13 Thread Simon Riggs
On 13 November 2012 13:05, Alvaro Herrera alvhe...@2ndquadrant.com wrote: Simon Riggs escribió: So even if this solution doesn't meet all requirements of single process solution (and neither I think it is written to address all) but can't we think of it as first version and then based on

Re: [HACKERS] Proof of concept: standalone backend with full FE/BE protocol

2012-11-13 Thread Tom Lane
Simon Riggs si...@2ndquadrant.com writes: The most popular relational database in the world is Microsoft Access, not MySQL. Access appears desirable because it allows a single user to create and use a database (which is very good). But all business databases have a requirement for at least one

Re: [HACKERS] Proof of concept: standalone backend with full FE/BE protocol

2012-11-13 Thread Robert Haas
On Tue, Nov 13, 2012 at 12:38 PM, Tom Lane t...@sss.pgh.pa.us wrote: Simon Riggs si...@2ndquadrant.com writes: The most popular relational database in the world is Microsoft Access, not MySQL. Access appears desirable because it allows a single user to create and use a database (which is very

Re: [HACKERS] Proof of concept: standalone backend with full FE/BE protocol

2012-11-13 Thread Simon Riggs
On 13 November 2012 17:38, Tom Lane t...@sss.pgh.pa.us wrote: Simon Riggs si...@2ndquadrant.com writes: The most popular relational database in the world is Microsoft Access, not MySQL. Access appears desirable because it allows a single user to create and use a database (which is very good).

Re: [HACKERS] Proof of concept: standalone backend with full FE/BE protocol

2012-11-13 Thread Tom Lane
Simon Riggs si...@2ndquadrant.com writes: On 13 November 2012 17:38, Tom Lane t...@sss.pgh.pa.us wrote: ... The fact of the matter is that there is *lots* of demand for simple single-user databases, and what I'm proposing is at least a first step towards getting there. I agree there is lots

Re: [HACKERS] Proof of concept: standalone backend with full FE/BE protocol

2012-11-13 Thread Christopher Browne
Preface: I think there's some great commentary here, and find myself agreeing pretty whole-heartedly. On Tue, Nov 13, 2012 at 2:45 PM, Simon Riggs si...@2ndquadrant.com wrote: On 13 November 2012 17:38, Tom Lane t...@sss.pgh.pa.us wrote: Simon Riggs si...@2ndquadrant.com writes: The most

Re: [HACKERS] Proof of concept: standalone backend with full FE/BE protocol

2012-11-13 Thread Dimitri Fontaine
Tom Lane t...@sss.pgh.pa.us writes: I agree there is lots of demand for simple single-user databases and I wish that too. What I don't agree with is something that casts that requirement in stone by architecturally/permanently disallowing secondary connections. If you want secondary

Re: [HACKERS] Proof of concept: standalone backend with full FE/BE protocol

2012-11-12 Thread Merlin Moncure
On Fri, Sep 14, 2012 at 6:42 AM, Amit kapila amit.kap...@huawei.com wrote: On Tuesday, September 11, 2012 7:07 PM Amit Kapila wrote: On Monday, September 10, 2012 8:20 PM Amit Kapila wrote: On Sunday, September 09, 2012 1:37 PM Amit Kapila wrote: On Friday, September 07, 2012 11:19 PM Tom Lane

Re: [HACKERS] Proof of concept: standalone backend with full FE/BE protocol

2012-11-12 Thread Simon Riggs
On 10 September 2012 17:50, Tom Lane t...@sss.pgh.pa.us wrote: The point of the proposal that I am making is to have a simple, low-maintenance solution for people who need a single-application database. A compromise somewhere in the middle isn't likely to be an improvement for anybody. For

Re: [HACKERS] Proof of concept: standalone backend with full FE/BE protocol

2012-11-12 Thread Merlin Moncure
On Mon, Nov 12, 2012 at 1:21 PM, Simon Riggs si...@2ndquadrant.com wrote: On 10 September 2012 17:50, Tom Lane t...@sss.pgh.pa.us wrote: The point of the proposal that I am making is to have a simple, low-maintenance solution for people who need a single-application database. A compromise

Re: [HACKERS] Proof of concept: standalone backend with full FE/BE protocol

2012-11-12 Thread Simon Riggs
On 12 November 2012 21:26, Merlin Moncure mmonc...@gmail.com wrote: I couldn't disagree more. The patch is small, logical, and fixes an awful problem, namely that --single mode is basically unusable. As to your wider point (namely, that you can't connect to it, therefore it's bad), it has

Re: [HACKERS] Proof of concept: standalone backend with full FE/BE protocol

2012-11-12 Thread Amit kapila
On Monday, November 12, 2012 8:31 PM Merlin Moncure wrote: On Fri, Sep 14, 2012 at 6:42 AM, Amit kapila amit.kap...@huawei.com wrote: On Tuesday, September 11, 2012 7:07 PM Amit Kapila wrote: On Monday, September 10, 2012 8:20 PM Amit Kapila wrote: On Sunday, September 09, 2012 1:37 PM Amit

Re: [HACKERS] Proof of concept: standalone backend with full FE/BE protocol

2012-11-12 Thread Amit kapila
On Tuesday, November 13, 2012 3:11 AM Simon Riggs wrote: On 12 November 2012 21:26, Merlin Moncure mmonc...@gmail.com wrote: I couldn't disagree more. The patch is small, logical, and fixes an awful problem, namely that --single mode is basically unusable. As to your wider point (namely,

Re: [HACKERS] Proof of concept: standalone backend with full FE/BE protocol

2012-09-14 Thread Amit kapila
On Tuesday, September 11, 2012 7:07 PM Amit Kapila wrote: On Monday, September 10, 2012 8:20 PM Amit Kapila wrote: On Sunday, September 09, 2012 1:37 PM Amit Kapila wrote: On Friday, September 07, 2012 11:19 PM Tom Lane wrote: Heikki Linnakangas hlinn...@iki.fi writes: Would socketpair(2) be

Re: [HACKERS] Proof of concept: standalone backend with full FE/BE protocol

2012-09-11 Thread Amit kapila
On Monday, September 10, 2012 8:20 PM Amit Kapila wrote: On Sunday, September 09, 2012 1:37 PM Amit Kapila wrote: On Friday, September 07, 2012 11:19 PM Tom Lane wrote: Heikki Linnakangas hlinn...@iki.fi writes: Would socketpair(2) be simpler? I've not done anything yet about the potential

Re: [HACKERS] Proof of concept: standalone backend with full FE/BE protocol

2012-09-10 Thread Amit Kapila
On Sunday, September 09, 2012 1:37 PM Amit Kapila wrote: On Friday, September 07, 2012 11:19 PM Tom Lane wrote: Heikki Linnakangas hlinn...@iki.fi writes: Would socketpair(2) be simpler? I've not done anything yet about the potential security issues associated with untrusted libpq connection

Re: [HACKERS] Proof of concept: standalone backend with full FE/BE protocol

2012-09-10 Thread Gurjeet Singh
On Sun, Sep 2, 2012 at 8:23 PM, Tom Lane t...@sss.pgh.pa.us wrote: Notably, while the lack of any background processes is just what you want for pg_upgrade and disaster recovery, an ordinary application is probably going to want to rely on autovacuum; and we need bgwriter and other background

Re: [HACKERS] Proof of concept: standalone backend with full FE/BE protocol

2012-09-10 Thread Aidan Van Dyk
On Mon, Sep 10, 2012 at 11:12 AM, Gurjeet Singh singh.gurj...@gmail.com wrote: On Sun, Sep 2, 2012 at 8:23 PM, Tom Lane t...@sss.pgh.pa.us wrote: Notably, while the lack of any background processes is just what you want for pg_upgrade and disaster recovery, an ordinary application is probably

Re: [HACKERS] Proof of concept: standalone backend with full FE/BE protocol

2012-09-10 Thread Heikki Linnakangas
On 10.09.2012 18:12, Gurjeet Singh wrote: On Sun, Sep 2, 2012 at 8:23 PM, Tom Lanet...@sss.pgh.pa.us wrote: Notably, while the lack of any background processes is just what you want for pg_upgrade and disaster recovery, an ordinary application is probably going to want to rely on autovacuum;

Re: [HACKERS] Proof of concept: standalone backend with full FE/BE protocol

2012-09-10 Thread Gurjeet Singh
On Mon, Sep 10, 2012 at 11:43 AM, Heikki Linnakangas hlinn...@iki.fiwrote: On 10.09.2012 18:12, Gurjeet Singh wrote: On Sun, Sep 2, 2012 at 8:23 PM, Tom Lanet...@sss.pgh.pa.us wrote: Notably, while the lack of any background processes is just what you want for pg_upgrade and disaster

Re: [HACKERS] Proof of concept: standalone backend with full FE/BE protocol

2012-09-10 Thread Tom Lane
Gurjeet Singh singh.gurj...@gmail.com writes: On Mon, Sep 10, 2012 at 11:43 AM, Heikki Linnakangas hlinn...@iki.fiwrote: [scratches head] How's that different from the normal postmaster mode? As I described in later paragraphs, it'd behave like an embedded database, like SQLite etc., so the

Re: [HACKERS] Proof of concept: standalone backend with full FE/BE protocol

2012-09-10 Thread Josh Berkus
The point of the proposal that I am making is to have a simple, low-maintenance solution for people who need a single-application database. A compromise somewhere in the middle isn't likely to be an improvement for anybody. For instance, if you want to have additional connections, you open

Re: [HACKERS] Proof of concept: standalone backend with full FE/BE protocol

2012-09-10 Thread Kevin Grittner
Josh Berkus j...@agliodbs.com wrote: In fact, most of the folks who would want an embedded PostgreSQL either want no authentication at all, or only a single password. So supporting authentication options other than trust or md5 is not required, or desired AFAIK. I don't know whether it's

Re: [HACKERS] Proof of concept: standalone backend with full FE/BE protocol

2012-09-10 Thread Daniel Farina
On Mon, Sep 10, 2012 at 9:59 AM, Josh Berkus j...@agliodbs.com wrote: The point of the proposal that I am making is to have a simple, low-maintenance solution for people who need a single-application database. A compromise somewhere in the middle isn't likely to be an improvement for

Re: [HACKERS] Proof of concept: standalone backend with full FE/BE protocol

2012-09-09 Thread Amit kapila
On Friday, September 07, 2012 11:19 PM Tom Lane wrote: Heikki Linnakangas hlinn...@iki.fi writes: Would socketpair(2) be simpler? I've not done anything yet about the potential security issues associated with untrusted libpq connection strings. I think this is still at the proof-of-concept

Re: [HACKERS] Proof of concept: standalone backend with full FE/BE protocol

2012-09-09 Thread Tom Lane
Amit kapila amit.kap...@huawei.com writes: 1. does this follow the behavior that admin users will not be allowed to invoke postgres child process? That's an interesting question. I'm not sure if we'd want to disable the no-root check on the Unix side, but it might make sense to. But this has

Re: [HACKERS] Proof of concept: standalone backend with full FE/BE protocol

2012-09-09 Thread Andres Freund
Hi, On Wednesday, September 05, 2012 06:00:18 PM Tom Lane wrote: anara...@anarazel.de and...@anarazel.de writes: I am not saying its bad that it is slower, that's absolutely OK. Just that it will take a variable amount of time till you can run pgdump again and its not easily detectable

Re: [HACKERS] Proof of concept: standalone backend with full FE/BE protocol

2012-09-09 Thread Amit Kapila
On Sunday, September 09, 2012 8:46 PM Tom Lane wrote: Amit kapila amit.kap...@huawei.com writes: 1. does this follow the behavior that admin users will not be allowed to invoke postgres child process? That's an interesting question. I'm not sure if we'd want to disable the no-root check on

Re: [HACKERS] Proof of concept: standalone backend with full FE/BE protocol

2012-09-08 Thread Albert Cervera i Areny
A Dijous, 6 de setembre de 2012 00:30:53, Josh Berkus va escriure: In general I think the selling point for such a feature would be no administrative hassles, and I believe that has to go not only for the end-user experience but also for the application-developer experience. If you have to

Re: [HACKERS] Proof of concept: standalone backend with full FE/BE protocol

2012-09-07 Thread Tom Lane
Heikki Linnakangas hlinn...@iki.fi writes: Would socketpair(2) be simpler? Attached is a revised version of the patch that uses socketpair(2). This is definitely a lot less invasive --- the backend side of the patch, in particular, is far shorter, and there are fewer portability hazards since

Re: [HACKERS] Proof of concept: standalone backend with full FE/BE protocol

2012-09-07 Thread Heikki Linnakangas
On 07.09.2012 10:49, Tom Lane wrote: Heikki Linnakangashlinn...@iki.fi writes: Would socketpair(2) be simpler? Attached is a revised version of the patch that uses socketpair(2). This is definitely a lot less invasive --- the backend side of the patch, in particular, is far shorter, and

Re: [HACKERS] Proof of concept: standalone backend with full FE/BE protocol

2012-09-07 Thread Tom Lane
Heikki Linnakangas hlinn...@iki.fi writes: It's worth noting that now that libpq constructs the command line to execute postgres --child= -D datadir, we'll be stuck with that set of arguments forever, because libpq needs to be able to talk to different versions. Or at least we'd need to

Re: [HACKERS] Proof of concept: standalone backend with full FE/BE protocol

2012-09-07 Thread Tom Lane
Heikki Linnakangas hlinn...@iki.fi writes: On 07.09.2012 10:49, Tom Lane wrote: I'm a bit tempted though to pull out and apply the portions of the patch that replace libpq's assorted ad-hoc closesocket() calls with a centralized pqDropConnection routine. I think that's probably a good idea

Re: [HACKERS] Proof of concept: standalone backend with full FE/BE protocol

2012-09-07 Thread Merlin Moncure
On Thu, Sep 6, 2012 at 12:56 PM, Jeff Davis pg...@j-davis.com wrote: On Wed, 2012-09-05 at 17:03 -0400, Tom Lane wrote: In general I think the selling point for such a feature would be no administrative hassles, and I believe that has to go not only for the end-user experience but also for the

Re: [HACKERS] Proof of concept: standalone backend with full FE/BE protocol

2012-09-07 Thread Andres Freund
On Friday, September 07, 2012 11:21:00 PM Merlin Moncure wrote: On Thu, Sep 6, 2012 at 12:56 PM, Jeff Davis pg...@j-davis.com wrote: On Wed, 2012-09-05 at 17:03 -0400, Tom Lane wrote: In general I think the selling point for such a feature would be no administrative hassles, and I believe

Re: [HACKERS] Proof of concept: standalone backend with full FE/BE protocol

2012-09-07 Thread Jim Nasby
On 9/2/12 7:23 PM, Tom Lane wrote: 4. As coded, the backend assumes the incoming pipe is on its FD 0 and the outgoing pipe is on its FD 1. This made the command line simple but I'm having second thoughts about it: if anything inside the backend tries to read stdin or write stdout, unpleasant

Re: [HACKERS] Proof of concept: standalone backend with full FE/BE protocol

2012-09-07 Thread Gurjeet Singh
On Wed, Sep 5, 2012 at 10:29 PM, Andrew Dunstan and...@dunslane.net wrote: On 09/05/2012 10:14 PM, Tom Lane wrote: The people who would be interested in this are currently using something like SQLite within a single application program. Exactly. I think it's worth stating that this has

Re: [HACKERS] Proof of concept: standalone backend with full FE/BE protocol

2012-09-06 Thread Jeff Davis
On Wed, 2012-09-05 at 17:03 -0400, Tom Lane wrote: In general I think the selling point for such a feature would be no administrative hassles, and I believe that has to go not only for the end-user experience but also for the application-developer experience. If you have to manage

Re: [HACKERS] Proof of concept: standalone backend with full FE/BE protocol

2012-09-05 Thread Magnus Hagander
On Wed, Sep 5, 2012 at 7:31 AM, Amit Kapila amit.kap...@huawei.com wrote: On Tuesday, September 04, 2012 12:40 AM Tom Lane wrote: Magnus Hagander mag...@hagander.net writes: On Mon, Sep 3, 2012 at 8:51 PM, Tom Lane t...@sss.pgh.pa.us wrote: I have another question after thinking about that for

Re: [HACKERS] Proof of concept: standalone backend with full FE/BE protocol

2012-09-05 Thread Amit Kapila
On Wednesday, September 05, 2012 3:58 PM Magnus Hagander wrote: On Wed, Sep 5, 2012 at 7:31 AM, Amit Kapila amit.kap...@huawei.com wrote: On Tuesday, September 04, 2012 12:40 AM Tom Lane wrote: Magnus Hagander mag...@hagander.net writes: On Mon, Sep 3, 2012 at 8:51 PM, Tom Lane

Re: [HACKERS] Proof of concept: standalone backend with full FE/BE protocol

2012-09-05 Thread Andres Freund
On Tuesday, September 04, 2012 12:11:28 PM Amit Kapila wrote: On Tuesday, September 04, 2012 11:00 AM Andres Freund wrote: On Tuesday, September 04, 2012 06:20:59 AM Tom Lane wrote: Andres Freund and...@2ndquadrant.com writes: I can see why that would be nice, but is it really realistic?

Re: [HACKERS] Proof of concept: standalone backend with full FE/BE protocol

2012-09-05 Thread Tom Lane
Andres Freund and...@2ndquadrant.com writes: I don't find that a convincing comparison. Normally don't need to shutdown the server between two pg_dump commands. Which very well might be scripted. Especially as for now, without a background writer/checkpointer writing stuff beforehand, the

Re: [HACKERS] Proof of concept: standalone backend with full FE/BE protocol

2012-09-05 Thread Tom Lane
anara...@anarazel.de and...@anarazel.de writes: I am not saying its bad that it is slower, that's absolutely OK. Just that it will take a variable amount of time till you can run pgdump again and its not easily detectable without looping and trying again. Well, that's why the proposed libpq

Re: [HACKERS] Proof of concept: standalone backend with full FE/BE protocol

2012-09-05 Thread anara...@anarazel.de
Tom Lane t...@sss.pgh.pa.us schrieb: Andres Freund and...@2ndquadrant.com writes: I don't find that a convincing comparison. Normally don't need to shutdown the server between two pg_dump commands. Which very well might be scripted. Especially as for now, without a background

Re: [HACKERS] Proof of concept: standalone backend with full FE/BE protocol

2012-09-05 Thread Josh Berkus
Tom, However, there are some additional things we'd need to think about before advertising it as a fit solution for that. Notably, while the lack of any background processes is just what you want for pg_upgrade and disaster recovery, an ordinary application is probably going to want to rely

Re: [HACKERS] Proof of concept: standalone backend with full FE/BE protocol

2012-09-05 Thread Tom Lane
Josh Berkus j...@agliodbs.com writes: However, there are some additional things we'd need to think about before advertising it as a fit solution for that. Notably, while the lack of any background processes is just what you want for pg_upgrade and disaster recovery, an ordinary application is

Re: [HACKERS] Proof of concept: standalone backend with full FE/BE protocol

2012-09-05 Thread Bruce Momjian
On Wed, Sep 5, 2012 at 01:50:06PM -0700, Josh Berkus wrote: Tom, However, there are some additional things we'd need to think about before advertising it as a fit solution for that. Notably, while the lack of any background processes is just what you want for pg_upgrade and disaster

Re: [HACKERS] Proof of concept: standalone backend with full FE/BE protocol

2012-09-05 Thread Peter Eisentraut
On 9/5/12 5:03 PM, Tom Lane wrote: I don't see people wanting to use this feature for unit tests. If this is going to become an official feature (as opposed to an internal interface only for use by pg_upgrade), then I think that's exactly what people will want to use it for. In fact, it might

  1   2   >