Re: [HACKERS] Proof of concept: standalone backend with full FE/BE protocol

2013-12-09 Thread Bruce Momjian
On Fri, Dec 6, 2013 at 04:04:36PM +0100, Andres Freund wrote: > On 2013-12-05 23:01:28 +0200, Heikki Linnakangas wrote: > > On 12/05/2013 10:37 PM, Robert Haas wrote: > > >On Thu, Dec 5, 2013 at 3:05 PM, Tom Lane wrote: > > >>It might be unpleasant to use in some cases, though. > > > > > >Why wou

Re: [HACKERS] Proof of concept: standalone backend with full FE/BE protocol

2013-12-06 Thread Tom Lane
Andres Freund writes: > On 2013-12-06 11:02:48 -0500, Tom Lane wrote: >> I think the special-purpose command line switches you mention can be >> passed through PGOPTIONS, rather than inventing a new parameter -- do you >> have an objection to that? > I am not sure if they currently will get recog

Re: [HACKERS] Proof of concept: standalone backend with full FE/BE protocol

2013-12-06 Thread Andres Freund
On 2013-12-06 11:02:48 -0500, Tom Lane wrote: > Andres Freund writes: > My feeling is that we should just treat the executable name and data > directory path as new connection parameters, which'd be ignored in > normal-connection mode, just as some other parameters will be ignored in > single-user

Re: [HACKERS] Proof of concept: standalone backend with full FE/BE protocol

2013-12-06 Thread Tom Lane
Andres Freund writes: > On 2013-12-05 23:01:28 +0200, Heikki Linnakangas wrote: >> Right. Not all of the parameters will make sense for a stand-alone backend >> though, like the hostname and port number. And I think you need need a new >> parameter to pass the path to the 'postgres' executable, un

Re: [HACKERS] Proof of concept: standalone backend with full FE/BE protocol

2013-12-06 Thread Andres Freund
On 2013-12-05 23:01:28 +0200, Heikki Linnakangas wrote: > On 12/05/2013 10:37 PM, Robert Haas wrote: > >On Thu, Dec 5, 2013 at 3:05 PM, Tom Lane wrote: > >>It might be unpleasant to use in some cases, though. > > > >Why would there be more than a few cases in the first place? Who is > >going to u

Re: [HACKERS] Proof of concept: standalone backend with full FE/BE protocol

2013-12-05 Thread Heikki Linnakangas
On 12/05/2013 10:37 PM, Robert Haas wrote: On Thu, Dec 5, 2013 at 3:05 PM, Tom Lane wrote: It might be unpleasant to use in some cases, though. Why would there be more than a few cases in the first place? Who is going to use this beyond psql, pg_dump(all), and pg_upgrade, and why? Well, yo

Re: [HACKERS] Proof of concept: standalone backend with full FE/BE protocol

2013-12-05 Thread Robert Haas
On Thu, Dec 5, 2013 at 3:05 PM, Tom Lane wrote: > I'm pretty much persuaded by Andres' point that we should not allow a > child process to be launched under a client app without clear permission > from the code of the app (and *not* just some environment variable that > might have been set far awa

Re: [HACKERS] Proof of concept: standalone backend with full FE/BE protocol

2013-12-05 Thread Tom Lane
Robert Haas writes: > Yeah, seriously. I don't understand what the big deal is here. The > right design here is 99.44% clear here, and the committer (presumably > Tom) can handle the other 0.56% however he'd like. Let's do this and > move on. Yeah, but the remaining 0.56% is an important decis

Re: [HACKERS] Proof of concept: standalone backend with full FE/BE protocol

2013-12-05 Thread Robert Haas
On Thu, Dec 5, 2013 at 11:52 AM, Andres Freund wrote: > On 2013-12-05 11:39:29 -0500, Peter Eisentraut wrote: >> I think this proposal is a bit deadlocked now. >> >> - There are technical concerns about launching a server executable from >> within a client. >> >> - There are conceptual concerns ab

Re: [HACKERS] Proof of concept: standalone backend with full FE/BE protocol

2013-12-05 Thread Andres Freund
On 2013-12-05 11:39:29 -0500, Peter Eisentraut wrote: > I think this proposal is a bit deadlocked now. > > - There are technical concerns about launching a server executable from > within a client. > > - There are conceptual concerns about promoting an embedded database mode. > > On the other ha

Re: [HACKERS] Proof of concept: standalone backend with full FE/BE protocol

2013-12-05 Thread Peter Eisentraut
I think this proposal is a bit deadlocked now. - There are technical concerns about launching a server executable from within a client. - There are conceptual concerns about promoting an embedded database mode. On the other hand: - Everyone would like to have a way to use psql (and other basic

Re: [HACKERS] Proof of concept: standalone backend with full FE/BE protocol

2013-12-05 Thread Peter Eisentraut
On 12/5/13, 6:07 AM, Simon Riggs wrote: > On 5 December 2013 01:55, Peter Eisentraut wrote: >> On Thu, 2013-11-14 at 12:11 +0530, Amit Kapila wrote: >>>If an application wants to allow these connection parameters to be >>> used, it would need to do PQenableStartServer() first. If it doesn't, >

Re: [HACKERS] Proof of concept: standalone backend with full FE/BE protocol

2013-12-05 Thread Andres Freund
On 2013-12-04 20:55:08 -0500, Peter Eisentraut wrote: > On Thu, 2013-11-14 at 12:11 +0530, Amit Kapila wrote: > >If an application wants to allow these connection parameters to be > > used, it would need to do PQenableStartServer() first. If it doesn't, > > those connection parameters will be r

Re: [HACKERS] Proof of concept: standalone backend with full FE/BE protocol

2013-12-05 Thread Simon Riggs
On 5 December 2013 01:55, Peter Eisentraut wrote: > On Thu, 2013-11-14 at 12:11 +0530, Amit Kapila wrote: >>If an application wants to allow these connection parameters to be >> used, it would need to do PQenableStartServer() first. If it doesn't, >> those connection parameters will be rejecte

Re: [HACKERS] Proof of concept: standalone backend with full FE/BE protocol

2013-12-04 Thread Peter Eisentraut
On Thu, 2013-12-05 at 09:02 +0530, Amit Kapila wrote: > This is certainly not a stupid idea, rather something on similar lines > has been discussed previously in this thread. > Tom has suggested something similar, but I am not sure if there was a > conclusion on that point. Please see the > relavan

Re: [HACKERS] Proof of concept: standalone backend with full FE/BE protocol

2013-12-04 Thread Amit Kapila
On Thu, Dec 5, 2013 at 7:25 AM, Peter Eisentraut wrote: > On Thu, 2013-11-14 at 12:11 +0530, Amit Kapila wrote: >>If an application wants to allow these connection parameters to be >> used, it would need to do PQenableStartServer() first. If it doesn't, >> those connection parameters will be r

Re: [HACKERS] Proof of concept: standalone backend with full FE/BE protocol

2013-12-04 Thread Peter Eisentraut
On Thu, 2013-11-14 at 12:11 +0530, Amit Kapila wrote: >If an application wants to allow these connection parameters to be > used, it would need to do PQenableStartServer() first. If it doesn't, > those connection parameters will be rejected. Stupid idea: Would it work that we require an enviro

Re: [HACKERS] Proof of concept: standalone backend with full FE/BE protocol

2013-11-22 Thread Amit Kapila
On Thu, Nov 21, 2013 at 9:54 PM, Amit Kapila wrote: > On Thu, Nov 21, 2013 at 8:14 PM, Tom Lane wrote: >> Amit Kapila writes: >>> Here what I have in mind is that: >> >> Why would you make psql behave differently from our other command-line >> clients? > >No, psql should not behave different

Re: [HACKERS] Proof of concept: standalone backend with full FE/BE protocol

2013-11-21 Thread Amit Kapila
On Thu, Nov 21, 2013 at 8:14 PM, Tom Lane wrote: > Amit Kapila writes: >> Here what I have in mind is that: >> a. In pg_dump or other internal utilities where we want to use this >> feature, they should call PQenableStart() or some other API before >> calling PQConnect() which will indicate

Re: [HACKERS] Proof of concept: standalone backend with full FE/BE protocol

2013-11-21 Thread Amit Kapila
On Thu, Nov 21, 2013 at 9:11 AM, Amit Kapila wrote: > On Thu, Nov 21, 2013 at 2:14 AM, Tom Lane wrote: >> Peter Eisentraut writes: >>> The argument elsewhere in this thread was that the reason for putting >>> this in the connection options was so that you do *not* have to patch up >>> every clie

Re: [HACKERS] Proof of concept: standalone backend with full FE/BE protocol

2013-11-21 Thread Tom Lane
Amit Kapila writes: > Here what I have in mind is that: > a. In pg_dump or other internal utilities where we want to use this > feature, they should call PQenableStart() or some other API before > calling PQConnect() which will indicate that it wants to operate > as a standalone mode. > b. In

Re: [HACKERS] Proof of concept: standalone backend with full FE/BE protocol

2013-11-20 Thread Amit Kapila
On Thu, Nov 21, 2013 at 2:14 AM, Tom Lane wrote: > Peter Eisentraut writes: >> The argument elsewhere in this thread was that the reason for putting >> this in the connection options was so that you do *not* have to patch up >> every client to be able to use this functionality. If you have to ad

Re: [HACKERS] Proof of concept: standalone backend with full FE/BE protocol

2013-11-20 Thread Bruce Momjian
On Wed, Nov 20, 2013 at 05:38:14PM -0500, Gurjeet Singh wrote: > On Wed, Nov 20, 2013 at 3:44 PM, Tom Lane wrote: > > > To my mind, the "create a socket and hope nobody else can get to it" > approach is exactly one of the main things we're trying to avoid here. > If you'll recall, aw

Re: [HACKERS] Proof of concept: standalone backend with full FE/BE protocol

2013-11-20 Thread Andres Freund
On 2013-11-20 15:44:03 -0500, Tom Lane wrote: > In practice, as long as psql and pg_dump and pg_upgrade can do it, I > think we've covered most of the interesting bases. I'd say vacuumdb/reindexdb should be added to that list. In my experience xid wraparound and corrupted system indexes are the mo

Re: [HACKERS] Proof of concept: standalone backend with full FE/BE protocol

2013-11-20 Thread Gurjeet Singh
On Wed, Nov 20, 2013 at 3:44 PM, Tom Lane wrote: > > To my mind, the "create a socket and hope nobody else can get to it" > approach is exactly one of the main things we're trying to avoid here. > If you'll recall, awhile back we had a big discussion about how pg_upgrade > could positively guaran

Re: [HACKERS] Proof of concept: standalone backend with full FE/BE protocol

2013-11-20 Thread Tom Lane
Peter Eisentraut writes: > The argument elsewhere in this thread was that the reason for putting > this in the connection options was so that you do *not* have to patch up > every client to be able to use this functionality. If you have to add > separate options everywhere, then you might as well

Re: [HACKERS] Proof of concept: standalone backend with full FE/BE protocol

2013-11-20 Thread Robert Haas
On Wed, Nov 20, 2013 at 3:32 PM, Peter Eisentraut wrote: > On 11/20/13, 3:24 PM, Robert Haas wrote: >> The point is that client applications should expose whether or not to >> set this function as a command-line switch separate from whatever they >> accept in terms of connection strings. So pg_du

Re: [HACKERS] Proof of concept: standalone backend with full FE/BE protocol

2013-11-20 Thread Peter Eisentraut
On 11/20/13, 3:24 PM, Robert Haas wrote: > The point is that client applications should expose whether or not to > set this function as a command-line switch separate from whatever they > accept in terms of connection strings. So pg_dump should have a flag > called --standalone-server or something

Re: [HACKERS] Proof of concept: standalone backend with full FE/BE protocol

2013-11-20 Thread Robert Haas
On Wed, Nov 20, 2013 at 10:13 AM, Peter Eisentraut wrote: > On 11/14/13, 1:41 AM, Amit Kapila wrote: >>Security Concern >>- >>If a user can specify libpq connection options, he can now execute >> any file he wants by passing it as standalone_backend. >> >>

Re: [HACKERS] Proof of concept: standalone backend with full FE/BE protocol

2013-11-20 Thread Peter Eisentraut
On 11/20/13, 10:48 AM, Tom Lane wrote: > Perhaps more to the point, I think this approach actually breaks one of > the principal good-thing-in-emergencies attributes of standalone mode, > namely being sure that nobody but you can connect. With this, you're > right back to having a race condition a

Re: [HACKERS] Proof of concept: standalone backend with full FE/BE protocol

2013-11-20 Thread Peter Eisentraut
On 11/20/13, 11:31 AM, Stephen Frost wrote: > Couldn't that be an issue for people who have multiple major versions of > binaries installed? In particular, the "default" on the system for psql > might be 9.3 while the cluster you're trying to recover may be 9.2. Of > course, in that case you migh

Re: [HACKERS] Proof of concept: standalone backend with full FE/BE protocol

2013-11-20 Thread Tom Lane
Andres Freund writes: > On 2013-11-20 11:08:33 -0500, Tom Lane wrote: >> Andres Freund writes: >>> Something like PQstartSingleUser(dsn) returning a established connection >>> seems better to me. >> That just pushes the problem up a level --- how are you going to tell >> psql, pg_dump, or other

Re: [HACKERS] Proof of concept: standalone backend with full FE/BE protocol

2013-11-20 Thread Andres Freund
On 2013-11-20 17:19:42 +0100, Andres Freund wrote: > > That just pushes the problem up a level --- how are you going to tell > > psql, pg_dump, or other programs that they should do that? > > An explicit parameter. A program imo explicitly needs to be aware that a > PQconnect() suddenly starts for

Re: [HACKERS] Proof of concept: standalone backend with full FE/BE protocol

2013-11-20 Thread Stephen Frost
* Tom Lane (t...@sss.pgh.pa.us) wrote: > I think we'd be better off trying to fix the security issue by > constraining what can be executed as a "standalone backend". Would > it work to insist that psql/pg_dump launch the program named postgres > from the same bin directory they're in, rather than

Re: [HACKERS] Proof of concept: standalone backend with full FE/BE protocol

2013-11-20 Thread Andres Freund
On 2013-11-20 11:08:33 -0500, Tom Lane wrote: > Andres Freund writes: > > On 2013-11-20 10:48:20 -0500, Tom Lane wrote: > >> constraining what can be executed as a "standalone backend". Would > >> it work to insist that psql/pg_dump launch the program named postgres > >> from the same bin directo

Re: [HACKERS] Proof of concept: standalone backend with full FE/BE protocol

2013-11-20 Thread Tom Lane
Andres Freund writes: > On 2013-11-20 10:48:20 -0500, Tom Lane wrote: >> constraining what can be executed as a "standalone backend". Would >> it work to insist that psql/pg_dump launch the program named postgres >> from the same bin directory they're in, rather than accepting a path >> from the

Re: [HACKERS] Proof of concept: standalone backend with full FE/BE protocol

2013-11-20 Thread Andres Freund
On 2013-11-20 10:48:20 -0500, Tom Lane wrote: > constraining what can be executed as a "standalone backend". Would > it work to insist that psql/pg_dump launch the program named postgres > from the same bin directory they're in, rather than accepting a path > from the connection string? But why d

Re: [HACKERS] Proof of concept: standalone backend with full FE/BE protocol

2013-11-20 Thread Tom Lane
Peter Eisentraut writes: > I would consider sidestepping this entire issue by having the > stand-alone backend create a Unix-domain socket and have a client > connect to that in the normal way. Hmm. But that requires the "stand-alone backend" to take on at least some properties of a postmaster;

Re: [HACKERS] Proof of concept: standalone backend with full FE/BE protocol

2013-11-20 Thread Peter Eisentraut
On 11/14/13, 1:41 AM, Amit Kapila wrote: >Security Concern >- >If a user can specify libpq connection options, he can now execute > any file he wants by passing it as standalone_backend. > >Method to resolve Security concern >---

Re: [HACKERS] Proof of concept: standalone backend with full FE/BE protocol

2013-11-19 Thread Tom Lane
Robert Haas writes: > On Fri, Nov 15, 2013 at 6:51 AM, Simon Riggs wrote: >> Not enough. This feature is clearly being suggested as a way to offer >> Postgres in embedded mode for users by a back door. Doing that forces >> us to turn off many of the server's features and we will take a huge >> st

Re: [HACKERS] Proof of concept: standalone backend with full FE/BE protocol

2013-11-19 Thread Robert Haas
On Fri, Nov 15, 2013 at 6:51 AM, Simon Riggs wrote: > Not enough. This feature is clearly being suggested as a way to offer > Postgres in embedded mode for users by a back door. Doing that forces > us to turn off many of the server's features and we will take a huge > step backwards in features, t

Re: [HACKERS] Proof of concept: standalone backend with full FE/BE protocol

2013-11-16 Thread Amit Kapila
On Fri, Nov 15, 2013 at 5:21 PM, Simon Riggs wrote: > On 14 November 2013 03:41, Amit Kapila wrote: > >> I have gone through the mail chain of this thread and tried to find >> the different concerns or open ends for this patch. > > Not enough. This feature is clearly being suggested as a way to o

Re: [HACKERS] Proof of concept: standalone backend with full FE/BE protocol

2013-11-15 Thread Merlin Moncure
On Fri, Nov 15, 2013 at 6:06 AM, Dimitri Fontaine wrote: >> But: I very, very much agree with the other concerns around this. This >> should be a patch to fix single user mode, not one to make postgres into >> a single process database. It's not, and trying to make it by using >> single user mode

Re: [HACKERS] Proof of concept: standalone backend with full FE/BE protocol

2013-11-15 Thread Simon Riggs
On 15 November 2013 09:00, Andres Freund wrote: > This > should be a patch to fix single user mode, not one to make postgres into > a single process database. +1 -- Simon Riggs http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training & Services -- Sent

Re: [HACKERS] Proof of concept: standalone backend with full FE/BE protocol

2013-11-15 Thread Dimitri Fontaine
Andres Freund writes: > I think fixing single user mode to work halfway reasonable is enough > justification for the feature. Having to deal with that when solving > critical issues is just embarassing. +1 > But: I very, very much agree with the other concerns around this. This > should be a pat

Re: [HACKERS] Proof of concept: standalone backend with full FE/BE protocol

2013-11-15 Thread Andres Freund
On 2013-11-15 09:51:28 -0200, Simon Riggs wrote: > On 14 November 2013 03:41, Amit Kapila wrote: > > > I have gone through the mail chain of this thread and tried to find > > the different concerns or open ends for this patch. > > Not enough. This feature is clearly being suggested as a way to o

Re: [HACKERS] Proof of concept: standalone backend with full FE/BE protocol

2013-11-15 Thread Simon Riggs
On 14 November 2013 03:41, Amit Kapila wrote: > I have gone through the mail chain of this thread and tried to find > the different concerns or open ends for this patch. Not enough. This feature is clearly being suggested as a way to offer Postgres in embedded mode for users by a back door. Doin

Re: [HACKERS] Proof of concept: standalone backend with full FE/BE protocol

2013-11-13 Thread Tom Lane
Amit Kapila writes: > Any objections for adding this idea/patch to CF? You should certainly add it to the CF. You've listed lots of matters for review, but that's no reason to not get it in the queue to be reviewed. regards, tom lane -- Sent via pgsql-hackers mailing

Re: [HACKERS] Proof of concept: standalone backend with full FE/BE protocol

2013-11-13 Thread Amit Kapila
I have gone through the mail chain of this thread and tried to find the different concerns or open ends for this patch. Summarisation of the discussion and concerns for this patch: 1. Security concern in interface 2. Security concern in Windows implementation 3. Handling of Ctrl-C/SIGTERM 4. Seco

Re: [HACKERS] Proof of concept: standalone backend with full FE/BE protocol

2012-11-13 Thread Dimitri Fontaine
Tom Lane writes: >> I agree there is lots of demand for simple single-user databases and I >> wish that too. What I don't agree with is something that casts that >> requirement in stone by architecturally/permanently disallowing >> secondary connections. > > If you want secondary connections, then

Re: [HACKERS] Proof of concept: standalone backend with full FE/BE protocol

2012-11-13 Thread Christopher Browne
Preface: I think there's some great commentary here, and find myself agreeing pretty whole-heartedly. On Tue, Nov 13, 2012 at 2:45 PM, Simon Riggs wrote: > On 13 November 2012 17:38, Tom Lane wrote: > > Simon Riggs writes: > >> The most popular relational database in the world is Microsoft Acc

Re: [HACKERS] Proof of concept: standalone backend with full FE/BE protocol

2012-11-13 Thread Tom Lane
Simon Riggs writes: > On 13 November 2012 17:38, Tom Lane wrote: >> ... The fact of the matter is that there is *lots* of demand >> for simple single-user databases, and what I'm proposing is at least a >> first step towards getting there. > I agree there is lots of demand for simple single-use

Re: [HACKERS] Proof of concept: standalone backend with full FE/BE protocol

2012-11-13 Thread Simon Riggs
On 13 November 2012 17:38, Tom Lane wrote: > Simon Riggs writes: >> The most popular relational database in the world is Microsoft Access, >> not MySQL. Access appears desirable because it allows a single user to >> create and use a database (which is very good). But all business >> databases hav

Re: [HACKERS] Proof of concept: standalone backend with full FE/BE protocol

2012-11-13 Thread Robert Haas
On Tue, Nov 13, 2012 at 12:38 PM, Tom Lane wrote: > Simon Riggs writes: >> The most popular relational database in the world is Microsoft Access, >> not MySQL. Access appears desirable because it allows a single user to >> create and use a database (which is very good). But all business >> databa

Re: [HACKERS] Proof of concept: standalone backend with full FE/BE protocol

2012-11-13 Thread Tom Lane
Simon Riggs writes: > The most popular relational database in the world is Microsoft Access, > not MySQL. Access appears desirable because it allows a single user to > create and use a database (which is very good). But all business > databases have a requirement for at least one of: high availabi

Re: [HACKERS] Proof of concept: standalone backend with full FE/BE protocol

2012-11-13 Thread Simon Riggs
On 13 November 2012 13:05, Alvaro Herrera wrote: > Simon Riggs escribió: > >> > So even if this solution doesn't meet all requirements of single >> > process solution (and neither I think it is written to address all) >> > but can't we think of it as first version and then based on >> > requiremen

Re: [HACKERS] Proof of concept: standalone backend with full FE/BE protocol

2012-11-13 Thread Alvaro Herrera
Simon Riggs escribió: > > So even if this solution doesn't meet all requirements of single > > process solution (and neither I think it is written to address all) > > but can't we think of it as first version and then based on > > requirements extend it to have other capabilities: > > a. to have a

Re: [HACKERS] Proof of concept: standalone backend with full FE/BE protocol

2012-11-13 Thread Simon Riggs
On 13 November 2012 06:14, Amit kapila wrote: >>I get the installability thang, very very much, I just don't see the >>single process thing as the only solution. At very least an open >>minded analysis of the actual problem and ways of solving it is called >>for, not just reach for a close to han

Re: [HACKERS] Proof of concept: standalone backend with full FE/BE protocol

2012-11-13 Thread Andres Freund
On 2012-11-12 19:21:28 +, Simon Riggs wrote: > On 10 September 2012 17:50, Tom Lane wrote: > > > The point of the proposal that I am making is to have a simple, > > low-maintenance solution for people who need a single-application > > database. A compromise somewhere in the middle isn't like

Re: [HACKERS] Proof of concept: standalone backend with full FE/BE protocol

2012-11-12 Thread Amit kapila
On Tuesday, November 13, 2012 3:11 AM Simon Riggs wrote: On 12 November 2012 21:26, Merlin Moncure wrote: >> I couldn't disagree more. The patch is small, logical, and fixes an >> awful problem, namely that --single mode is basically unusable. As to >> your wider point (namely, that you can't c

Re: [HACKERS] Proof of concept: standalone backend with full FE/BE protocol

2012-11-12 Thread Amit kapila
On Monday, November 12, 2012 8:31 PM Merlin Moncure wrote: On Fri, Sep 14, 2012 at 6:42 AM, Amit kapila wrote: > On Tuesday, September 11, 2012 7:07 PM Amit Kapila wrote: > On Monday, September 10, 2012 8:20 PM Amit Kapila wrote: > On Sunday, September 09, 2012 1:37 PM Amit Kapila wrote: > On Frid

Re: [HACKERS] Proof of concept: standalone backend with full FE/BE protocol

2012-11-12 Thread Simon Riggs
On 12 November 2012 21:26, Merlin Moncure wrote: > I couldn't disagree more. The patch is small, logical, and fixes an > awful problem, namely that --single mode is basically unusable. As to > your wider point (namely, that you can't connect to it, therefore it's > bad), it has got to be refute

Re: [HACKERS] Proof of concept: standalone backend with full FE/BE protocol

2012-11-12 Thread Merlin Moncure
On Mon, Nov 12, 2012 at 1:21 PM, Simon Riggs wrote: > On 10 September 2012 17:50, Tom Lane wrote: > >> The point of the proposal that I am making is to have a simple, >> low-maintenance solution for people who need a single-application >> database. A compromise somewhere in the middle isn't like

Re: [HACKERS] Proof of concept: standalone backend with full FE/BE protocol

2012-11-12 Thread Simon Riggs
On 10 September 2012 17:50, Tom Lane wrote: > The point of the proposal that I am making is to have a simple, > low-maintenance solution for people who need a single-application > database. A compromise somewhere in the middle isn't likely to be an > improvement for anybody. For instance, if yo

Re: [HACKERS] Proof of concept: standalone backend with full FE/BE protocol

2012-11-12 Thread Merlin Moncure
On Fri, Sep 14, 2012 at 6:42 AM, Amit kapila wrote: > On Tuesday, September 11, 2012 7:07 PM Amit Kapila wrote: > On Monday, September 10, 2012 8:20 PM Amit Kapila wrote: > On Sunday, September 09, 2012 1:37 PM Amit Kapila wrote: > On Friday, September 07, 2012 11:19 PM Tom Lane wrote: > Heikki Li

Re: [HACKERS] Proof of concept: standalone backend with full FE/BE protocol

2012-09-14 Thread Amit kapila
On Tuesday, September 11, 2012 7:07 PM Amit Kapila wrote: On Monday, September 10, 2012 8:20 PM Amit Kapila wrote: On Sunday, September 09, 2012 1:37 PM Amit Kapila wrote: On Friday, September 07, 2012 11:19 PM Tom Lane wrote: Heikki Linnakangas writes: Would socketpair(2) be simpler?

Re: [HACKERS] Proof of concept: standalone backend with full FE/BE protocol

2012-09-11 Thread Amit kapila
On Monday, September 10, 2012 8:20 PM Amit Kapila wrote: On Sunday, September 09, 2012 1:37 PM Amit Kapila wrote: On Friday, September 07, 2012 11:19 PM Tom Lane wrote: Heikki Linnakangas writes: >>> Would socketpair(2) be simpler? >>>I've not done anything yet about the potential security issu

Re: [HACKERS] Proof of concept: standalone backend with full FE/BE protocol

2012-09-10 Thread Daniel Farina
On Mon, Sep 10, 2012 at 9:59 AM, Josh Berkus wrote: > >> The point of the proposal that I am making is to have a simple, >> low-maintenance solution for people who need a single-application >> database. A compromise somewhere in the middle isn't likely to be an >> improvement for anybody. For in

Re: [HACKERS] Proof of concept: standalone backend with full FE/BE protocol

2012-09-10 Thread Kevin Grittner
Josh Berkus wrote: > In fact, most of the folks who would want an embedded PostgreSQL > either want no authentication at all, or only a single password. > So supporting authentication options other than trust or md5 is > not required, or desired AFAIK. I don't know whether it's worth the trou

Re: [HACKERS] Proof of concept: standalone backend with full FE/BE protocol

2012-09-10 Thread Josh Berkus
> The point of the proposal that I am making is to have a simple, > low-maintenance solution for people who need a single-application > database. A compromise somewhere in the middle isn't likely to be an > improvement for anybody. For instance, if you want to have additional > connections, you

Re: [HACKERS] Proof of concept: standalone backend with full FE/BE protocol

2012-09-10 Thread Tom Lane
Gurjeet Singh writes: > On Mon, Sep 10, 2012 at 11:43 AM, Heikki Linnakangas wrote: >> [scratches head] How's that different from the normal postmaster mode? > As I described in later paragraphs, it'd behave like an embedded database, > like SQLite etc., so the database will startup and shutdown

Re: [HACKERS] Proof of concept: standalone backend with full FE/BE protocol

2012-09-10 Thread Gurjeet Singh
On Mon, Sep 10, 2012 at 11:43 AM, Heikki Linnakangas wrote: > On 10.09.2012 18:12, Gurjeet Singh wrote: > >> On Sun, Sep 2, 2012 at 8:23 PM, Tom Lane wrote: >> >> Notably, while the lack of any background processes is just what you want >>> for pg_upgrade and disaster recovery, an ordinary appli

Re: [HACKERS] Proof of concept: standalone backend with full FE/BE protocol

2012-09-10 Thread Heikki Linnakangas
On 10.09.2012 18:12, Gurjeet Singh wrote: On Sun, Sep 2, 2012 at 8:23 PM, Tom Lane wrote: Notably, while the lack of any background processes is just what you want for pg_upgrade and disaster recovery, an ordinary application is probably going to want to rely on autovacuum; and we need bgwrite

Re: [HACKERS] Proof of concept: standalone backend with full FE/BE protocol

2012-09-10 Thread Aidan Van Dyk
On Mon, Sep 10, 2012 at 11:12 AM, Gurjeet Singh wrote: > On Sun, Sep 2, 2012 at 8:23 PM, Tom Lane wrote: >> >> Notably, while the lack of any background processes is just what you want >> for pg_upgrade and disaster recovery, an ordinary application is probably >> going to want to rely on autovac

Re: [HACKERS] Proof of concept: standalone backend with full FE/BE protocol

2012-09-10 Thread Gurjeet Singh
On Sun, Sep 2, 2012 at 8:23 PM, Tom Lane wrote: > Notably, while the lack of any background processes is just what you want > for pg_upgrade and disaster recovery, an ordinary application is probably > going to want to rely on autovacuum; and we need bgwriter and other > background processes for

Re: [HACKERS] Proof of concept: standalone backend with full FE/BE protocol

2012-09-10 Thread Amit Kapila
On Sunday, September 09, 2012 1:37 PM Amit Kapila wrote: On Friday, September 07, 2012 11:19 PM Tom Lane wrote: Heikki Linnakangas writes: >>> Would socketpair(2) be simpler? >>I've not done anything yet about the potential security issues >>associated with untrusted libpq connection strings.

Re: [HACKERS] Proof of concept: standalone backend with full FE/BE protocol

2012-09-09 Thread Amit Kapila
On Sunday, September 09, 2012 8:46 PM Tom Lane wrote: Amit kapila writes: >> 1. does this follow the behavior that admin users will not be allowed to invoke postgres child process? > That's an interesting question. I'm not sure if we'd want to disable > the no-root check on the Unix side, but it

Re: [HACKERS] Proof of concept: standalone backend with full FE/BE protocol

2012-09-09 Thread Andres Freund
Hi, On Wednesday, September 05, 2012 06:00:18 PM Tom Lane wrote: > "anara...@anarazel.de" writes: > > I am not saying its bad that it is slower, that's absolutely OK. Just > > that it will take a variable amount of time till you can run pgdump > > again and its not easily detectable without loopi

Re: [HACKERS] Proof of concept: standalone backend with full FE/BE protocol

2012-09-09 Thread Tom Lane
Amit kapila writes: > 1. does this follow the behavior that admin users will not be allowed to > invoke postgres child process? That's an interesting question. I'm not sure if we'd want to disable the no-root check on the Unix side, but it might make sense to. But this has no bearing on what l

Re: [HACKERS] Proof of concept: standalone backend with full FE/BE protocol

2012-09-09 Thread Amit kapila
On Friday, September 07, 2012 11:19 PM Tom Lane wrote: Heikki Linnakangas writes: >> Would socketpair(2) be simpler? >I've not done anything yet about the potential security issues >associated with untrusted libpq connection strings. I think this >is still at the proof-of-concept stage; in pa

Re: [HACKERS] Proof of concept: standalone backend with full FE/BE protocol

2012-09-08 Thread Albert Cervera i Areny
A Dijous, 6 de setembre de 2012 00:30:53, Josh Berkus va escriure: > > In general I think the selling point for such a feature would be "no > > administrative hassles", and I believe that has to go not only for the > > end-user experience but also for the application-developer experience. > > If yo

Re: [HACKERS] Proof of concept: standalone backend with full FE/BE protocol

2012-09-07 Thread Gurjeet Singh
On Wed, Sep 5, 2012 at 10:29 PM, Andrew Dunstan wrote: > > On 09/05/2012 10:14 PM, Tom Lane wrote: > >> >> The people who would be interested in this are currently using something >> like SQLite within a single application program. >> > > Exactly. I think it's worth stating that this has a HUGE

Re: [HACKERS] Proof of concept: standalone backend with full FE/BE protocol

2012-09-07 Thread Jim Nasby
On 9/2/12 7:23 PM, Tom Lane wrote: 4. As coded, the backend assumes the incoming pipe is on its FD 0 and the outgoing pipe is on its FD 1. This made the command line simple but I'm having second thoughts about it: if anything inside the backend tries to read stdin or write stdout, unpleasant thi

Re: [HACKERS] Proof of concept: standalone backend with full FE/BE protocol

2012-09-07 Thread Andres Freund
On Friday, September 07, 2012 11:21:00 PM Merlin Moncure wrote: > On Thu, Sep 6, 2012 at 12:56 PM, Jeff Davis wrote: > > On Wed, 2012-09-05 at 17:03 -0400, Tom Lane wrote: > >> In general I think the selling point for such a feature would be "no > >> administrative hassles", and I believe that has

Re: [HACKERS] Proof of concept: standalone backend with full FE/BE protocol

2012-09-07 Thread Merlin Moncure
On Thu, Sep 6, 2012 at 12:56 PM, Jeff Davis wrote: > On Wed, 2012-09-05 at 17:03 -0400, Tom Lane wrote: >> In general I think the selling point for such a feature would be "no >> administrative hassles", and I believe that has to go not only for the >> end-user experience but also for the applicat

Re: [HACKERS] Proof of concept: standalone backend with full FE/BE protocol

2012-09-07 Thread Tom Lane
Heikki Linnakangas writes: > On 07.09.2012 10:49, Tom Lane wrote: >> I'm a bit tempted though to pull out and apply the portions of the >> patch that replace libpq's assorted ad-hoc closesocket() calls with >> a centralized pqDropConnection routine. I think that's probably a good >> idea independ

Re: [HACKERS] Proof of concept: standalone backend with full FE/BE protocol

2012-09-07 Thread Tom Lane
Heikki Linnakangas writes: > It's worth noting that now that libpq constructs the command line to > execute "postgres --child= -D ", we'll be stuck with that set > of arguments forever, because libpq needs to be able to talk to > different versions. Or at least we'd need to teach libpq to check

Re: [HACKERS] Proof of concept: standalone backend with full FE/BE protocol

2012-09-07 Thread Heikki Linnakangas
On 07.09.2012 10:49, Tom Lane wrote: Heikki Linnakangas writes: Would socketpair(2) be simpler? Attached is a revised version of the patch that uses socketpair(2). This is definitely a lot less invasive --- the backend side of the patch, in particular, is far shorter, and there are fewer port

Re: [HACKERS] Proof of concept: standalone backend with full FE/BE protocol

2012-09-07 Thread Tom Lane
Heikki Linnakangas writes: > Would socketpair(2) be simpler? Attached is a revised version of the patch that uses socketpair(2). This is definitely a lot less invasive --- the backend side of the patch, in particular, is far shorter, and there are fewer portability hazards since we're not trying

Re: [HACKERS] Proof of concept: standalone backend with full FE/BE protocol

2012-09-06 Thread Jeff Davis
On Wed, 2012-09-05 at 17:03 -0400, Tom Lane wrote: > In general I think the selling point for such a feature would be "no > administrative hassles", and I believe that has to go not only for the > end-user experience but also for the application-developer experience. > If you have to manage checkpo

Re: [HACKERS] Proof of concept: standalone backend with full FE/BE protocol

2012-09-05 Thread Daniel Farina
On Wed, Sep 5, 2012 at 7:14 PM, Tom Lane wrote: > This seems to me to be going in exactly the wrong direction. What > I visualize this feature as responding to is demand for a *simple*, > minimal configuration, minimal administration, quasi-embedded database. > What you propose above is not that,

Re: [HACKERS] Proof of concept: standalone backend with full FE/BE protocol

2012-09-05 Thread Andrew Dunstan
On 09/05/2012 10:14 PM, Tom Lane wrote: Aidan Van Dyk writes: So, in the spirit of not painting ourselves into a tiny corner here on the whole "single backend" and "embedded database" problem with pg options, can we generalize this a bit? Any way we could make psql connect to a "given fd", as

Re: [HACKERS] Proof of concept: standalone backend with full FE/BE protocol

2012-09-05 Thread Tom Lane
Aidan Van Dyk writes: > So, in the spirit of not painting ourselves into a tiny corner here on > the whole "single backend" and "embedded database" problem with pg > options, can we generalize this a bit? > Any way we could make psql connect to a "given fd", as an option? In > theory, that could

Re: [HACKERS] Proof of concept: standalone backend with full FE/BE protocol

2012-09-05 Thread Aidan Van Dyk
So, in the spirit of not painting ourselves into a tiny corner here on the whole "single backend" and "embedded database" problem with pg options, can we generalize this a bit? Any way we could make psql connect to a "given fd", as an option? In theory, that could be something opened by some out-

Re: [HACKERS] Proof of concept: standalone backend with full FE/BE protocol

2012-09-05 Thread Daniel Farina
On Wed, Sep 5, 2012 at 3:17 PM, Peter Eisentraut wrote: > On 9/5/12 5:59 PM, Daniel Farina wrote: >> I agree with this, even though in theory (but not in practice) >> creative use of unix sockets (sorry windows, perhaps some >> port-allocating and URL mangling can be done instead) and conventions

Re: [HACKERS] Proof of concept: standalone backend with full FE/BE protocol

2012-09-05 Thread Josh Berkus
> Um ... true with respect to autovacuum, perhaps, but what about > checkpoints? A standalone backend will never perform a checkpoint > unless explicitly told to. Hmmm, that's definitely an issue. > (Before we invented the bgwriter, the > postmaster was in charge of launching checkpoints every

Re: [HACKERS] Proof of concept: standalone backend with full FE/BE protocol

2012-09-05 Thread Josh Berkus
On 9/5/12 2:50 PM, Peter Eisentraut wrote: > On 9/5/12 5:03 PM, Tom Lane wrote: >> I don't see people wanting to use this feature for unit tests. > > If this is going to become an official feature (as opposed to an > internal interface only for use by pg_upgrade), then I think that's > exactly wha

Re: [HACKERS] Proof of concept: standalone backend with full FE/BE protocol

2012-09-05 Thread Peter Eisentraut
On 9/5/12 5:59 PM, Daniel Farina wrote: > I agree with this, even though in theory (but not in practice) > creative use of unix sockets (sorry windows, perhaps some > port-allocating and URL mangling can be done instead) and conventions > for those would allow even better almost-like-embedded resul

Re: [HACKERS] Proof of concept: standalone backend with full FE/BE protocol

2012-09-05 Thread Daniel Farina
On Wed, Sep 5, 2012 at 2:50 PM, Peter Eisentraut wrote: > On 9/5/12 5:03 PM, Tom Lane wrote: >> I don't see people wanting to use this feature for unit tests. > > If this is going to become an official feature (as opposed to an > internal interface only for use by pg_upgrade), then I think that's

Re: [HACKERS] Proof of concept: standalone backend with full FE/BE protocol

2012-09-05 Thread Peter Eisentraut
On 9/5/12 5:03 PM, Tom Lane wrote: > I don't see people wanting to use this feature for unit tests. If this is going to become an official feature (as opposed to an internal interface only for use by pg_upgrade), then I think that's exactly what people will want to use it for. In fact, it might e

  1   2   >