Re: [HACKERS] WAL file location

2002-08-03 Thread Curt Sampson
On Fri, 2 Aug 2002, Thomas Lockhart wrote: > [Symlinks] don't scale, Given that we have only one directory for the log file, this would not appear to be a problem. > they are not portable, That's certainly a problem if we intend to run on systems without them. > and it is difficult for > appl

Re: [HACKERS] WAL file location

2002-08-02 Thread Bruce Momjian
Thomas Lockhart wrote: > ... > > I was just wondering why we would deal with environment variables or > > postgresql.conf settings. Just make it an initdb flag, create it in the > > desired location with a symlink in /data and then we don't have to do > > any more work for WAL locations unless pe

Re: [HACKERS] WAL file location

2002-08-02 Thread Thomas Lockhart
... > I was just wondering why we would deal with environment variables or > postgresql.conf settings. Just make it an initdb flag, create it in the > desired location with a symlink in /data and then we don't have to do > any more work for WAL locations unless people want to move it around > aft

Re: [HACKERS] WAL file location

2002-08-02 Thread Bruce Momjian
Thomas Lockhart wrote: > > > I am wondering why we even want to specify the WAL location anywhere > > > except as a flag to initdb. If you specify a location at initdb time, > > > it creates the /xlog directory, then symlinks it into /data. > > Does this have any negative implications for Win32 p

Re: [HACKERS] WAL file location

2002-08-02 Thread Greg Copeland
On Fri, 2002-08-02 at 13:46, Thomas Lockhart wrote: > > > I am wondering why we even want to specify the WAL location anywhere > > > except as a flag to initdb. If you specify a location at initdb time, > > > it creates the /xlog directory, then symlinks it into /data. > > Does this have any nega

Re: [HACKERS] WAL file location

2002-08-02 Thread scott.marlowe
On Fri, 2 Aug 2002, Thomas Lockhart wrote: > > > I am wondering why we even want to specify the WAL location anywhere > > > except as a flag to initdb. If you specify a location at initdb time, > > > it creates the /xlog directory, then symlinks it into /data. > > Does this have any negative imp

Re: [HACKERS] WAL file location

2002-08-02 Thread Thomas Lockhart
> > I am wondering why we even want to specify the WAL location anywhere > > except as a flag to initdb. If you specify a location at initdb time, > > it creates the /xlog directory, then symlinks it into /data. > Does this have any negative implications for Win32 ports? Sure. the symlinks thing

Re: [HACKERS] WAL file location

2002-08-02 Thread Greg Copeland
On Wed, 2002-07-31 at 22:20, Bruce Momjian wrote: > I am wondering why we even want to specify the WAL location anywhere > except as a flag to initdb. If you specify a location at initdb time, > it creates the /xlog directory, then symlinks it into /data. > Does this have any negative implicati

Re: [HACKERS] WAL file location

2002-08-01 Thread Andrew Sullivan
On Wed, Jul 31, 2002 at 11:20:35PM -0400, Bruce Momjian wrote: > > I am wondering why we even want to specify the WAL location anywhere > except as a flag to initdb. If you specify a location at initdb time, > it creates the /xlog directory, then symlinks it into /data. I thought the whole poin

Re: [HACKERS] WAL file location

2002-07-31 Thread Bruce Momjian
Curt Sampson wrote: > On Wed, 31 Jul 2002, Andrew Sullivan wrote: > > > Ok, how then would one set the location of the config file? > > Option on the command line. Works for lots of different servers out > there already (BIND, apache, etc.). > > Whether we also want to emulate them using a comp

Re: [HACKERS] WAL file location

2002-07-31 Thread Curt Sampson
On Wed, 31 Jul 2002, Peter Eisentraut wrote: > But the location of the WAL logs, and the commit logs, if anyone is > thinking in that direction, needs to be known to initdb. And if you want > to move them later you'd need to halt the entire system I don't see this as a big problem. Right no

Re: [HACKERS] WAL file location

2002-07-31 Thread Curt Sampson
On Wed, 31 Jul 2002, Andrew Sullivan wrote: > Ok, how then would one set the location of the config file? Option on the command line. Works for lots of different servers out there already (BIND, apache, etc.). Whether we also want to emulate them using a compiled-in default if the command line

Re: [HACKERS] WAL file location

2002-07-31 Thread Peter Eisentraut
Bruce Momjian writes: > Thomas, are you going to extend this to locations for any table/index? > Seems whatever we do for WAL should fix in that scheme. The trick is that it might not. For relations you simply need a system table mapping location names to file system locations, and you can add

Re: [HACKERS] WAL file location

2002-07-31 Thread Andrew Sullivan
On Wed, Jul 31, 2002 at 11:09:58AM -0400, Tom Lane wrote: > Andrew Sullivan <[EMAIL PROTECTED]> writes: > > Ok, how then would one set the location of the config file? > > The config file itself has to be found the same way we do it now, > obviously: either a command line argument or the environm

Re: [HACKERS] WAL file location

2002-07-31 Thread Tom Lane
Andrew Sullivan <[EMAIL PROTECTED]> writes: > Ok, how then would one set the location of the config file? The config file itself has to be found the same way we do it now, obviously: either a command line argument or the environment variable $PGDATA. But that's a red herring. This thread is not

Re: [HACKERS] WAL file location

2002-07-31 Thread Andrew Sullivan
On Wed, Jul 31, 2002 at 10:23:07AM -0400, Tom Lane wrote: > Andrew Sullivan <[EMAIL PROTECTED]> writes: > > a. The system uses no environment variables at all; some other > > method is used to determine where the config file is (maybe compiled > > into the code); > > > If I understand it, nobody

Re: [HACKERS] WAL file location

2002-07-31 Thread Tom Lane
Andrew Sullivan <[EMAIL PROTECTED]> writes: > a.The system uses no environment variables at all; some other > method is used to determine where the config file is (maybe compiled > into the code); > If I understand it, nobody is really arguing for (a). I am. I see absolutely no advantage in

Re: [HACKERS] WAL file location

2002-07-31 Thread Thomas Lockhart
... > Is this a fair account? Yes. You may note that we have not explored the implementation details on any of these, so the attributes of each are not cast in stone (except for the purposes of argument of course ;) - Thomas ---(end of broadcast)-

Re: [HACKERS] WAL file location

2002-07-31 Thread Andrew Sullivan
On Wed, Jul 31, 2002 at 02:00:52AM -0400, Tom Lane wrote: > let alone how to configure it so that it works reliably. I really > fail to understand why you want to drive this new feature off environment > variables. You say you've "pointed out the utility and desirability" > of doing it that way,

Re: [HACKERS] WAL file location

2002-07-30 Thread Thomas Lockhart
... > But ... my recollection is that we've had a *huge* number of complaints > about the initlocation behavior, at least by comparison to the number > of people using the feature. No one can understand how it works, > let alone how to configure it so that it works reliably. I really > fail to u

Re: [HACKERS] WAL file location

2002-07-30 Thread Curt Sampson
On Tue, 30 Jul 2002, Thomas Lockhart wrote: > The "no envar" camp has not thought through the issues yet, though the > issues can be found in the threads. Better to decide what the > requirements are before throwing out the solution. Ok, so what issues has the "no envvar" camp not yet dealt with

Re: [HACKERS] WAL file location

2002-07-30 Thread Thomas Lockhart
... > Agreed. Consistency argues for the postgresql.conf solution, not > security. Also, I would like to see initlocation removed as soon as we > get a 100% functional replacement. We have fielded too many questions > about how to set it up. Hmm. I'm not sure the best way to look, but I was ab

Re: [HACKERS] WAL file location

2002-07-30 Thread Tom Lane
Thomas Lockhart <[EMAIL PROTECTED]> writes: > ... The behavior of initlocation has > been absolutely no burden on -hackers for the nearly *5 years* that it > has been available, and that is the best evidence that we're just > talking through hats. Let's get on with it, or at least get back to > be

Re: [HACKERS] WAL file location

2002-07-30 Thread Thomas Lockhart
> Whether you think that there is a potentially-exploitable security hole > here is not really the issue. The point is that two different arguments > have been advanced against using environment variables for configuration > (if you weren't counting, (1) possible security issues now or in the > f

Re: [HACKERS] WAL file location

2002-07-30 Thread Curt Sampson
On Wed, 31 Jul 2002, Lamar Owen wrote: > On Tuesday 30 July 2002 11:51 pm, Tom Lane wrote: > > Lamar Owen <[EMAIL PROTECTED]> writes: > > >> CREATE DATABASE foo WITH LOCATION = 'BAR' > > > And requires you to be a database superuser anyway. > > > CREATE DATABASE does not require superuser privs,

Re: [HACKERS] WAL file location

2002-07-30 Thread Curt Sampson
On Tue, 30 Jul 2002, Lamar Owen wrote: > On Tuesday 30 July 2002 07:46 pm, Curt Sampson wrote: > > > Ah. See, we already have a failure in a security analysis here. This > > command: > > > CREATE DATABASE foo WITH LOCATION = 'BAR' > > > uses a string that's in the environment. > > And require

Re: [HACKERS] WAL file location

2002-07-30 Thread Bruce Momjian
Lamar Owen wrote: > On Tuesday 30 July 2002 11:51 pm, Tom Lane wrote: > > Lamar Owen <[EMAIL PROTECTED]> writes: > > >> CREATE DATABASE foo WITH LOCATION = 'BAR' > > > And requires you to be a database superuser anyway. > > > CREATE DATABASE does not require superuser privs, only createdb > > whi

Re: [HACKERS] WAL file location

2002-07-30 Thread Lamar Owen
On Tuesday 30 July 2002 11:51 pm, Tom Lane wrote: > Lamar Owen <[EMAIL PROTECTED]> writes: > >> CREATE DATABASE foo WITH LOCATION = 'BAR' > > And requires you to be a database superuser anyway. > CREATE DATABASE does not require superuser privs, only createdb > which is not usually considered par

Re: [HACKERS] WAL file location

2002-07-30 Thread Tom Lane
Lamar Owen <[EMAIL PROTECTED]> writes: >> Ah. See, we already have a failure in a security analysis here. This >> command: >> CREATE DATABASE foo WITH LOCATION = 'BAR' >> uses a string that's in the environment. > And requires you to be a database superuser anyway. CREATE DATABASE does not requi

Re: [HACKERS] WAL file location

2002-07-30 Thread Lamar Owen
On Tuesday 30 July 2002 07:46 pm, Curt Sampson wrote: > On Tue, 30 Jul 2002, Lamar Owen wrote: > > I said it. In any case, using strings that are in the environment > > requires an untrusted PL, or a C function. > Ah. See, we already have a failure in a security analysis here. This > command: >

Re: [HACKERS] WAL file location

2002-07-30 Thread Curt Sampson
On Tue, 30 Jul 2002, Thomas Lockhart wrote: > But the zeroth-order issue is not security. It is storage management for > large databases. Any scheme we have for accomplishing that must hold up > to scrutiny, but we can not refuse to proceed just because there are > "lions tigers and bears" out th

Re: [HACKERS] WAL file location

2002-07-30 Thread Thomas Lockhart
... > I've been securing systems since I started an ISP in 1995, and so I've > seen a lot of security vulnerabilities come and go, and I've got a bit > of a feel for what kinds of things are typically exploited. And this one > one just screams, "potential security vulnerability!" to me. Sure, the

Re: [HACKERS] WAL file location

2002-07-30 Thread Curt Sampson
On Tue, 30 Jul 2002, Lamar Owen wrote: > On Tuesday 30 July 2002 02:34 pm, Tom Lane wrote: > > > Who said anything about poisoning the environment? My point was that > > there will be strings in the environment that were put there perfectly > > legitimately, but could still serve as an attack ve

Re: [HACKERS] WAL file location

2002-07-30 Thread Curt Sampson
On Tue, 30 Jul 2002, Lamar Owen wrote: > Now, let me make the statement that the environment in this case is > not likely to be a security issue any worse than having the stuff > in postgresql.conf, as any attacker that can poison the postmaster > environment can probably poison postgresql.conf.

Re: [HACKERS] WAL file location

2002-07-30 Thread Thomas Lockhart
... > Thomas, are you going to extend this to locations for any table/index? > Seems whatever we do for WAL should fix in that scheme. Yes, the longer-term goal is enabling table/index-specific locations. I'm not certain whether WAL can use *exactly* the same mechanism, since 1) the location for

Re: [HACKERS] WAL file location

2002-07-30 Thread Tom Lane
Thomas Lockhart <[EMAIL PROTECTED]> writes: >> If we add more environment-variable-dependent mechanisms to allow more >> different things to be done, we increase substantially the odds of >> creating an exploitable security hole. > No. See above. Your argument seems to reduce to "it's not insecu

Re: [HACKERS] WAL file location

2002-07-30 Thread Peter Eisentraut
Thomas Lockhart writes: > I've developed patches to be able to specify the location of the WAL > directory, with the default location being where it is now. The patches > define a new environment variable PGXLOG (a la PGDATA) and postmaster, > postgres, initdb and pg_ctl have been taught to recog

Re: [HACKERS] WAL file location

2002-07-30 Thread Bruce Momjian
Tom Lane wrote: > Lamar Owen <[EMAIL PROTECTED]> writes: > > Having said all that, I still believe that this is something tailor-made for > > postgresql.conf. > > Well, exactly. Regardless of how serious you may think the security > argument is, it still remains that a config-file entry seems t

Re: [HACKERS] WAL file location

2002-07-30 Thread Tom Lane
Lamar Owen <[EMAIL PROTECTED]> writes: > Having said all that, I still believe that this is something tailor-made for > postgresql.conf. Well, exactly. Regardless of how serious you may think the security argument is, it still remains that a config-file entry seems the ideal way to do it. I ca

Re: [HACKERS] WAL file location

2002-07-30 Thread Lamar Owen
On Tuesday 30 July 2002 02:34 pm, Tom Lane wrote: > Andrew Sullivan <[EMAIL PROTECTED]> writes: > > On Tue, Jul 30, 2002 at 02:05:57PM -0400, Tom Lane wrote: > >> If we add more environment-variable-dependent mechanisms to allow more > >> different things to be done, we increase substantially the

Re: [HACKERS] WAL file location

2002-07-30 Thread Bruce Momjian
In my logic, we have PGDATA environment variable for the server only so the server can find the /data directory. After that, everything should be in /data. I see no reason to make it an environment variable. In fact, a file in /data should be able to track the xlog directory a lot better than

Re: [HACKERS] WAL file location

2002-07-30 Thread Tom Lane
Andrew Sullivan <[EMAIL PROTECTED]> writes: > On Tue, Jul 30, 2002 at 02:05:57PM -0400, Tom Lane wrote: >> If we add more environment-variable-dependent mechanisms to allow more >> different things to be done, we increase substantially the odds of >> creating an exploitable security hole. > Ok, t

Re: [HACKERS] WAL file location

2002-07-30 Thread Andrew Sullivan
On Tue, Jul 30, 2002 at 02:05:57PM -0400, Tom Lane wrote: > > If we add more environment-variable-dependent mechanisms to allow more > different things to be done, we increase substantially the odds of > creating an exploitable security hole. Ok, true enough, but I'm not sure that a config file

Re: [HACKERS] WAL file location

2002-07-30 Thread Tom Lane
Andrew Sullivan <[EMAIL PROTECTED]> writes: > I guess I'm dumb, but I'm not seeing how these environment variables > are a big security risk. The trouble with relying on environment variables for paths (especially paths to places that we might scribble on) is that the postmaster has no idea which

Re: [HACKERS] WAL file location

2002-07-30 Thread Lamar Owen
On Tuesday 30 July 2002 07:10 am, Curt Sampson wrote: > BTW, you mention in another message that environment variables work > well for you. Well, they are a security problem waiting to happen, > IMHO. Do you have any objections to having a file containing a list > of the various data directories?

Re: [HACKERS] WAL file location

2002-07-30 Thread Bruce Momjian
Andrew Sullivan wrote: > On Tue, Jul 30, 2002 at 08:10:02PM +0900, Curt Sampson wrote: > > > BTW, you mention in another message that environment variables work > > well for you. Well, they are a security problem waiting to happen, > > IMHO. Do you have any objections to having a file containing

Re: [HACKERS] WAL file location

2002-07-30 Thread Andrew Sullivan
On Tue, Jul 30, 2002 at 08:10:02PM +0900, Curt Sampson wrote: > BTW, you mention in another message that environment variables work > well for you. Well, they are a security problem waiting to happen, > IMHO. Do you have any objections to having a file containing a list > of the various data dire

Re: [HACKERS] WAL file location

2002-07-30 Thread Curt Sampson
On Mon, 29 Jul 2002, Thomas Lockhart wrote: > It is supported by the installation environment, and does not require > the explicit three steps of > > 1) creating a new directory area > 2) moving files to the new area > 3) creating a symlink to point to the new area So basically it gives you the

Re: [HACKERS] WAL file location

2002-07-29 Thread Thomas Lockhart
> > I've developed patches to be able to specify the location of the WAL > > directory, with the default location being where it is now. The patches > > define a new environment variable PGXLOG (a la PGDATA) and postmaster, > > postgres, initdb and pg_ctl have been taught to recognize a new comman

Re: [HACKERS] WAL file location

2002-07-29 Thread Thomas Lockhart
> > I've developed patches to be able to specify the location of the WAL > > directory, with the default location being where it is now. The patches > > define a new environment variable PGXLOG (a la PGDATA) and postmaster, > > postgres, initdb and pg_ctl have been taught to recognize a new comman

Re: [HACKERS] WAL file location

2002-07-29 Thread Bruce Momjian
Tom Lane wrote: > Thomas Lockhart <[EMAIL PROTECTED]> writes: > > I've developed patches to be able to specify the location of the WAL > > directory, with the default location being where it is now. The patches > > define a new environment variable PGXLOG (a la PGDATA) and postmaster, > > postgres

Re: [HACKERS] WAL file location

2002-07-29 Thread Curt Sampson
On Tue, 30 Jul 2002, Tom Lane wrote: > More generally, I do not like depending on postmaster environment > variables --- our experience with environment variables for database > locations has been uniformly bad > The existing secondary-location mechanism is horrible. Please do not > emulate

Re: [HACKERS] WAL file location

2002-07-29 Thread Curt Sampson
On Mon, 29 Jul 2002, Thomas Lockhart wrote: > I've developed patches to be able to specify the location of the WAL > directory, with the default location being where it is now. The patches > define a new environment variable PGXLOG (a la PGDATA) and postmaster, > postgres, initdb and pg_ctl have

Re: [HACKERS] WAL file location

2002-07-29 Thread Tom Lane
Thomas Lockhart <[EMAIL PROTECTED]> writes: > I've developed patches to be able to specify the location of the WAL > directory, with the default location being where it is now. The patches > define a new environment variable PGXLOG (a la PGDATA) and postmaster, > postgres, initdb and pg_ctl have b

[HACKERS] WAL file location

2002-07-29 Thread Thomas Lockhart
I've developed patches to be able to specify the location of the WAL directory, with the default location being where it is now. The patches define a new environment variable PGXLOG (a la PGDATA) and postmaster, postgres, initdb and pg_ctl have been taught to recognize a new command line switch "-