Re: [HACKERS] [COMMITTERS] pgsql-server/src backend/tcop/postgres.cbacke
Thomas Lockhart <[EMAIL PROTECTED]> writes: > Sorry to hear that this is the way it turned out. It is a bad precedent > imho, and I see no way forward on my interest in this area. Hopefully > someone else will pick it up; perhaps one of those so vehemently against > the details of this? I said I would be willing to make initdb create a symlink given a -X switch; if you don't want to pick it up then I will do that. regards, tom lane ---(end of broadcast)--- TIP 1: subscribe and unsubscribe commands go to [EMAIL PROTECTED]
Re: [HACKERS] [COMMITTERS] pgsql-server/src backend/tcop/postgres.cbacke
Yes, perhaps a bad precedent. I have very rarely done this. I do have the patch here if anyone wants to use it. My guess is if someone implements it, it will be done only in initdb, and use symlinks, which you and Marc don't like, so we may be best leaving it undone for 7.3 and returning with a clear slate in 7.4. --- Thomas Lockhart wrote: > > OK, with two people now asking to have the patch removed, and with no > > comment from Thomas, I have removed the patch. This removes XLogDir > > environment variable, and -X postmaster/postgres/initdb/pg_ctl flag. > > I have also removed the code that dynamically sized xlogdir. > > ... Back in town... > > Sorry to hear that this is the way it turned out. It is a bad precedent > imho, and I see no way forward on my interest in this area. Hopefully > someone else will pick it up; perhaps one of those so vehemently against > the details of this? > > - Thomas > -- Bruce Momjian| http://candle.pha.pa.us [EMAIL PROTECTED] | (610) 359-1001 + If your life is a hard drive, | 13 Roberts Road + Christ can be your backup.| Newtown Square, Pennsylvania 19073 ---(end of broadcast)--- TIP 3: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [HACKERS] [COMMITTERS] pgsql-server/src backend/tcop/postgres.cbacke
> OK, with two people now asking to have the patch removed, and with no > comment from Thomas, I have removed the patch. This removes XLogDir > environment variable, and -X postmaster/postgres/initdb/pg_ctl flag. > I have also removed the code that dynamically sized xlogdir. ... Back in town... Sorry to hear that this is the way it turned out. It is a bad precedent imho, and I see no way forward on my interest in this area. Hopefully someone else will pick it up; perhaps one of those so vehemently against the details of this? - Thomas ---(end of broadcast)--- TIP 6: Have you searched our list archives? http://archives.postgresql.org
Re: [HACKERS] [COMMITTERS] pgsql-server/src backend/tcop/postgres.cbacke
Tom Lane wrote: > Bruce Momjian <[EMAIL PROTECTED]> writes: > > OK, with two people now asking to have the patch removed, and with no > > comment from Thomas, I have removed the patch. This removes XLogDir > > environment variable, and -X postmaster/postgres/initdb/pg_ctl flag. > > I thought we intended to keep the -X switch for initdb (only), and have > it make a symlink. The majority of the patch wasn't needed, so rather than muck it up, I just backed it all out. If we want that, and I think we do, someone should implent it as a separate patch that people can review. All the work is going to be done in initdb anyway. -- Bruce Momjian| http://candle.pha.pa.us [EMAIL PROTECTED] | (610) 359-1001 + If your life is a hard drive, | 13 Roberts Road + Christ can be your backup.| Newtown Square, Pennsylvania 19073 ---(end of broadcast)--- TIP 2: you can get off all lists at once with the unregister command (send "unregister YourEmailAddressHere" to [EMAIL PROTECTED])
Re: [HACKERS] [COMMITTERS] pgsql-server/src backend/tcop/postgres.cbacke
Bruce Momjian <[EMAIL PROTECTED]> writes: > OK, with two people now asking to have the patch removed, and with no > comment from Thomas, I have removed the patch. This removes XLogDir > environment variable, and -X postmaster/postgres/initdb/pg_ctl flag. I thought we intended to keep the -X switch for initdb (only), and have it make a symlink. regards, tom lane ---(end of broadcast)--- TIP 5: Have you checked our extensive FAQ? http://www.postgresql.org/users-lounge/docs/faq.html
Re: [HACKERS] [COMMITTERS] pgsql-server/src backend/tcop/postgres.cbacke
OK, with two people now asking to have the patch removed, and with no comment from Thomas, I have removed the patch. This removes XLogDir environment variable, and -X postmaster/postgres/initdb/pg_ctl flag. I have also removed the code that dynamically sized xlogdir. I will post the patch to patches, and keep the patch here in case it is needed later. --- Curt Sampson wrote: > On Fri, 16 Aug 2002, Bruce Momjian wrote: > > > Part of the reason we can't "just continue to use the symlink method" is > > that the PGXLOG environment variable situation is currently in CVS > > beyond initdb and in postmaster, postgres, and pg_ctl, so we do have to > > do something before 7.3 or we will have new environment variable > > handling in all those commands, and I don't think we have agreement on > > that. > > Well, let's take it out, then, and use the symlink instead. It may > be in CVS now, but it's never been in a release, so there should > be no problem with removing it. > > I think we've got some pretty strong opinions here that distributing > configuration information amongst multiple environment variables > is a Bad Idea. > > cjs > -- > Curt Sampson <[EMAIL PROTECTED]> +81 90 7737 2974 http://www.netbsd.org > Don't you know, in this new Dark Age, we're all light. --XTC > > > ---(end of broadcast)--- > TIP 6: Have you searched our list archives? > > http://archives.postgresql.org > -- Bruce Momjian| http://candle.pha.pa.us [EMAIL PROTECTED] | (610) 359-1001 + If your life is a hard drive, | 13 Roberts Road + Christ can be your backup.| Newtown Square, Pennsylvania 19073 ---(end of broadcast)--- TIP 2: you can get off all lists at once with the unregister command (send "unregister YourEmailAddressHere" to [EMAIL PROTECTED])
Re: [HACKERS] [COMMITTERS] pgsql-server/src backend/tcop/postgres.cbacke
Curt Sampson wrote: > On Thu, 15 Aug 2002, Bruce Momjian wrote: > > > I would like to know how to move this item forward. > > Right now (i.e., in 7.2), the only two options we have for moving the > log file to a different spindle are mounting it on pg_xlog and using a > symlink. I doubt many people do the the former, and if they do they do > not need an option to init_db to move the logfile away from its default > location. > > So I propose we just continue to use the symlink method for the moment, > until we agree on another way to store the log file location within the > data directory, and at that time we implement the code to do that. > > Note that if we don't move forward at all, we're still left in the symlink > situation, with the exception that you init_db, move the log directory and > create the symlink by hand, and then start up the database. So this partial > move forward makes no difference to the symlink argument. Part of the reason we can't "just continue to use the symlink method" is that the PGXLOG environment variable situation is currently in CVS beyond initdb and in postmaster, postgres, and pg_ctl, so we do have to do something before 7.3 or we will have new environment variable handling in all those commands, and I don't think we have agreement on that. -- Bruce Momjian| http://candle.pha.pa.us [EMAIL PROTECTED] | (610) 359-1001 + If your life is a hard drive, | 13 Roberts Road + Christ can be your backup.| Newtown Square, Pennsylvania 19073 ---(end of broadcast)--- TIP 1: subscribe and unsubscribe commands go to [EMAIL PROTECTED]
Re: [HACKERS] [COMMITTERS] pgsql-server/src backend/tcop/postgres.cbacke
On Thu, 15 Aug 2002, Bruce Momjian wrote: > I would like to know how to move this item forward. Right now (i.e., in 7.2), the only two options we have for moving the log file to a different spindle are mounting it on pg_xlog and using a symlink. I doubt many people do the the former, and if they do they do not need an option to init_db to move the logfile away from its default location. So I propose we just continue to use the symlink method for the moment, until we agree on another way to store the log file location within the data directory, and at that time we implement the code to do that. Note that if we don't move forward at all, we're still left in the symlink situation, with the exception that you init_db, move the log directory and create the symlink by hand, and then start up the database. So this partial move forward makes no difference to the symlink argument. cjs -- Curt Sampson <[EMAIL PROTECTED]> +81 90 7737 2974 http://www.netbsd.org Don't you know, in this new Dark Age, we're all light. --XTC ---(end of broadcast)--- TIP 6: Have you searched our list archives? http://archives.postgresql.org
Re: [HACKERS] [COMMITTERS] pgsql-server/src backend/tcop/postgres.cbacke
I would like to know how to move this item forward. --- Tom Lane wrote: > Curt Sampson <[EMAIL PROTECTED]> writes: > > ... just for the record I'm with the "don't > > use an environment variable" crowd here, too. It's way, way to easy > > to start up with the wrong setting in your environment. > > What he said ... > > > Oh, and yes, it does need to be changable after an initdb. Say you > > start out with only one disk on your system, but add a second disk > > later, and want to move the log to that? > > Sure, there should be *a* way to do that. It does not have to be as > easy as "change an environment variable". And in fact the primary > objection to this patch is exactly that it is *not* as easy as "change > an environment variable" --- what you get if you just change your > environment variable is not a moved xlog, but a broken database. > Possibly an irredeemably broken database. > > regards, tom lane > -- Bruce Momjian| http://candle.pha.pa.us [EMAIL PROTECTED] | (610) 359-1001 + If your life is a hard drive, | 13 Roberts Road + Christ can be your backup.| Newtown Square, Pennsylvania 19073 ---(end of broadcast)--- TIP 6: Have you searched our list archives? http://archives.postgresql.org
Re: [HACKERS] [COMMITTERS] pgsql-server/src backend/tcop/postgres.cbacke
Curt Sampson <[EMAIL PROTECTED]> writes: > ... just for the record I'm with the "don't > use an environment variable" crowd here, too. It's way, way to easy > to start up with the wrong setting in your environment. What he said ... > Oh, and yes, it does need to be changable after an initdb. Say you > start out with only one disk on your system, but add a second disk > later, and want to move the log to that? Sure, there should be *a* way to do that. It does not have to be as easy as "change an environment variable". And in fact the primary objection to this patch is exactly that it is *not* as easy as "change an environment variable" --- what you get if you just change your environment variable is not a moved xlog, but a broken database. Possibly an irredeemably broken database. regards, tom lane ---(end of broadcast)--- TIP 4: Don't 'kill -9' the postmaster
Re: [HACKERS] [COMMITTERS] pgsql-server/src backend/tcop/postgres.cbacke
On Tue, 13 Aug 2002, scott.marlowe wrote: > My non-coding vote goes with Tom Lane on this. initdb can set pg_xlog, > and if you need to change it, use symlinks. I've not been following this thread, and thus I suppose I missed my opportunity to vote, but just for the record I'm with the "don't use an environment variable" crowd here, too. It's way, way to easy to start up with the wrong setting in your environment. The log is part of the database. Therefore you should store the information on its location along with the rest of the database information. The idea is, you pass *one* piece of information to your program when you start it (in this case the database data directory location), and all of the rest of the information comes from there. Then you have guaranteed consistency. How the log location is stored within that area, I'm not so fussy about. If a symlink is so terrible, why not put this information in the database config file? Oh, and yes, it does need to be changable after an initdb. Say you start out with only one disk on your system, but add a second disk later, and want to move the log to that? cjs -- Curt Sampson <[EMAIL PROTECTED]> +81 90 7737 2974 http://www.netbsd.org Don't you know, in this new Dark Age, we're all light. --XTC ---(end of broadcast)--- TIP 2: you can get off all lists at once with the unregister command (send "unregister YourEmailAddressHere" to [EMAIL PROTECTED])
Re: [HACKERS] [COMMITTERS] pgsql-server/src backend/tcop/postgres.cbacke
On Tue, 13 Aug 2002, Bruce Momjian wrote: > If you move pg_xlog, you have to create a symlink in /data that points > to the new location. Initdb would do that automatically, but if you > move it after initdb, you would have to create the symlink yourself. > With Thomas's current code, you would add/change PGXLOG instead to point > to the new location, rather than modify the symlink. Right, but if the use of PGXLOG/-X it make sense then tryin got document "hey, if you move pg_xlog, go into this directory and change the symlink to point to hte new locaiton" ... as I said, I do not like symlinks ... > Uh, seems it could get messy, but, yea, that would work. It means > adding a file to pg_xlog and /data and somehow matching them. My > feeling was that the symlink was unambiguous and allowed for fewer > mistakes. I think that was Tom's opinion too. Hrmmm ... how about some sort of 'tag' that gets written to the xlog file(s) themselves instead? ---(end of broadcast)--- TIP 6: Have you searched our list archives? http://archives.postgresql.org
Re: [HACKERS] [COMMITTERS] pgsql-server/src backend/tcop/postgres.cbacke
On Mon, 12 Aug 2002, Thomas Lockhart wrote: > > If you move pg_xlog, you have to create a symlink in /data that points > > to the new location. Initdb would do that automatically, but if you > > move it after initdb, you would have to create the symlink yourself. > > With Thomas's current code, you would add/change PGXLOG instead to point > > to the new location, rather than modify the symlink. > > There is no "the symlink", but of course that tinkering is in no way > precluded by the new code. Although some seem to like symlinks, others > (including myself) see no good engineering practice in making them the > only foundation for distributing files across file systems. Why? You often say you don't like them, but I have yet to see you say why you don't like them. > The patches as-is follow existing PostgreSQL practice, using environmental variables is a practice we should discontinue if possible, and use as little as possible. They ARE a security hole waiting to happen. > have complete and > perfect backward compatibility, and do not preclude changes in > underlying implementation in the future if those who are objecting > choose to do a complete and thorough job of meeting my objections to the > current counter-suggestions. As an example, two lines of code in initdb > would add "the beloved symlink" to $PGDATA, eliminating one objection > though (of course) one I don't support. > > > > One thought at the back of my mind is why not have something like a > > > 'PG_VERSION' for XLOG? Maybe something so simple as a text file in both > > > the data and xlog directory that just contains a timestamp from the > > > initdb? then, when you startup postmaster with a -X option, it compares > > > the two files and makes sure that they belong to each other? > > Uh, seems it could get messy, but, yea, that would work. It means > > adding a file to pg_xlog and /data and somehow matching them. My > > feeling was that the symlink was unambiguous and allowed for fewer > > mistakes. I think that was Tom's opinion too. > > In the spirit of gratutious overstatement, I'll point out again: > symlinks are evil. Any sense of a job well done is misplaced if our > underpinnings rely on them for distributing files across file systems. > As an ad hoc hack to work around current limitations they may have some > utility. Why are symlinks evil? They exist on every major OS I know of, and they work. They allow the user to quickly point the postgresql engine in different places, and they are simple and easy to use. I found the use of environmental variables far more confusing when I first started using postgresql than symlinks. In particular, which operating systems does Postgresql run don't have symlink capability? > Anyway, istm that this is way too much discussion for a small extension > of capability, and it has likely cost a table and index "with location" > implementation for the upcoming release just due to time wasted > discussing it. Hope it was worth it :/ Well, if it averts a security problem, or makes the database easier to use in the long run, then it probably was. It may seem like too much discussion for such a simple topic, but it's not. My non-coding vote goes with Tom Lane on this. initdb can set pg_xlog, and if you need to change it, use symlinks. They're safe, secure, and they just plain work. The only argument I can possibly think of against the symlink boogie is if there is an os we run on that can't do symlinks. And then I'd still think it would belong in postgresql.conf, be set by initdb, and not be an environmental variable. Of course that's just my opinion, I could be wrong (with apologies to Dennis Miller) Scott Marlowe ---(end of broadcast)--- TIP 3: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [HACKERS] [COMMITTERS] pgsql-server/src backend/tcop/postgres.cbacke
Yea, the problem with postgresql.conf is that we don't have any automatic modifications of that file, and I don't think we want to start just to avoid symlinks. I personally like symlinks too. I use them all the time. What is the problem with them, exactly? Can someone show me some commands that cause problems? And the problem with a separate file is that when the move pg_xlog, it isn't going to be obvious what they need to change to find the new directory. Of course, they could just create a symlink and leave the file unchanged. Aside from the arg bloat problem, the real danger is that someone is going to forget PGDATA and PGXLOG, try to start the postmaster, add -D for PGDATA, then when they see that they need PGXLOG, they may just create data/pg_xlog as an empty directory and start the postmaster. That is a very real possibility. I just tried it and it does complain about the missing checkpoint records so maybe it isn't as bad as I thought, but still, it opens a place for error where none existed before. --- Tom Lane wrote: > Oliver Elphick <[EMAIL PROTECTED]> writes: > > On Tue, 2002-08-13 at 14:24, Tom Lane wrote: > >> But there's more than one way to record the xlog location in the data > >> directory. If you don't like a symlink, what of putting it in > >> postgresql.conf as a postmaster-start-time-only config option? > > > Please don't! > > > The Debian package at least provides a default postgresql.conf and it > > will be all too easy for someone installing an updated package to let > > the default file overwrite the existing configuration. That could be > > disastrous. > > Ouch. That's a mighty good point ... although if we were to implement > Marc's idea of matching signature files, we'd certainly catch the error. > > If we didn't, we'd need to use a separate, one-purpose config file that > just records the xlog location. Curiously enough, that seems to me to > be exactly what a symlink does, except that the symlink is OS-level code > rather than something we have to write for ourselves. So I'm back to > thinking that a symlink is a perfectly respectable answer. > > regards, tom lane > > ---(end of broadcast)--- > TIP 1: subscribe and unsubscribe commands go to [EMAIL PROTECTED] > -- Bruce Momjian| http://candle.pha.pa.us [EMAIL PROTECTED] | (610) 359-1001 + If your life is a hard drive, | 13 Roberts Road + Christ can be your backup.| Newtown Square, Pennsylvania 19073 ---(end of broadcast)--- TIP 4: Don't 'kill -9' the postmaster
Re: [HACKERS] [COMMITTERS] pgsql-server/src backend/tcop/postgres.cbacke
Oliver Elphick <[EMAIL PROTECTED]> writes: > On Tue, 2002-08-13 at 14:24, Tom Lane wrote: >> But there's more than one way to record the xlog location in the data >> directory. If you don't like a symlink, what of putting it in >> postgresql.conf as a postmaster-start-time-only config option? > Please don't! > The Debian package at least provides a default postgresql.conf and it > will be all too easy for someone installing an updated package to let > the default file overwrite the existing configuration. That could be > disastrous. Ouch. That's a mighty good point ... although if we were to implement Marc's idea of matching signature files, we'd certainly catch the error. If we didn't, we'd need to use a separate, one-purpose config file that just records the xlog location. Curiously enough, that seems to me to be exactly what a symlink does, except that the symlink is OS-level code rather than something we have to write for ourselves. So I'm back to thinking that a symlink is a perfectly respectable answer. regards, tom lane ---(end of broadcast)--- TIP 1: subscribe and unsubscribe commands go to [EMAIL PROTECTED]
Re: [HACKERS] [COMMITTERS] pgsql-server/src backend/tcop/postgres.cbacke
Rod Taylor <[EMAIL PROTECTED]> writes: > Symlinks seem to break all over the place (windows, novell, os/2), The portability argument carries little weight with me. Recent versions of Windows have symlinks. If anyone wants to run a PG installation on a symlink-less platform, okay; they just won't have the option to move the xlog directory. That's probably not the only functionality they lose by using such an inadequate filesystem... regards, tom lane ---(end of broadcast)--- TIP 5: Have you checked our extensive FAQ? http://www.postgresql.org/users-lounge/docs/faq.html
Re: [HACKERS] [COMMITTERS] pgsql-server/src backend/tcop/postgres.cbacke
"Marc G. Fournier" <[EMAIL PROTECTED]> writes: > One thought at the back of my mind is why not have something like a > 'PG_VERSION' for XLOG? Maybe something so simple as a text file in both > the data and xlog directory that just contains a timestamp from the > initdb? then, when you startup postmaster with a -X option, it compares > the two files and makes sure that they belong to each other? While that isn't a bad idea, it seems to me that it's adding mechanism to get around a problem that we don't need to have in the first place. The only reason this risk exists is that the patch changes a monolithic postmaster option (-D) into two independent options (-D/-X) that in reality should never be independent. Essentially, you're proposing Kevlar shoes as a solution for the problem that you want to walk around carrying a loaded gun aimed at your foot. The shoes might be a good idea anyway, but the primary problem is elsewhere... regards, tom lane ---(end of broadcast)--- TIP 1: subscribe and unsubscribe commands go to [EMAIL PROTECTED]
Re: [HACKERS] [COMMITTERS] pgsql-server/src backend/tcop/postgres.cbacke
Thomas Lockhart <[EMAIL PROTECTED]> writes: > In the spirit of gratutious overstatement, I'll point out again: > symlinks are evil. Please justify that claim. They work really nicely in my experience... and I don't know of any modern Unix system that doesn't rely on them *heavily*. Possibly more to the point, I can assert "environment variables are evil" with at least as much foundation. We have seen many many reports of trouble from people who were bit by environment-variable problems with Postgres. Do I need to trawl the archives for examples? However, as I just commented to Marc the real issue in my mind is that the xlog needs to be solidly tied to the data directory, because we can't risk starting a postmaster with the wrong combination. I do not think that external specification of the xlog as a separate env-var or postmaster command-line arg gives the appropriate amount of safety. But there's more than one way to record the xlog location in the data directory. If you don't like a symlink, what of putting it in postgresql.conf as a postmaster-start-time-only config option? regards, tom lane ---(end of broadcast)--- TIP 6: Have you searched our list archives? http://archives.postgresql.org
Re: [HACKERS] [COMMITTERS] pgsql-server/src backend/tcop/postgres.cbacke
Bruce Momjian <[EMAIL PROTECTED]> writes: > Tom Lane wrote: >> We will? It looks to me like Thomas lost the vote 2-to-1. > Well, you didn't vote again in my follow up email, I thought my vote was obvious already ... > Can two guys override another guy if he is doing the work? I usually > like to have a larger margin than that. I don't know what to do. I'm not pleased about it either; I'd have preferred to see a few more opinions given (and I'm surprised that no one else bothered to weigh in; lack of opinions is usually not a problem for pghackers ;-)). But I really seriously feel that this feature is a bad idea as presently implemented. If necessary, I'll volunteer to change it the way I think it should be (viz, initdb can set up a symlink to a specified xlog directory; no change from previous behavior anywhere else). regards, tom lane ---(end of broadcast)--- TIP 5: Have you checked our extensive FAQ? http://www.postgresql.org/users-lounge/docs/faq.html
Re: [HACKERS] [COMMITTERS] pgsql-server/src backend/tcop/postgres.cbacke
> > OK, seeing as no one voted, and only Tom and I objected originally, we > > will keep the code as Thomas has applied it, namely that PGXLOG/-X is > > recognized by initdb, postmaster, postgres, and pg_ctl. > > We will? It looks to me like Thomas lost the vote 2-to-1. > > Unless there are more votes, I'm going to *insist* that this code be > changed. It's dangerous and offers no offsetting benefit. XLOG > location should be settable at initdb, noplace later. 2 would get my vote too. My approach though would be that initdb simply creates a symlink. I like to find the files without resorting to additional utilities or an sql, at least on platforms that support symlinks. This makes the task of knowing what needs to be backed up easier. My approach to tablespaces would probably also have symlinks in the datadir. Andreas ---(end of broadcast)--- TIP 2: you can get off all lists at once with the unregister command (send "unregister YourEmailAddressHere" to [EMAIL PROTECTED])
Re: [HACKERS] [COMMITTERS] pgsql-server/src backend/tcop/postgres.cbacke
> If you move pg_xlog, you have to create a symlink in /data that points > to the new location. Initdb would do that automatically, but if you > move it after initdb, you would have to create the symlink yourself. > With Thomas's current code, you would add/change PGXLOG instead to point > to the new location, rather than modify the symlink. There is no "the symlink", but of course that tinkering is in no way precluded by the new code. Although some seem to like symlinks, others (including myself) see no good engineering practice in making them the only foundation for distributing files across file systems. The patches as-is follow existing PostgreSQL practice, have complete and perfect backward compatibility, and do not preclude changes in underlying implementation in the future if those who are objecting choose to do a complete and thorough job of meeting my objections to the current counter-suggestions. As an example, two lines of code in initdb would add "the beloved symlink" to $PGDATA, eliminating one objection though (of course) one I don't support. > > One thought at the back of my mind is why not have something like a > > 'PG_VERSION' for XLOG? Maybe something so simple as a text file in both > > the data and xlog directory that just contains a timestamp from the > > initdb? then, when you startup postmaster with a -X option, it compares > > the two files and makes sure that they belong to each other? > Uh, seems it could get messy, but, yea, that would work. It means > adding a file to pg_xlog and /data and somehow matching them. My > feeling was that the symlink was unambiguous and allowed for fewer > mistakes. I think that was Tom's opinion too. In the spirit of gratutious overstatement, I'll point out again: symlinks are evil. Any sense of a job well done is misplaced if our underpinnings rely on them for distributing files across file systems. As an ad hoc hack to work around current limitations they may have some utility. Anyway, istm that this is way too much discussion for a small extension of capability, and it has likely cost a table and index "with location" implementation for the upcoming release just due to time wasted discussing it. Hope it was worth it :/ - Thomas ---(end of broadcast)--- TIP 2: you can get off all lists at once with the unregister command (send "unregister YourEmailAddressHere" to [EMAIL PROTECTED])
Re: [HACKERS] [COMMITTERS] pgsql-server/src backend/tcop/postgres.cbacke
Marc G. Fournier wrote: > > Well, you didn't vote again in my follow up email, so I thought you > > didn't care anymore, and Thomas didn't reply to my email either. I am > > clearly concerned, as you are, but Thomas isn't, and no one else seems > > to care. > > k, why are you concerned? see my other message, but if the major concern > is the xlog directory for a postmaster, then put in a consistency check > for when starting ... I really have two concerns; one, the ability to point to the wrong place or to the default place accidentally if PGXLOG isn't set, and two, the addition of PGXLOG/-X into postmaster, postgres, pg_ctl where it really isn't adding any functionality and just adds bloat to the commands arg list. > then again, how many out there are running multiple instances of > postmaster on their machine that would move xlog to a different location > without realizing they did? I know in my case, we're working at reducing > the # of instances on our servers to 2 (internal vs external), but, our > servers are all on a RAID5 array, so there is nowhere to really "move" > xlog to out then its default location ... but I can see the usefulness of > doing so ... > > Myself, if I'm going to move something around, it will be after the server > has been running for a while and I've added in more drive space in order > to correct a problem i didn't anticipate (namely, disk I/O) ... at that > point, I really don't wan tto have to dump/re-initdb/load just to move the > xlog directory to that new drive ... No dump/reload required, but the creation of a _dreaded_ symlink is required. -- Bruce Momjian| http://candle.pha.pa.us [EMAIL PROTECTED] | (610) 359-1001 + If your life is a hard drive, | 13 Roberts Road + Christ can be your backup.| Newtown Square, Pennsylvania 19073 ---(end of broadcast)--- TIP 1: subscribe and unsubscribe commands go to [EMAIL PROTECTED]
Re: [HACKERS] [COMMITTERS] pgsql-server/src backend/tcop/postgres.cbacke
Marc G. Fournier wrote: > On Tue, 13 Aug 2002, Tom Lane wrote: > > > Bruce Momjian <[EMAIL PROTECTED]> writes: > > > OK, seeing as no one voted, and only Tom and I objected originally, we > > > will keep the code as Thomas has applied it, namely that PGXLOG/-X is > > > recognized by initdb, postmaster, postgres, and pg_ctl. > > > > We will? It looks to me like Thomas lost the vote 2-to-1. > > > > Unless there are more votes, I'm going to *insist* that this code be > > changed. It's dangerous and offers no offsetting benefit. XLOG > > location should be settable at initdb, noplace later. > > Okay, I'm going to pop up here and side with Thomas ... I think ... I may > have missed some issues here, but, quite frankly, I hate symlinks, as I've > seen it create more evil then good .. hardlinks is a different story ... OK, that agrees with Thomas's aversion to symlinks. > And for that reason, I will side with Thomas ... > > initdb should allow you to specify a seperate location, which I don't > think anyone disagrees with ... but, what happens if, for some reason, I > have to move it to another location? I have to then dump/reload after > doing a new initdb? If you move pg_xlog, you have to create a symlink in /data that points to the new location. Initdb would do that automatically, but if you move it after initdb, you would have to create the symlink yourself. With Thomas's current code, you would add/change PGXLOG instead to point to the new location, rather than modify the symlink. > One thought at the back of my mind is why not have something like a > 'PG_VERSION' for XLOG? Maybe something so simple as a text file in both > the data and xlog directory that just contains a timestamp from the > initdb? then, when you startup postmaster with a -X option, it compares > the two files and makes sure that they belong to each other? Uh, seems it could get messy, but, yea, that would work. It means adding a file to pg_xlog and /data and somehow matching them. My feeling was that the symlink was unambiguous and allowed for fewer mistakes. I think that was Tom's opinion too. > Bruce, if I'm missing something, could you post a summary/scorecard for > pros-cons on this issue? OK, I tried. :-) -- Bruce Momjian| http://candle.pha.pa.us [EMAIL PROTECTED] | (610) 359-1001 + If your life is a hard drive, | 13 Roberts Road + Christ can be your backup.| Newtown Square, Pennsylvania 19073 ---(end of broadcast)--- TIP 3: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [HACKERS] [COMMITTERS] pgsql-server/src backend/tcop/postgres.cbacke
On Tue, 13 Aug 2002, Bruce Momjian wrote: > Tom Lane wrote: > > Bruce Momjian <[EMAIL PROTECTED]> writes: > > > OK, seeing as no one voted, and only Tom and I objected originally, we > > > will keep the code as Thomas has applied it, namely that PGXLOG/-X is > > > recognized by initdb, postmaster, postgres, and pg_ctl. > > > > We will? It looks to me like Thomas lost the vote 2-to-1. > > > > Unless there are more votes, I'm going to *insist* that this code be > > changed. It's dangerous and offers no offsetting benefit. XLOG > > location should be settable at initdb, noplace later. > > Well, you didn't vote again in my follow up email, so I thought you > didn't care anymore, and Thomas didn't reply to my email either. I am > clearly concerned, as you are, but Thomas isn't, and no one else seems > to care. k, why are you concerned? see my other message, but if the major concern is the xlog directory for a postmaster, then put in a consistency check for when starting ... then again, how many out there are running multiple instances of postmaster on their machine that would move xlog to a different location without realizing they did? I know in my case, we're working at reducing the # of instances on our servers to 2 (internal vs external), but, our servers are all on a RAID5 array, so there is nowhere to really "move" xlog to out then its default location ... but I can see the usefulness of doing so ... Myself, if I'm going to move something around, it will be after the server has been running for a while and I've added in more drive space in order to correct a problem i didn't anticipate (namely, disk I/O) ... at that point, I really don't wan tto have to dump/re-initdb/load just to move the xlog directory to that new drive ... ---(end of broadcast)--- TIP 6: Have you searched our list archives? http://archives.postgresql.org
Re: [HACKERS] [COMMITTERS] pgsql-server/src backend/tcop/postgres.cbacke
On Tue, 13 Aug 2002, Tom Lane wrote: > Bruce Momjian <[EMAIL PROTECTED]> writes: > > OK, seeing as no one voted, and only Tom and I objected originally, we > > will keep the code as Thomas has applied it, namely that PGXLOG/-X is > > recognized by initdb, postmaster, postgres, and pg_ctl. > > We will? It looks to me like Thomas lost the vote 2-to-1. > > Unless there are more votes, I'm going to *insist* that this code be > changed. It's dangerous and offers no offsetting benefit. XLOG > location should be settable at initdb, noplace later. Okay, I'm going to pop up here and side with Thomas ... I think ... I may have missed some issues here, but, quite frankly, I hate symlinks, as I've seen it create more evil then good .. hardlinks is a different story ... And for that reason, I will side with Thomas ... initdb should allow you to specify a seperate location, which I don't think anyone disagrees with ... but, what happens if, for some reason, I have to move it to another location? I have to then dump/reload after doing a new initdb? One thought at the back of my mind is why not have something like a 'PG_VERSION' for XLOG? Maybe something so simple as a text file in both the data and xlog directory that just contains a timestamp from the initdb? then, when you startup postmaster with a -X option, it compares the two files and makes sure that they belong to each other? Bruce, if I'm missing something, could you post a summary/scorecard for pros-cons on this issue? ---(end of broadcast)--- TIP 4: Don't 'kill -9' the postmaster
Re: [HACKERS] [COMMITTERS] pgsql-server/src backend/tcop/postgres.cbacke
Tom Lane wrote: > Bruce Momjian <[EMAIL PROTECTED]> writes: > > OK, seeing as no one voted, and only Tom and I objected originally, we > > will keep the code as Thomas has applied it, namely that PGXLOG/-X is > > recognized by initdb, postmaster, postgres, and pg_ctl. > > We will? It looks to me like Thomas lost the vote 2-to-1. > > Unless there are more votes, I'm going to *insist* that this code be > changed. It's dangerous and offers no offsetting benefit. XLOG > location should be settable at initdb, noplace later. Well, you didn't vote again in my follow up email, so I thought you didn't care anymore, and Thomas didn't reply to my email either. I am clearly concerned, as you are, but Thomas isn't, and no one else seems to care. Can two guys override another guy if he is doing the work? I usually like to have a larger margin than that. I don't know what to do. -- Bruce Momjian| http://candle.pha.pa.us [EMAIL PROTECTED] | (610) 359-1001 + If your life is a hard drive, | 13 Roberts Road + Christ can be your backup.| Newtown Square, Pennsylvania 19073 ---(end of broadcast)--- TIP 3: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [HACKERS] [COMMITTERS] pgsql-server/src backend/tcop/postgres.cbacke
Bruce Momjian <[EMAIL PROTECTED]> writes: > OK, seeing as no one voted, and only Tom and I objected originally, we > will keep the code as Thomas has applied it, namely that PGXLOG/-X is > recognized by initdb, postmaster, postgres, and pg_ctl. We will? It looks to me like Thomas lost the vote 2-to-1. Unless there are more votes, I'm going to *insist* that this code be changed. It's dangerous and offers no offsetting benefit. XLOG location should be settable at initdb, noplace later. regards, tom lane ---(end of broadcast)--- TIP 5: Have you checked our extensive FAQ? http://www.postgresql.org/users-lounge/docs/faq.html
Re: [HACKERS] [COMMITTERS] pgsql-server/src backend/tcop/postgres.cbacke
OK, seeing as no one voted, and only Tom and I objected originally, we will keep the code as Thomas has applied it, namely that PGXLOG/-X is recognized by initdb, postmaster, postgres, and pg_ctl. There is no symlink from the /data directory to the WAL location. Thomas, you mentioned implementing table spaces. Are you planning to use environment variables there too? You mentioned that Ingres's use of environment variables wasn't well implemented. How will you improve on it? --- Bruce Momjian wrote: > Thomas Lockhart wrote: > > > Thomas, would you remind me of the concusions because I thought everyone > > > involved felt that it should be an initdb-only option, but I still see > > > it in CVS. > > > > ?? "Concussions" as in brain bruises? ;) > > Uh, conclusions. Sorry. New keyboard desk in new house. :-) > > > I'm not sure I understand the question. I assume that we are talking > > about the WAL log location feature I implemented recently. It is an > > initdb-only option, and defaults to the current behavior *exactly*. > > Yep. What bothers me is the clutter to the other commands that allow > XLOG location specification when you would never really want to specify > it except as part of initdb. I just see those extra flags as > cruft/confusion. > > Look at pg_ctl: > > pg_ctl start [-w] [-D DATADIR] [-s] [-X PGXLOG] [-l FILENAME] [-o "OPTIONS"] > > Which option doesn't make sense? -X. It is way beyond the > functionality of the command. > > > > The new feature is to allow an argument to initdb to locate the WAL file > > to another location. That location can be specified on the command line, > > or through an environment variable. Neither form precludes use of the > > other, and either form can be considered "best practice" depending on > > your opinion of what that is. > > > > The postmaster also recognizes the command line option and environment > > variable. The only suggestion I got as an alternative involved soft > > links, which is not portable, which is not robust, and which is not used > > anywhere else in the system. If we moved toward relying on soft links > > for distributing resources we will be moving in the wrong direction for > > many reasons, some of which I've mentioned previously. GUC parameters > > were also mentioned as a possibility, and the infrastructure does not > > preclude that at any time. > > I don't think anyone agreed with your concerns about symlinks. If you > want to be careful, do the "ln -s" in initdb and exit on failure, and > tell them not to use -X on that platform, though we use symlinks for > postmaster/postgres identification, so I know the only OS that doesn't > support symlinks is Netware, only because Netware folks just sent in a > patch to add a -post flag to work around lack of symlinks. (I have > asked for clarification from them.) > > I actually requested a vote, and got several people who wanted my > compromise (PGXLOG or initdb -X flag only), and I didn't see anyone who > liked the addition of -X into non-initdb commands. Should I have a > specific vote? OK, three options: > > 1) -X, PGXLOG in initdb, postmaster, postgres, pg_ctl > 2) -X, PGXLOG in initdb only > 3) nothing > > I remember a number of people liking 2, but we can vote again. > > > I don't recall that there were very many folks "involved". There were > > several opinions, though most were from folks who were not thinking of > > implementing disk management features. Some opinions dealt with details, > > and some seemed to deal with the wisdom of allowing anything other than > > a "one partition" model of the database, which is nothing if not short > > sighted. Current default behavior is as first implemented, and the new > > feature allows locating the WAL logs in another area. For the current > > state of the art, that seems competitive with features found in other > > database products, and an essential step in teaching PostgreSQL to work > > with very large databases. > > > > I had thought to extend the capabilities to allow resource allocation > > for individual tables and indices, which has *long* been identified as a > > desired capability by folks who are managing large systems. It seemed > > reasonable to have done in time for 7.3. I'm rethinking that, not > > because it shouldn't happen, but because the process of discussing these > > issues has become so argumentative, divisive, impolite, and unpleasant. > > Which is a shame imho... > > I clearly want tablespaces, and it would be great for 7.3, and I don't > think it is a huge job. However, I think it will require symlinks to be > usable, and you probably do not, so it may be an issue. > > As for the argumentativeness, we do have folks with some strong > opinions, and I guess on the PGXLOG issue, I am one of them. Maybe that > is bad? Are people expressing themselves badly? If so, I would like to > hear d
Re: [HACKERS] [COMMITTERS] pgsql-server/src backend/tcop/postgres.cbacke
On Fri, Aug 09, 2002 at 06:54:44PM -0700, Thomas Lockhart wrote: > I had thought to extend the capabilities to allow resource allocation > for individual tables and indices, which has *long* been identified as a > desired capability by folks who are managing large systems. Without wishing to pour fuel on any fires, or advocate any implementation, I can say for sure that this is very much a feature we'd love to see. A -- Andrew Sullivan 87 Mowat Avenue Liberty RMS Toronto, Ontario Canada <[EMAIL PROTECTED]> M6K 3E3 +1 416 646 3304 x110 ---(end of broadcast)--- TIP 1: subscribe and unsubscribe commands go to [EMAIL PROTECTED]
Re: [HACKERS] [COMMITTERS] pgsql-server/src backend/tcop/postgres.cbacke
Thomas Lockhart wrote: > > Thomas, would you remind me of the concusions because I thought everyone > > involved felt that it should be an initdb-only option, but I still see > > it in CVS. > > ?? "Concussions" as in brain bruises? ;) Uh, conclusions. Sorry. New keyboard desk in new house. :-) > I'm not sure I understand the question. I assume that we are talking > about the WAL log location feature I implemented recently. It is an > initdb-only option, and defaults to the current behavior *exactly*. Yep. What bothers me is the clutter to the other commands that allow XLOG location specification when you would never really want to specify it except as part of initdb. I just see those extra flags as cruft/confusion. Look at pg_ctl: pg_ctl start [-w] [-D DATADIR] [-s] [-X PGXLOG] [-l FILENAME] [-o "OPTIONS"] Which option doesn't make sense? -X. It is way beyond the functionality of the command. > The new feature is to allow an argument to initdb to locate the WAL file > to another location. That location can be specified on the command line, > or through an environment variable. Neither form precludes use of the > other, and either form can be considered "best practice" depending on > your opinion of what that is. > > The postmaster also recognizes the command line option and environment > variable. The only suggestion I got as an alternative involved soft > links, which is not portable, which is not robust, and which is not used > anywhere else in the system. If we moved toward relying on soft links > for distributing resources we will be moving in the wrong direction for > many reasons, some of which I've mentioned previously. GUC parameters > were also mentioned as a possibility, and the infrastructure does not > preclude that at any time. I don't think anyone agreed with your concerns about symlinks. If you want to be careful, do the "ln -s" in initdb and exit on failure, and tell them not to use -X on that platform, though we use symlinks for postmaster/postgres identification, so I know the only OS that doesn't support symlinks is Netware, only because Netware folks just sent in a patch to add a -post flag to work around lack of symlinks. (I have asked for clarification from them.) I actually requested a vote, and got several people who wanted my compromise (PGXLOG or initdb -X flag only), and I didn't see anyone who liked the addition of -X into non-initdb commands. Should I have a specific vote? OK, three options: 1) -X, PGXLOG in initdb, postmaster, postgres, pg_ctl 2) -X, PGXLOG in initdb only 3) nothing I remember a number of people liking 2, but we can vote again. > I don't recall that there were very many folks "involved". There were > several opinions, though most were from folks who were not thinking of > implementing disk management features. Some opinions dealt with details, > and some seemed to deal with the wisdom of allowing anything other than > a "one partition" model of the database, which is nothing if not short > sighted. Current default behavior is as first implemented, and the new > feature allows locating the WAL logs in another area. For the current > state of the art, that seems competitive with features found in other > database products, and an essential step in teaching PostgreSQL to work > with very large databases. > > I had thought to extend the capabilities to allow resource allocation > for individual tables and indices, which has *long* been identified as a > desired capability by folks who are managing large systems. It seemed > reasonable to have done in time for 7.3. I'm rethinking that, not > because it shouldn't happen, but because the process of discussing these > issues has become so argumentative, divisive, impolite, and unpleasant. > Which is a shame imho... I clearly want tablespaces, and it would be great for 7.3, and I don't think it is a huge job. However, I think it will require symlinks to be usable, and you probably do not, so it may be an issue. As for the argumentativeness, we do have folks with some strong opinions, and I guess on the PGXLOG issue, I am one of them. Maybe that is bad? Are people expressing themselves badly? If so, I would like to hear details either on list or privately so I can address them. -- Bruce Momjian| http://candle.pha.pa.us [EMAIL PROTECTED] | (610) 359-1001 + If your life is a hard drive, | 13 Roberts Road + Christ can be your backup.| Newtown Square, Pennsylvania 19073 ---(end of broadcast)--- TIP 5: Have you checked our extensive FAQ? http://www.postgresql.org/users-lounge/docs/faq.html
Re: [HACKERS] [COMMITTERS] pgsql-server/src backend/tcop/postgres.cbacke
> Thomas, would you remind me of the concusions because I thought everyone > involved felt that it should be an initdb-only option, but I still see > it in CVS. ?? "Concussions" as in brain bruises? ;) I'm not sure I understand the question. I assume that we are talking about the WAL log location feature I implemented recently. It is an initdb-only option, and defaults to the current behavior *exactly*. The new feature is to allow an argument to initdb to locate the WAL file to another location. That location can be specified on the command line, or through an environment variable. Neither form precludes use of the other, and either form can be considered "best practice" depending on your opinion of what that is. The postmaster also recognizes the command line option and environment variable. The only suggestion I got as an alternative involved soft links, which is not portable, which is not robust, and which is not used anywhere else in the system. If we moved toward relying on soft links for distributing resources we will be moving in the wrong direction for many reasons, some of which I've mentioned previously. GUC parameters were also mentioned as a possibility, and the infrastructure does not preclude that at any time. I don't recall that there were very many folks "involved". There were several opinions, though most were from folks who were not thinking of implementing disk management features. Some opinions dealt with details, and some seemed to deal with the wisdom of allowing anything other than a "one partition" model of the database, which is nothing if not short sighted. Current default behavior is as first implemented, and the new feature allows locating the WAL logs in another area. For the current state of the art, that seems competitive with features found in other database products, and an essential step in teaching PostgreSQL to work with very large databases. I had thought to extend the capabilities to allow resource allocation for individual tables and indices, which has *long* been identified as a desired capability by folks who are managing large systems. It seemed reasonable to have done in time for 7.3. I'm rethinking that, not because it shouldn't happen, but because the process of discussing these issues has become so argumentative, divisive, impolite, and unpleasant. Which is a shame imho... - Thomas ---(end of broadcast)--- TIP 3: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [HACKERS] [COMMITTERS] pgsql-server/src backend/tcop/postgres.cbacke
Thomas, would you remind me of the concusions because I thought everyone involved felt that it should be an initdb-only option, but I still see it in CVS. --- Thomas Lockhart wrote: > > Thomas, have you commented on the objections to this patch? If so, I > > didn't see it. > > Yes, there was quite a long thread on this. > >- Thomas > > ---(end of broadcast)--- > TIP 6: Have you searched our list archives? > > http://archives.postgresql.org > -- Bruce Momjian| http://candle.pha.pa.us [EMAIL PROTECTED] | (610) 359-1001 + If your life is a hard drive, | 13 Roberts Road + Christ can be your backup.| Newtown Square, Pennsylvania 19073 ---(end of broadcast)--- TIP 5: Have you checked our extensive FAQ? http://www.postgresql.org/users-lounge/docs/faq.html
