Re: [HACKERS] location of the configuration files
Bruce Momjian wrote: I have a new idea. You know how we have search_path where you can specify multiple schema names. What if we allow the config_dirs/-C to specify multiple directories to search for config files. That way, we can use only one variable, and we can allow people to place different config files in different directories. That's an interesting idea. Were you thinking, perhaps, that you could put, say, a postgresql.conf file in multiple directories and have the settings in the latest one override the settings in earlier ones? That would mean you could set up a single postgresql.conf that has settings common to all of your instances (settings related to the system such as shared buffers, and default settings that would apply to any instance if not overridden), and a postgresql.conf file for each instance that defines the instance-specific configuration information. I'm not sure that's quite what you had in mind, though. :-) -- Kevin Brown [EMAIL PROTECTED] ---(end of broadcast)--- TIP 3: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [HACKERS] location of the configuration files
Bruce Momjian [EMAIL PROTECTED] writes: I have a new idea. You know how we have search_path where you can specify multiple schema names. What if we allow the config_dirs/-C to specify multiple directories to search for config files. That way, we can use only one variable, and we can allow people to place different config files in different directories. Hm, a search path for config files? I could support that if it satisfies the folks who object to specifying config directories rather than file names. One thing that would have to be thought about is whether to re-search the path on each config file reload --- if you first find pg_hba.conf in, say, the third directory on the path, should you pay attention if one materializes in the second directory later? Or do you keep going back to the same well? I can see arguments either way. regards, tom lane ---(end of broadcast)--- TIP 4: Don't 'kill -9' the postmaster
Re: [HACKERS] location of the configuration files
Tom Lane wrote: Bruce Momjian [EMAIL PROTECTED] writes: I have a new idea. You know how we have search_path where you can specify multiple schema names. What if we allow the config_dirs/-C to specify multiple directories to search for config files. That way, we can use only one variable, and we can allow people to place different config files in different directories. Hm, a search path for config files? I could support that if it satisfies the folks who object to specifying config directories rather than file names. One thing that would have to be thought about is whether to re-search the path on each config file reload --- if you first find pg_hba.conf in, say, the third directory on the path, should you pay attention if one materializes in the second directory later? Or do you keep going back to the same well? I can see arguments either way. Oh, I hadn't thought of that. I would vote for researching the path, but I am not sure why. The bigger question is whether you can modify config_dirs while the postmaster is running. I would think not. -- Bruce Momjian| http://candle.pha.pa.us [EMAIL PROTECTED] | (610) 359-1001 + If your life is a hard drive, | 13 Roberts Road + Christ can be your backup.| Newtown Square, Pennsylvania 19073 ---(end of broadcast)--- TIP 3: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [HACKERS] location of the configuration files
Bruce Momjian [EMAIL PROTECTED] writes: The bigger question is whether you can modify config_dirs while the postmaster is running. I would think not. There would be no way to do that, because the only way to set it would be from -C on the command line or a PGCONFIG environment variable. But I can't see a real good reason why you'd need to. regards, tom lane ---(end of broadcast)--- TIP 2: you can get off all lists at once with the unregister command (send unregister YourEmailAddressHere to [EMAIL PROTECTED])
Re: [HACKERS] location of the configuration files
On Sat, 15 Feb 2003, Curt Sampson wrote: On Fri, 14 Feb 2003, scott.marlowe wrote: Asking for everything in a directory with the name local in it to be shared is kind of counter intuitive to me. Not really. If you install a particular program that doesn't come with the OS on one machine on your site, why would you not want to install it separately on all of the others? Typically, I want my favourite non-OS utilities on all machines, not just one. (Even if I don't use them on all machines.) Thus /usr/local is for site-local stuff. Good point. Of course, in apache, it's quite easy to use the -f switch to pick the file you're running on. so, with a httpd -f /usr/local/apache/conf/`uname -a|cut -d -f 2`.conf I can pick and choose the file to run. So, yes, I would gladly use it in a cluster, and all the files would be in one place, easy to backup. ---(end of broadcast)--- TIP 3: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [HACKERS] location of the configuration files
I have a new idea. You know how we have search_path where you can specify multiple schema names. What if we allow the config_dirs/-C to specify multiple directories to search for config files. That way, we can use only one variable, and we can allow people to place different config files in different directories. --- Andrew Sullivan wrote: On Sun, Feb 16, 2003 at 12:16:44AM -0500, Tom Lane wrote: Nor will I buy an argument that only a few developers have need for test installations. Ordinary users will want to do that anytime they are doing preliminary tests on a new PG version before migrating their production database to it. To the extent that you make manual selection of a nonstandard data_dir location more difficult and error-prone, you are hurting them too. Not only that. For safety's sake, you may need to run multiple postmasters on one machine (so that database user X can't DoS database user Y, for instance). And making that sort of production-grade work more difficult and error-prone would also be bad. A -- Andrew Sullivan 204-4141 Yonge Street Liberty RMS Toronto, Ontario Canada [EMAIL PROTECTED] M2P 2A8 +1 416 646 3304 x110 ---(end of broadcast)--- TIP 2: you can get off all lists at once with the unregister command (send unregister YourEmailAddressHere to [EMAIL PROTECTED]) -- Bruce Momjian| http://candle.pha.pa.us [EMAIL PROTECTED] | (610) 359-1001 + If your life is a hard drive, | 13 Roberts Road + Christ can be your backup.| Newtown Square, Pennsylvania 19073 ---(end of broadcast)--- TIP 6: Have you searched our list archives? http://archives.postgresql.org
Re: [HACKERS] location of the configuration files
Kevin Brown [EMAIL PROTECTED] writes: Is it possible for the database engine to properly deal with a database when it is told to use a different database encoding than the one the database was initdb'd with? It can't be told to use a different database encoding. However, the default *client* encoding matches the database encoding, and that is something that can be set in the config file. If we do it the way I suggested (dump into the datadir, which is initially empty, same as always) then it cannot overwrite your existing config files. Think of it as providing a suggested config file to compare against what you have. There is one minor complication: what if there's an existing config file in the target directory? If there's anything at all in the target directory, initdb refuses to run. regards, tom lane ---(end of broadcast)--- TIP 6: Have you searched our list archives? http://archives.postgresql.org
Re: [HACKERS] location of the configuration files
On Fri, Feb 14, 2003 at 10:35:41AM -0500, Tom Lane wrote: FHS or no FHS, I would think that the preferred arrangement would be to keep Postgres' config files in a postgres-owned subdirectory, not directly in /etc. That way you need not be root to edit them. (My idea Besides, what are you going to do for people installing on a box where they don't have root? Are they going to need a whole mess of extra directories in their private copy? of an editor, Emacs, always wants to write a backup file, so I dislike having to edit files that live in directories I can't write.) Here's a pretty topic for a flamewar: should it be /etc/postgres/ or /etc/postgresql/ ? Wow, two flamewar topics in one mail. I'm impressed. Andrew ed is the one true editor Sullivan -- Andrew Sullivan 204-4141 Yonge Street Liberty RMS Toronto, Ontario Canada [EMAIL PROTECTED] M2P 2A8 +1 416 646 3304 x110 ---(end of broadcast)--- TIP 3: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [HACKERS] location of the configuration files
On Sun, Feb 16, 2003 at 12:16:44AM -0500, Tom Lane wrote: Nor will I buy an argument that only a few developers have need for test installations. Ordinary users will want to do that anytime they are doing preliminary tests on a new PG version before migrating their production database to it. To the extent that you make manual selection of a nonstandard datadir location more difficult and error-prone, you are hurting them too. Not only that. For safety's sake, you may need to run multiple postmasters on one machine (so that database user X can't DoS database user Y, for instance). And making that sort of production-grade work more difficult and error-prone would also be bad. A -- Andrew Sullivan 204-4141 Yonge Street Liberty RMS Toronto, Ontario Canada [EMAIL PROTECTED] M2P 2A8 +1 416 646 3304 x110 ---(end of broadcast)--- TIP 2: you can get off all lists at once with the unregister command (send unregister YourEmailAddressHere to [EMAIL PROTECTED])
Re: [HACKERS] location of the configuration files
Tom Lane writes: AFAICS, you can either set -C to /etc if you want your PG config files loose in /etc, or you can set it to /etc/postgresql/ if you want them in a privately-owned directory. Which other arrangements are needed? People might want to share them between servers, or allow a user to select from a few pre-configured ones that which reside in the same directory. -- Peter Eisentraut [EMAIL PROTECTED] ---(end of broadcast)--- TIP 1: subscribe and unsubscribe commands go to [EMAIL PROTECTED]
Re: [HACKERS] location of the configuration files
Peter Eisentraut [EMAIL PROTECTED] writes: Tom Lane writes: AFAICS, you can either set -C to /etc if you want your PG config files loose in /etc, or you can set it to /etc/postgresql/ if you want them in a privately-owned directory. Which other arrangements are needed? People might want to share them between servers, or allow a user to select from a few pre-configured ones that which reside in the same directory. You can accomplish that without the need to customize the .conf file names; you just make, eg, /etc/postgres/myconfig/postgresql.conf /etc/postgres/yourconfig/postgresql.conf /etc/postgres/herconfig/postgresql.conf (plus additional config files as needed in each of these directories) and then the postmaster start command is postmaster -C /etc/postgres/myconfig I see no real gain in flexibility in allowing people to choose random names for the individual config files. Also, it'd defeat the ultimate-fallback approach of doing find / -name postgresql.conf to figure out where the config files are hiding in an unfamiliar installation. regards, tom lane ---(end of broadcast)--- TIP 4: Don't 'kill -9' the postmaster
Re: [HACKERS] location of the configuration files
Lamar Owen [EMAIL PROTECTED] writes: The script's been out there for awhile. It does some things well, and some things not so well. The config files are still coresident with the database, and backup is more difficult than it can be. Meeting all these needs (with configure switches, configuration file directives, etc) would be a good thing. Sure. I'm happy to change the software in a way that *allows* moving the config files elsewhere. But it's not apparent to me why you insist on forcing people who are perfectly happy with their existing configuration arrangements to change them. I have not seen any reason in this discussion why we can't support both a separate-config-location approach and the traditional single-location one. Please remember that the existing approach has been evolved over quite a few releases. It may not satisfy the dictates of the FHS religion, but it does meet some people's needs perfectly well. Let's look for a solution that permits coexistence, rather than one that forces change on people who don't need or want change. regards, tom lane ---(end of broadcast)--- TIP 4: Don't 'kill -9' the postmaster
Re: [HACKERS] location of the configuration files
Kevin Brown [EMAIL PROTECTED] writes: So in your case, what's the advantage of having initdb write anything to a config file, when you're probably also relying on PGDATA or -D to start the database (if you're not, then fair enough. But see below)? Keep in mind that initdb doesn't currently *need* to put the datadir location into the config file. It *will* need to do so if we separate config and data dirs. Or at least, *somebody* will need to do so. It's not apparent to me how it simplifies life not to have initdb do it. Especially when there are other configuration items that initdb should or already does record: locale settings, database encoding. And we have already been talking about improving PG's self-tuning capability. initdb would be the natural place to look around for information like available RAM and adjust the config-file settings like sort_mem accordingly. Basically, the notion that initdb shouldn't write a config file seems like a complete dead end to me. It cannot possibly be more convenient than the alternatives. We'd be giving up a lot of current and future functionality --- and for what? I'd expect initdb to initialize a database. If I were running initdb without a lot of foreknowledge of its side effects, I think I'd probably be a bit surprised to find that it had touched my config file. If we do it the way I suggested (dump into the datadir, which is initially empty, same as always) then it cannot overwrite your existing config files. Think of it as providing a suggested config file to compare against what you have. regards, tom lane ---(end of broadcast)--- TIP 4: Don't 'kill -9' the postmaster
Re: [HACKERS] location of the configuration files
Tom Lane wrote: Peter Eisentraut [EMAIL PROTECTED] writes: Tom Lane writes: I would favor a setup that allows a -C *directory* (not file) to be specified as a postmaster parameter separately from the -D directory; A directory is not going to satisfy people. Why not? Who won't it satisfy, and what's their objection? AFAICS, you can either set -C to /etc if you want your PG config files loose in /etc, or you can set it to /etc/postgresql/ if you want them in a privately-owned directory. Which other arrangements are needed? The idea of using a directory puts us back to using symlinks to share files. While I know the core development teams thinks that symlinks are a viable configuration option, most admins, myself included, do not like to use symlinks because they do not have the ability to carry documentation, i.e. comments in a configuration file, and are DANGEROUS in a production environment. Any configuration strategy that depends on symlinks is inadequate and poorly designed. ---(end of broadcast)--- TIP 1: subscribe and unsubscribe commands go to [EMAIL PROTECTED]
Re: [HACKERS] location of the configuration files
mlw [EMAIL PROTECTED] writes: The idea of using a directory puts us back to using symlinks to share files. So? If you want to share files, you're probably sharing all three config files and don't need a separate directory at all. This is not a sufficient argument to make me buy into the mess of letting people choose nonstandard configuration file names --- especially when most of the opposite camp seems to be more interested in choosing *standard* names for things. Why does that policy stop short at the directory name? regards, tom lane ---(end of broadcast)--- TIP 1: subscribe and unsubscribe commands go to [EMAIL PROTECTED]
Re: [HACKERS] location of the configuration files
Tom Lane wrote: Keep in mind that initdb doesn't currently *need* to put the datadir location into the config file. It *will* need to do so if we separate config and data dirs. Or at least, *somebody* will need to do so. It's not apparent to me how it simplifies life not to have initdb do it. Especially when there are other configuration items that initdb should or already does record: locale settings, database encoding. Is it possible for the database engine to properly deal with a database when it is told to use a different database encoding than the one the database was initdb'd with? If it's not, then that suggests to me that the database encoding is something that doesn't belong in the configuration file but rather in some other place that is intimately tied with the database itself and which is difficult/impossible to change, like perhaps a read-only system table that gets created at initdb time. And we have already been talking about improving PG's self-tuning capability. initdb would be the natural place to look around for information like available RAM and adjust the config-file settings like sort_mem accordingly. I agree here, and since you're thinking of just putting the resulting config file in the database data directory, then as a DBA I wouldn't be terribly surprised by it ... especially if it came back with a message that told me what it had done. If we do it the way I suggested (dump into the datadir, which is initially empty, same as always) then it cannot overwrite your existing config files. Think of it as providing a suggested config file to compare against what you have. There is one minor complication: what if there's an existing config file in the target directory? One use for initdb would be as a quick way to completely wipe the database and start over (e.g., if the encoding were found to be incorrect), but the config file that's already there could easily contain a lot of customization that the administrator would want to retain. Which suggests that we should consider writing to a file using a slightly different name (e.g., postgresql.conf.initdb), at least in the event that a config file already exists in the target directory. Not sure what the overall right thing to do here is... -- Kevin Brown [EMAIL PROTECTED] ---(end of broadcast)--- TIP 1: subscribe and unsubscribe commands go to [EMAIL PROTECTED]
Re: [HACKERS] location of the configuration files
mlw wrote: symlinks suck. Sorry Tom, but they are *BAD* in a production server. Well, at least they're better than hard links. ;-) -- Kevin Brown [EMAIL PROTECTED] ---(end of broadcast)--- TIP 2: you can get off all lists at once with the unregister command (send unregister YourEmailAddressHere to [EMAIL PROTECTED])
Re: [HACKERS] location of the configuration files
Tom Lane wrote: mlw [EMAIL PROTECTED] writes: The idea of using a directory puts us back to using symlinks to share files. So? If you want to share files, you're probably sharing all three config files and don't need a separate directory at all. This is not a sufficient argument to make me buy into the mess of letting people choose nonstandard configuration file names --- especially when most of the opposite camp seems to be more interested in choosing *standard* names for things. Why does that policy stop short at the directory name? symlinks suck. Sorry Tom, but they are *BAD* in a production server. You can not add comments to symlinks. Most of the admins I know, myself included, HATE symlinks and use them as a last resort. Requiring symlinks is just pointless, we are talking about a few lines of code hat has nothing to do with performance. The patch that I submitted allows PostgreSQL to work as it always has, but adds the ability for a configuration file to do what is normally done with fixed names in $PGDATA. I have said before, I do not like policy, I like flexibility, forcing a directory is similarly restricting as requiring the files in $PGDATA. Why is this such a problem? MANY people want to configure PostgreSQL this way, but the patch I submitted allows it, but does not force anything. Any configuration solution that requires symlinks is flawed. ---(end of broadcast)--- TIP 2: you can get off all lists at once with the unregister command (send unregister YourEmailAddressHere to [EMAIL PROTECTED])
Re: [HACKERS] location of the configuration files
Martin Coxall wrote: On Thu, 2003-02-13 at 20:28, Steve Crawford wrote: I don't see why we can't keep everyone happy and let the users choose the setup they want. To wit, make the following, probably simple, changes: 1) Have postgresql default to using /etc/postgresql.conf /etc/postgres/postgresql.conf, if we want to be proper FHS-bitches. 2) Add a setting in postgresql.conf specifying the data directory 3) Change the meaning of -D to mean use this config file 4) In the absence of a specified data directory in postgresql.conf, use the location of the postgresql.conf file as the data directory Shouldn't it in that case default to, say /var/lib/postgres? I would really like to push back this whole discussion to adding the ability the flexibility to configure PostgreSQ as opposed to determining a specific configuration strategy. Adding the ability is easy. Let the distros determine their strategy. Trying to enforce one way over another will make this continue on forever and will never be solved. ---(end of broadcast)--- TIP 3: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [HACKERS] location of the configuration files
On Friday 14 February 2003 11:02, Shridhar Daithankar wrote: Especially the whole discussion of PGDATA stuff fails to register as significant IMO. Right now, I can do things the way I want to do and I guess it is pretty much same with everyone else. Is it last topic left to improve? If it weren't significant to a few, then there wouldn't be the traffic. If there's too much traffic, well, there are alternatives. Keep it simple and on tpoic guys. This is hackers. Keep it low volume otherwise, two years down the lines, archives will be unsearchable.. The system configuration of PostgreSQL is on topic for -hackers. IMNSHO. -- Lamar Owen WGCR Internet Radio 1 Peter 4:11 ---(end of broadcast)--- TIP 6: Have you searched our list archives? http://archives.postgresql.org
Re: [HACKERS] location of the configuration files
Tom Lane wrote: Currently, I don't need to do anything more than set PGDATA or say -D to initdb in order to set up the data directory wherever I like. I also don't need to worry about whether I'm selecting the wrong config file. So in your case, what's the advantage of having initdb write anything to a config file, when you're probably also relying on PGDATA or -D to start the database (if you're not, then fair enough. But see below)? I'd expect initdb to initialize a database. If I were running initdb without a lot of foreknowledge of its side effects, I think I'd probably be a bit surprised to find that it had touched my config file. Initdb doesn't have prior knowledge of how you intend to start the database so that it refers to the database initdb just created, so it can't really know whether it's desirable to touch the config file. If it's desirable for initdb to be able to write to the config file, wouldn't it be more appropriate for that to an option that has to be explicitly enabled on initdb's command line? I don't know how often you'd want it to write into the config file for your purposes, but having it do so automatically seems to violate the principle of least surprise. -- Kevin Brown [EMAIL PROTECTED] ---(end of broadcast)--- TIP 1: subscribe and unsubscribe commands go to [EMAIL PROTECTED]
Re: [HACKERS] location of the configuration files
On Friday 14 February 2003 15:10, Tom Lane wrote: I don't see why we don't just let initdb install suggested config files into the new $PGDATA directory, same as it ever did. Ok, let me take another tack. Just exactly why does initdb need to drop any config files anywhere? We provide templates; initdb can initialize the data structure. If we can by default (as part of make install) put the config file templates in $SYSCONFDIR (as set by ./configure), then why does initdb need to retouch them? I say that having configured PostgreSQL like this: (this is for 7.2.4, not 7.3.x) --enable-locale --with-CXX --prefix=/usr --disable-rpath --with-perl --enable-multibyte --with-tcl --with-odbc --enable-syslog --with-python --with-openssl --with-pam --with-krb5=/usr/kerberos --enable-nls --sysconfdir=/etc/pgsql --mandir=/usr/share/man --docdir=/usr/share/doc --includedir=/usr/include --datadir=/usr/share/pgsql So, in my case, it would be preferable to me for initdb to not make a default postgresql.conf, pg_hba.conf, and pg_ident.conf. The make install process should populate sysconfdir (/etc/pgsql here) with those files. Why does initdb even need to be involved now (I know the historical reason)? Comments? -- Lamar Owen WGCR Internet Radio 1 Peter 4:11 ---(end of broadcast)--- TIP 4: Don't 'kill -9' the postmaster
Re: [HACKERS] location of the configuration files
Lamar Owen [EMAIL PROTECTED] writes: Just exactly why does initdb need to drop any config files anywhere? Because we'd like it to edit the correct datadir into the config file, to take just the most obvious example. There has also been a great deal of discussion recently about other things initdb might automatically put into the config file after looking at the system environment. That's not happened yet, but we'd really be restricting ourselves to say that initdb can never customize the config files. regards, tom lane ---(end of broadcast)--- TIP 6: Have you searched our list archives? http://archives.postgresql.org
Re: [HACKERS] location of the configuration files
On Saturday 15 February 2003 20:19, Tom Lane wrote: Lamar Owen [EMAIL PROTECTED] writes: Just exactly why does initdb need to drop any config files anywhere? Because we'd like it to edit the correct datadir into the config file, to take just the most obvious example. Shouldn't we be consistent and have initdb use the datadir set in the config file, which could be supplied by a ./configure switch? There has also been a great deal of discussion recently about other things initdb might automatically put into the config file after looking at the system environment. That's not happened yet, but we'd really be restricting ourselves to say that initdb can never customize the config files. Customize != writing the original. I'm looking at a packager point of view here. The initdb is done well after the package is made, and installed. It would be ideal from this point of view to have things fully configured pre-initdb (and thus pre-packaging). But I understand that this might not be ideal from a multipostmaster point of view. Surely these two can be reconciled. -- Lamar Owen WGCR Internet Radio 1 Peter 4:11 ---(end of broadcast)--- TIP 3: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [HACKERS] location of the configuration files
Lamar Owen [EMAIL PROTECTED] writes: On Saturday 15 February 2003 20:19, Tom Lane wrote: Lamar Owen [EMAIL PROTECTED] writes: Just exactly why does initdb need to drop any config files anywhere? Because we'd like it to edit the correct datadir into the config file, to take just the most obvious example. Shouldn't we be consistent and have initdb use the datadir set in the config file, which could be supplied by a ./configure switch? That'd mean there is no way to perform an initdb into a nonstandard location without first hand-preparing a config file. I don't much care for that. I'm looking at a packager point of view here. The initdb is done well after the package is made, and installed. It would be ideal from this point of view to have things fully configured pre-initdb (and thus pre-packaging). This point of view means that no post-configure knowledge can be applied. We might as well forget the separate initdb step altogether and have make install do it. I realize that from a packager's point of view, the separate initdb step is not very useful. But it is from my point of view. regards, tom lane ---(end of broadcast)--- TIP 1: subscribe and unsubscribe commands go to [EMAIL PROTECTED]
Re: [HACKERS] location of the configuration files
On Saturday 15 February 2003 09:48 am, mlw wrote: Robert Treat wrote: Seems like some are saying one of the problems with the current system is it doesn't follow FHS or LSB. If those are valid reasons to change the system, it seems like a change which doesn't actually address those concerns would not be acceptable. (Unless those really aren't valid concerns...) I did not start this thread to make PostgreSQL FHS compatible, someone else brought that up. As I said somewhere else, I'm an old fashioned UNIX guy, capability without policy. The patch that I submitted for 7.3.2 will allow the user to configure PostgreSQL with a configuration file outside the $PGDATA directory. That's all I care about. If someone wants to get on the FHS bandwagon, that's fine. PostgreSQL should allow that ability but should not require it. If we're going to go through the trouble to change the way things work, we might as well try to get something that will allow instalation to match real desired configurations out there, like FHS and LSB, or how Oliver and Lamar want for packaging without symlinks. If the goal is just to get something that you like, apply the patch locally and be done with it. Robert Treat ---(end of broadcast)--- TIP 4: Don't 'kill -9' the postmaster
Re: [HACKERS] location of the configuration files
On Saturday 15 February 2003 21:06, Tom Lane wrote: Lamar Owen [EMAIL PROTECTED] writes: Shouldn't we be consistent and have initdb use the datadir set in the config file, which could be supplied by a ./configure switch? That'd mean there is no way to perform an initdb into a nonstandard location without first hand-preparing a config file. I don't much care for that. Six of one and half-dozen of another. And that's my real point. We do things quite differently from many other standard services, even those which are built from the ground up for multiple instances. Making things more consistent for admins, even if it's not what we are used to or what we might like (because it's familiar) should at least be thought about. I'm not advocating changing just for the sake of change; but getting a new fresh look at our current setup can't hurt. I'm looking at a packager point of view here. The initdb is done well after the package is made, and installed. It would be ideal from this point of view to have things fully configured pre-initdb (and thus pre-packaging). This point of view means that no post-configure knowledge can be applied. We might as well forget the separate initdb step altogether and have make install do it. I wouldn't complain. Although that isn't conducive to the multiple instance case. The necessary post-configure knowledge would be in the instance postgresql.conf file. One place for it. But this is hypothetical; fishing around the waters here at this point. Realize that my own packages apply an initdb automatically if an install isn't found the first time the system initscript is started. It is virtually automatic. With the multiple postmaster support, creating a couple of files and a symlink (for the initscript), and starting the new initscript symlink does all the dirty work. But it could be easier. I realize that from a packager's point of view, the separate initdb step is not very useful. But it is from my point of view. Would you mind elucidating which point of view is yours? General idea of who else might have the same point of view, and why you find the initdb in its current form to be more useful than alternatives. Again, I'm fishing for knowledge -- if nothing else it gives me an answer to those users who send me nastygrams about the way things are right now. -- Lamar Owen WGCR Internet Radio 1 Peter 4:11 ---(end of broadcast)--- TIP 5: Have you checked our extensive FAQ? http://www.postgresql.org/users-lounge/docs/faq.html
Re: [HACKERS] location of the configuration files
Lamar Owen [EMAIL PROTECTED] writes: I realize that from a packager's point of view, the separate initdb step is not very useful. But it is from my point of view. Would you mind elucidating which point of view is yours? Primarily, one that wants to have multiple postmasters running, of the same or different versions; including test and temporary installations that mustn't conflict with the existing primary installation on a machine. Currently, I don't need to do anything more than set PGDATA or say -D to initdb in order to set up the data directory wherever I like. I also don't need to worry about whether I'm selecting the wrong config file. You're talking about making manual installations significantly more difficult (and error-prone, I think) in order to simplify automated installs. Now you've acknowledged that your script can do what you want it to, and in fact already does. Why is it good to make my life more difficult to make a script's easier? Cycles are cheap. I like to think that my time is worth something. Nor will I buy an argument that only a few developers have need for test installations. Ordinary users will want to do that anytime they are doing preliminary tests on a new PG version before migrating their production database to it. To the extent that you make manual selection of a nonstandard datadir location more difficult and error-prone, you are hurting them too. regards, tom lane ---(end of broadcast)--- TIP 6: Have you searched our list archives? http://archives.postgresql.org
Re: [HACKERS] location of the configuration files
On Sunday 16 February 2003 00:16, Tom Lane wrote: Lamar Owen [EMAIL PROTECTED] writes: Would you mind elucidating which point of view is yours? Primarily, one that wants to have multiple postmasters running, of the same or different versions; including test and temporary installations that mustn't conflict with the existing primary installation on a machine. Well, due to our upgrading difficulty, having multiple versions running has its advantages. You're talking about making manual installations significantly more difficult (and error-prone, I think) in order to simplify automated installs. Now you've acknowledged that your script can do what you want it to, and in fact already does. Why is it good to make my life more difficult to make a script's easier? Cycles are cheap. I like to think that my time is worth something. The script's been out there for awhile. It does some things well, and some things not so well. The config files are still coresident with the database, and backup is more difficult than it can be. Meeting all these needs (with configure switches, configuration file directives, etc) would be a good thing. And that's what I'm after; maximum usability for the maximum audience. I believe pretty strongly that the usage to which you or I would put PostgreSQL is probably quite different from the average user's way of using PostgreSQL. Most probably the typical user has a single installation with multiple databases with little need to run isolated postmasters. Nor will I buy an argument that only a few developers have need for test installations. Ordinary users will want to do that anytime they are doing preliminary tests on a new PG version before migrating their production database to it. To the extent that you make manual selection of a nonstandard datadir location more difficult and error-prone, you are hurting them too. While I'm not going to speak for all users, I know that I don't put a development database system on my production servers. The production machine only runs production servers, period. Hardware is cheap. I have development machines for development databases. One also has the error-prone PATH issues with multiple versions, which, if you are running a typical RPM installation becomes quite difficult to manage, since the RPM version's executables are in /usr/bin. This could be changed; I've thought about changing it. But I'm not sure of the best way to make multiple versions peacefully and seamlessly coexist -- particularly when older versions may not even build on the newer OS: but we've been over that discussion. Care for a poll? -- Lamar Owen WGCR Internet Radio 1 Peter 4:11 ---(end of broadcast)--- TIP 5: Have you checked our extensive FAQ? http://www.postgresql.org/users-lounge/docs/faq.html
Re: [HACKERS] location of the configuration files
Tom Lane wrote: Kevin Brown [EMAIL PROTECTED] writes: I agree with your assessment for the most part, except for PGDATA. There's no good reason I can think of for the postmaster to look at it. The other side of that coin is, what's the good reason to remove it? There's a long way between I don't want my setup to depend on PGDATA and I don't think your setup should be allowed to depend on PGDATA. If you don't want to use it, then don't use it. Why do you need to tell me how I'm allowed to run my installation? I'm not talking about getting rid of ALL dependency on PGDATA in our entire distribution, only postmaster's. Recall that the main purpose of making any of these changes at all is to make life easier for the guys who have to manage the systems that will be running PostgreSQL. Agreed? So: imagine you're the newly-hired DBA and your boss points you to the system and says administrate the database on that. You go over to the computer and start looking around. You do a ps and see a postmaster process running. You know that it's the process that is listening for connections. The ps listing only says /usr/bin/postmaster. No arguments to clue you in, nothing. Where do you look to figure out where the data is? How do you figure out what port it's listening on? Well, we're already agreed on how to deal with that question: you look in /etc/postgresql, and because this is a relatively new install (and the PostgreSQL maintainers, who are very wise and benevolent, made that the default location for configs :-), it has a postgresql.conf file with a line that says data_directory = /var/lib/pgsql. It doesn't mention a port to listen to so you know that it's listening on port 5432. As a DBA, you're all set. Now let's repeat that scenario, except that instead of seeing one postmaster process, you see five. And they all say /usr/bin/postmaster in the ps listing. No arguments to clue you in or anything, as before. You might be able to figure out where one of them is going by looking at /etc/postgresql, but what about the rest? Now you're stuck unless you want to do a find (time consuming and I/O intensive -- a good way to slow the production database down a bit), or you're knowledgeable enough to use 'lsof' or black magic like digging into kernel memory to figure out where the config files and data directories are, or you have enough knowledge to pore through the startup scripts and understand what they're doing. Lest you think that this is an unlikely scenario, keep in mind that most startup scripts, including pg_ctl, currently start the postmaster without arguments and rely on PGDATA, so a shop that hasn't already been bitten by this *will* be. Right now shops that wish to avoid the trap I described have to go to *extra* lengths: they have to make exactly the same kinds of changes to the scripts that I'm talking about us making (putting an explicit '-D $PGDATA' where none now exists) or they have to resort to tricks like renaming the postmaster executable and creating a shell script in its place that will invoke the (renamed) postmaster with '-D $PGDATA'. It's not so bad if only a few shops have to make those changes. But what if it's thousands? Yeah, the distribution guys can patch the scripts to do this, but why should they have to? They, and the shops that run PostgreSQL, are our customers. All of that is made possible because the postmaster can use an inherited PGDATA for the location of the config files and (if the config files don't say differently in our new scheme) the data directory, and pg_ctl takes advantage of that fact (as do most startup scripts that I've seen, that don't just invoke pg_ctl). I'm not arguing that we should remove the use of PGDATA *everywhere*, only in postmaster (and then, only postmaster's use of an *inherited* PGDATA. It should still set PGDATA so its children can use it). It means changing pg_ctl and the startup scripts we ship. The earlier we make these changes, the less overall pain there will be in the long run. ... people who are starting things by hand hopefully aren't so inflexible as to demand that PGDATA remain treated as-is. Yes, I could reconfigure my scripts to not depend on this. You have not given me an adequate argument why I should have to. [By this I'm assuming you're referring to the scripts you use for testing, and not the ones that ship with the distribution] I'm not arguing that you should get rid of all the references to PGDATA in your scripts or anything crazy like that. The changes I'm talking about are minor: where you see postmaster without any -D arguments, you simply add '-D $PGDATA' to it, before any other arguments that you might also be passing. That's it. Nothing else should be needed. The reason for removing postmaster's use of an inherited PGDATA is the same as the reason for making the other changes we already agree should be made: to make things easier for the guys in the field who have to
Re: [MLIST] Re: [HACKERS] location of the configuration files
In reference to determining what port postgres or any program is listening on
On my Redhat Linux machines
netstat --inet -nlp
when run as root
produces a nice list of all programs listening on the network with IP and port
number the process is listening on, the name of the process and the pid.
The environment used to start each of these postmasters can be found at
cat /proc/${POSTMASTER-PID}/environ | tr \000 \n
I'm not arguing one way or the other on your issue, just hope these tips make
the black magic a little easier to use.
On Friday 14 February 2003 04:58 am, Kevin Brown wrote:
Now let's repeat that scenario, except that instead of seeing one
postmaster process, you see five. And they all say
/usr/bin/postmaster in the ps listing. No arguments to clue you
in or anything, as before. You might be able to figure out where one
of them is going by looking at /etc/postgresql, but what about the
rest? Now you're stuck unless you want to do a find (time consuming
and I/O intensive -- a good way to slow the production database down a
bit), or you're knowledgeable enough to use 'lsof' or black magic like
digging into kernel memory to figure out where the config files and
data directories are, or you have enough knowledge to pore through the
startup scripts and understand what they're doing.
---(end of broadcast)---
TIP 4: Don't 'kill -9' the postmaster
Re: [HACKERS] location of the configuration files
Bruce Momjian wrote: If you want ps to display the data dir, you should use -D. Remember, it is mostly important for multiple postmaster, so if you are doing that, just use -D, but don't prevent single-postmaster folks from using PGDATA. Perhaps the best compromise would be to change pg_ctl so that it uses -D explicitly when invoking postmaster. That's an easy change. Could you describe how you and other developers use PGDATA? I'm quite interested in knowing why there seems to be so much resistance to removing the potential_DataDir = getenv(PGDATA); line from postmaster.c. -- Kevin Brown [EMAIL PROTECTED] ---(end of broadcast)--- TIP 2: you can get off all lists at once with the unregister command (send unregister YourEmailAddressHere to [EMAIL PROTECTED])
Re: [HACKERS] location of the configuration files
Bruce Momjian wrote: If you want ps to display the data dir, you should use -D. Remember, it is mostly important for multiple postmaster, so if you are doing that, just use -D, but don't prevent single-postmaster folks from using PGDATA. Perhaps another reasonable approach would be to put an #ifdef/#endif around the potential_DataDir = getenv(PGDATA); line in postmater.c and create a configure option to enable it. That way you guys get the behavior you want for testing but production builds could disable it if that's viewed as desirable. You'd want to make the error message that's produced when no data directory is specified depend on the same #ifdef variable, of course. Then the group would get to fight it out over whether the configure default should be enable or disable. :-) -- Kevin Brown [EMAIL PROTECTED] ---(end of broadcast)--- TIP 1: subscribe and unsubscribe commands go to [EMAIL PROTECTED]
Re: [HACKERS] location of the configuration files
If you are interested in reading a contrary position, you can read Berstein's arguments for his recommended way to install services at: http://cr.yp.to/unix.html But since DJB is a class-A monomaniac, he may not be the best person to listen to. /var/qmail/control for qmail configuration files? Yeah, good one, DJB. -- Martin Coxall [EMAIL PROTECTED] ---(end of broadcast)--- TIP 4: Don't 'kill -9' the postmaster
Re: [HACKERS] location of the configuration files
Generally things that live in /etc are owned and operated by the OS. Postgresql, by it's definition is a userspace program, not an OS owned one. Partially true. The FHS specifies that the /etc top layer is for system-own3d stuff, but the subdirectories off it are explicitly used for user space programs and, well, everything. (/etc/apache, /etc/postgres, /etc/tomcat3, /etc/tomcat4...) Martin Coxall ---(end of broadcast)--- TIP 1: subscribe and unsubscribe commands go to [EMAIL PROTECTED]
Re: [HACKERS] location of the configuration files
On Thu, 2003-02-13 at 20:28, Steve Crawford wrote: I don't see why we can't keep everyone happy and let the users choose the setup they want. To wit, make the following, probably simple, changes: 1) Have postgresql default to using /etc/postgresql.conf /etc/postgres/postgresql.conf, if we want to be proper FHS-bitches. 2) Add a setting in postgresql.conf specifying the data directory 3) Change the meaning of -D to mean use this config file 4) In the absence of a specified data directory in postgresql.conf, use the location of the postgresql.conf file as the data directory Shouldn't it in that case default to, say /var/lib/postgres? -- Martin Coxall [EMAIL PROTECTED] ---(end of broadcast)--- TIP 4: Don't 'kill -9' the postmaster
Re: [HACKERS] location of the configuration files
Bruce Momjian wrote: I just set PGDATA in my login and I don't have to deal with it again. Hmm...you don't use pg_ctl to start/stop/whatever the database? You invoke the postmaster directly (I can easily see that you would, just asking if you do)? -- Kevin Brown [EMAIL PROTECTED] ---(end of broadcast)--- TIP 3: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [HACKERS] location of the configuration files
Bruce Momjian wrote: I just set PGDATA in my login and I don't have to deal with it again. DuhI just realized a reason you guys might care about this so much. It's because you want to be able to start the postmaster from within a debugger (or profiler, or whatever), and you don't want to have to mess with command line options from there, right? Sounds like fixing pg_ctl to use -D explicitly when invoking the postmaster is the right change to make here, since that's probably how the majority of the production shops are going to be starting the database anyway. Takes care of the majority of the visibility problem and leaves PGDATA intact. Thoughts? -- Kevin Brown [EMAIL PROTECTED] ---(end of broadcast)--- TIP 2: you can get off all lists at once with the unregister command (send unregister YourEmailAddressHere to [EMAIL PROTECTED])
Re: [HACKERS] location of the configuration files
On 14 Feb 2003, Martin Coxall wrote: If you are interested in reading a contrary position, you can read Berstein's arguments for his recommended way to install services at: http://cr.yp.to/unix.html But since DJB is a class-A monomaniac, he may not be the best person to listen to. /var/qmail/control for qmail configuration files? Yeah, good one, DJB. I'm guessing that rather than reading it the above mentioned link you chose to waste our time with this instead. Good one, MC. Vince. -- Fast, inexpensive internet service 56k and beyond! http://www.pop4.net/ http://www.meanstreamradio.com http://www.unknown-artists.com Internet radio: It's not file sharing, it's just radio. ---(end of broadcast)--- TIP 3: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [HACKERS] location of the configuration files
On Fri, 2003-02-14 at 14:21, Vince Vielhaber wrote: On 14 Feb 2003, Martin Coxall wrote: If you are interested in reading a contrary position, you can read Berstein's arguments for his recommended way to install services at: http://cr.yp.to/unix.html But since DJB is a class-A monomaniac, he may not be the best person to listen to. /var/qmail/control for qmail configuration files? Yeah, good one, DJB. I'm guessing that rather than reading it the above mentioned link you chose to waste our time with this instead. Good one, MC. Yeah, I've read it several times, and have often linked to it as an example of why one should be wary of DJB's software. It seems to me that since DJB doesn't follow his own advice regarding the filesystem hierarchy (see both qmail and djbdns), it'd be odd for him to expect anyone else to. *Especially* seing as he's a bit mental. (I'm not going to take this any more. I demand cross-platform compatibility!) -- Martin Coxall [EMAIL PROTECTED] ---(end of broadcast)--- TIP 1: subscribe and unsubscribe commands go to [EMAIL PROTECTED]
Re: [HACKERS] location of the configuration files
Bruce Momjian wrote: OK, here is an updated proposal. I think we have decided: Moving postmaster.pid and postmaster.opts isn't worth it. We don't want per-file GUC variables, but assume it is in the same config directory as postgresql.conf. I don't see any valid reason they would want to put them somewhere different than postgresql.conf. So, we add data_dir to postgresql.conf, and add -C/PGCONFIG to postmaster. Agreed. One additional thing: when pg_ctl invokes the postmaster, it should explicitly specify -C on the postmaster command line, and if it doesn't find a data_dir in $PGCONFIG/postgresql.conf then it should explicitly specify a -D as well. Pg_ctl is going to have to be modified to take a -C argument anyway, so we may as well go all the way to do the right thing here. This way, people who start the database using the standard tools we supply will know exactly what's going on when they get a ps listing. -- Kevin Brown [EMAIL PROTECTED] ---(end of broadcast)--- TIP 6: Have you searched our list archives? http://archives.postgresql.org
Re: [HACKERS] location of the configuration files
Bruce Momjian wrote: Give it up. As long as we have -D, we will allow PGDATA. If you don't want to use it, don't use it. Agreed. I'm not sure I see how this diminishes the argument for fixing pg_ctl so that it passes an explicit -D option to the postmaster when invoking it... -- Kevin Brown [EMAIL PROTECTED] ---(end of broadcast)--- TIP 4: Don't 'kill -9' the postmaster
Re: [HACKERS] location of the configuration files
Bruce Momjian wrote: This way, people who start the database using the standard tools we supply will know exactly what's going on when they get a ps listing. No. If you want ps to display, don't use environment variables. Many don't care --- especially those with only one postmaster. You know that the code in pg_ctl doesn't send an explicit -D to the postmaster even if pg_ctl itself is invoked with a -D argument, right? The only way to make pg_ctl do that is by using the -o option. A typical vendor-supplied install is going to invoke pg_ctl to do the dirty work. That's why I'm focusing on pg_ctl. I completely understand your need for keeping PGDATA in postmaster. I don't understand why pg_ctl *shouldn't* be changed to invoke postmaster with an explicit -D option. It might be desirable for ps to not show any arguments to postmaster in some circumstances (I have no idea what those would be), but why in the world would you want that to be the *default*? Why would we want the default behavior to make things harder on administrators and not easier? -- Kevin Brown [EMAIL PROTECTED] ---(end of broadcast)--- TIP 5: Have you checked our extensive FAQ? http://www.postgresql.org/users-lounge/docs/faq.html
Re: [HACKERS] location of the configuration files
Kevin Brown [EMAIL PROTECTED] writes: Tom Lane wrote: The other side of that coin is, what's the good reason to remove it? There's a long way between I don't want my setup to depend on PGDATA and I don't think your setup should be allowed to depend on PGDATA. If you don't want to use it, then don't use it. Why do you need to tell me how I'm allowed to run my installation? I'm not talking about getting rid of ALL dependency on PGDATA in our entire distribution, only postmaster's. We're obviously talking past each other. You are arguing that under circumstances X, Y, or Z, depending on a PGDATA setting is a bad idea. You are then drawing the conclusion that I should not be allowed to depend on PGDATA, whether or not I care about X, Y, or Z. I am happy to design an arrangement that allows you not to depend on PGDATA if you don't want to. But I don't see why you need to break my configuration procedures in order to fix yours. As I outlined last night, it's possible to do what you want without breaking backwards compatibility for those that like PGDATA. regards, tom lane ---(end of broadcast)--- TIP 1: subscribe and unsubscribe commands go to [EMAIL PROTECTED]
Re: [HACKERS] location of the configuration files
On 14 Feb 2003, Martin Coxall wrote: On Fri, 2003-02-14 at 14:21, Vince Vielhaber wrote: On 14 Feb 2003, Martin Coxall wrote: If you are interested in reading a contrary position, you can read Berstein's arguments for his recommended way to install services at: http://cr.yp.to/unix.html But since DJB is a class-A monomaniac, he may not be the best person to listen to. /var/qmail/control for qmail configuration files? Yeah, good one, DJB. I'm guessing that rather than reading it the above mentioned link you chose to waste our time with this instead. Good one, MC. Yeah, I've read it several times, and have often linked to it as an example of why one should be wary of DJB's software. It seems to me that since DJB doesn't follow his own advice regarding the filesystem hierarchy (see both qmail and djbdns), it'd be odd for him to expect anyone else to. *Especially* seing as he's a bit mental. (I'm not going to take this any more. I demand cross-platform compatibility!) I seriously doubt your ability to judge anyone's mental stability. I can also see that you prefer cross-platform INcompatibility. Your position and mindset are now crystal clear. Vince. -- Fast, inexpensive internet service 56k and beyond! http://www.pop4.net/ http://www.meanstreamradio.com http://www.unknown-artists.com Internet radio: It's not file sharing, it's just radio. ---(end of broadcast)--- TIP 5: Have you checked our extensive FAQ? http://www.postgresql.org/users-lounge/docs/faq.html
Re: [HACKERS] location of the configuration files
Tom Lane wrote: I am happy to design an arrangement that allows you not to depend on PGDATA if you don't want to. But I don't see why you need to break my configuration procedures in order to fix yours. As I outlined last night, it's possible to do what you want without breaking backwards compatibility for those that like PGDATA. Yes, I agree. I hadn't really thought of all the possible benefits of PGDATA. Sorry. :-( Would you agree that it would be a beneficial change to have pg_ctl pass explicit arguments to postmaster? It would go a long way towards eliminating most of the situations I described. A warning in the documentation about the consequences of using PGDATA might not be a bad idea, either... -- Kevin Brown [EMAIL PROTECTED] ---(end of broadcast)--- TIP 3: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [HACKERS] location of the configuration files
Kevin Brown [EMAIL PROTECTED] writes: I'm quite interested in knowing why there seems to be so much resistance to removing the potential_DataDir = getenv(PGDATA); line from postmaster.c. Backwards compatibility. Also, you still haven't explained why I don't want to use PGDATA should translate to no one should be allowed to use PGDATA. If you don't set PGDATA, what problem is there for you in that line being there? regards, tom lane ---(end of broadcast)--- TIP 6: Have you searched our list archives? http://archives.postgresql.org
Re: [HACKERS] location of the configuration files
Martin Coxall [EMAIL PROTECTED] writes: Partially true. The FHS specifies that the /etc top layer is for system-own3d stuff, but the subdirectories off it are explicitly used for user space programs and, well, everything. (/etc/apache, /etc/postgres, /etc/tomcat3, /etc/tomcat4...) FHS or no FHS, I would think that the preferred arrangement would be to keep Postgres' config files in a postgres-owned subdirectory, not directly in /etc. That way you need not be root to edit them. (My idea of an editor, Emacs, always wants to write a backup file, so I dislike having to edit files that live in directories I can't write.) Here's a pretty topic for a flamewar: should it be /etc/postgres/ or /etc/postgresql/ ? regards, tom lane ---(end of broadcast)--- TIP 3: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [HACKERS] location of the configuration files
On Thu, 2003-02-13 at 19:22, Adam Haberlach wrote: On Thu, Feb 13, 2003 at 05:59:17PM -0500, Robert Treat wrote: On Thu, 2003-02-13 at 15:08, mlw wrote: Stephan Szabo wrote: On Thu, 13 Feb 2003, mlw wrote: Personally I think a postgresql installation is much more like an apache installation, which generally contains all of the files (data and config) under /usr/local/apache. Maybe someone can dig more to see if that system is more appropriate a comparison than something like bind. I think you are making a pretty uninformed, if not just plain wrong generalization. I've run exactly one system with apache configuration files in /usr/local/apache, and even then, the data was not there. Uh... the last time I built apache from source, it stuck everything under /usr/local/apache. It uses a conf directory for the config files, and htdocs for the data files... That is it's default configuration. snip stories of all the different ways people run apache You know, this is why I actually suggested looking closer at apache. By default, everything is crammed in one directory, but if you want to, you can configure it six different ways to sunday. That seems to be a big plus IMO What does this mean? People will put things in different places, and there are typically very good reasons for this. This is ESPECIALLY true when one wants to have configuration files, at least the base ones in a common place such as /etc or /usr/local/etc in order to make backup of configuration easy and clean, while leaving data somewhere else for performance or magnitude of partition reasons. It just makes sense to ME to have postgresql.conf reside in /etc, yet put my data in /var/data/postgresql, yet retain the option to put my data in /raid/data/postgresql at a later date, when the new hardware comes in. Is anyone arguing against this? I'm certainly not. But maybe my needs are more varied than yours. On my local development box, I run multiple versions of apache, compiled with different versions of php. It really helps to keep all of apache's stuff centralized, and using things like rpms actually overly complicates this. Now sure, that's a development machine, but on the phppgadmin demo server, which is essentially a production system, I run three different versions of postgresql. In fact, I need to upgrade two of those (to 7.2.4 and 7.3.2), I shudder to think about doing that if postgresql forced me to use the /etc/ directory for all of my config files. Now sure, this probably isn't typical use, but I would say that when it comes time to upgrade major versions, unless you running an operation where you can have large amounts of downtime, postgresql needs to have the ability to have multiple versions install that don't conflict with each other, and it needs to do this easily. The upgrade process is hard enough already. snip However, this seems, to me, to be a very small addition that has some real-world (and yes, we need to start paying attention to the real world) advantages. And finally, don't go telling me that I'm wrong to put my data and config files where I am. You can offer advice, but I'm probably going to ignore it because I like where they are and don't need to explain why. Have I wronged you in some former life? I've very little concern for where you put your data files, and have no idea why you'd think I'd criticize your setup. Robert Treat ---(end of broadcast)--- TIP 5: Have you checked our extensive FAQ? http://www.postgresql.org/users-lounge/docs/faq.html
Re: [HACKERS] location of the configuration files
Bruce Momjian [EMAIL PROTECTED] writes:
My point is that folks with multiple postmasters may not want to use
PGDATA, but for folks who have single postmasters, it makes things
easier and less error-prone.
Actually, for multi postmasters too. I have little shell-environment
config files that switch my entire world view between different source
trees and installation trees, for example this one sets me up to mess
with the 7.2 branch:
STDPATH=${STDPATH:-$PATH}
STDMANPATH=${STDMANPATH:-$MANPATH}
PGSRCROOT=$HOME/REL7_2/pgsql
PGINSTROOT=$HOME/version72
PATH=$PGINSTROOT/bin:/opt/perl5.6.1/bin:$STDPATH
MANPATH=$PGINSTROOT/man:$STDMANPATH
PGLIB=$PGINSTROOT/lib
PGDATA=$PGINSTROOT/data
PMOPTIONS=-p 5472 -i -F
PMLOGFILE=server72.log
export PGSRCROOT PGINSTROOT PATH MANPATH PGLIB PGDATA STDPATH STDMANPATH
export PMOPTIONS PMLOGFILE
After sourcing one of these, I can use pg_ctl as well as a half dozen
other convenient little scripts that do things like remake and reinstall
the backend:
#!/bin/sh
pg_ctl -w stop
cd $PGSRCROOT/src/backend
make install-bin
startpg
or this one that fires up gdb on a crashed backend:
#!/bin/sh
# Usage: gdbcore
cd $HOME
CORES=`find $PGDATA/base -name core -type f -print`
if [ x$CORES != x ]
then
ls -l $CORES
fi
if [ `echo $CORES | wc -w` -eq 1 ]
then
exec gdb $PGINSTROOT/bin/postgres $CORES
else
exec gdb $PGINSTROOT/bin/postgres
fi
This is vastly less error-prone than keeping track of the various
related elements in my head.
Now, it's certainly true that I could still make this work if I had
to explicitly say -D $PGDATA to the postmaster. But that would clutter
my ps display. I am happy with -p as the ps indicator of which
postmaster is which; I don't want more stuff in there.
regards, tom lane
---(end of broadcast)---
TIP 1: subscribe and unsubscribe commands go to [EMAIL PROTECTED]
Re: [HACKERS] location of the configuration files
On Friday 14 Feb 2003 9:05 pm, you wrote: Martin Coxall [EMAIL PROTECTED] writes: Here's a pretty topic for a flamewar: should it be /etc/postgres/ or /etc/postgresql/ ? I vote for /etc/pgsql. Keeping in line of unix philosophy of cryptic and short names. Who wants a descriptive names anyway..:-) Seriously, the traffic on last three days ahd very high noise ratio. Especially the whole discussion of PGDATA stuff fails to register as significant IMO. Right now, I can do things the way I want to do and I guess it is pretty much same with everyone else. Is it last topic left to improve? Keep it simple and on tpoic guys. This is hackers. Keep it low volume otherwise, two years down the lines, archives will be unsearchable.. Shridhar ---(end of broadcast)--- TIP 3: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [HACKERS] location of the configuration files
On Thu, 13 Feb 2003, mlw wrote: scott.marlowe wrote: These are not issues at all. You could put the configuration file anywhere, just as you can for any UNIX service. postmaster --config=/home/myhome/mydb.conf I deal with a number of PG databases on a number of sites, and it is a real pain in the ass to get to a PG box and hunt around for data directory so as to be able to administer the system. What's really annoying is when you have to find the data directory when someone else set up the system. Really? I would think it's easier to do this: su - pgsuper cd $PGDATA pwd Than to try to figure out what someone entered when they ran ./configure --config=... Why do you think PGDATA would be set for root? Did you not notice the su - pgsuper line above? You know, the one where you become the account that runs that instance of the database. Again, I ask you, isn't that easier than trying to find out what someone typed when they typed ./configure --config=? Configuring postgresql via a configuration file which specifies all the data, i.e. data directory, name of other configuration files, etc. is the right way to do it. Even if you have reasons against it, even if you think it is a bad idea, a bad standard is almost always a better solution than an arcane work of perfection. Wrong, I strongly disagree with this sentament. Conformity to standards for simple conformity's sake is as wrong as sticking to the old way because it's what we're all comfy with. It isn't conformity for conformitys sake. It is following an established practice, like driving on the same side of the road or stopping at red lights. But this isn't the same thing at all. Apache, when built from a tar ball, goes into /usr/local/apache/ and ALL it's configuration files are there. When installed as a package, my OS manufacturer decides where that goes. Those are the two standard ways of doing things. I like that postgresql installs into the /usr/local/pgsql directory from a tar ball. I like the fact that it uses $PGDATA to tell it where the cluster is, so that all my scripts, like pg_ctl, just know where it is without a -D switch each time. ---(end of broadcast)--- TIP 2: you can get off all lists at once with the unregister command (send unregister YourEmailAddressHere to [EMAIL PROTECTED])
Re: [HACKERS] location of the configuration files
On Fri, 14 Feb 2003, Curt Sampson wrote: On Thu, 13 Feb 2003, scott.marlowe wrote: If I do a .tar.gz install of apache, I get /usr/local/apache/conf, which is not the standard way you're listing. I'm going to stay out of this argument from now on, but this struck a sore point. /usr is designed to be a filesystem that can be shared. Is the stuff in /usr/local/apache/conf really supposed to be shared amongst all machines of that architecture on your site that run apache? Interesting. I've always viewed usr EXCEPT for local this way. In fact, on most of my boxes I create a seperate mount point for /usr/local so it's easier to backup and maintain, and it doesn't fill up the /usr directory. Asking for everything in a directory with the name local in it to be shared is kind of counter intuitive to me. ---(end of broadcast)--- TIP 2: you can get off all lists at once with the unregister command (send unregister YourEmailAddressHere to [EMAIL PROTECTED])
Re: [HACKERS] location of the configuration files
Bruce Momjian wrote: The big question is whether PGDATA is still our driving config variable, and PGCONFIG/-C is just an additional option, or whether we are moving in a direction where PGCONFIG/-C is going to be the driving value, and data_dir is going to be read as part of that. I'm actually leaning towards PGCONFIG + PGDATA. Yeah, it may be a surprise given my previous arguments, but I can't help but think that the advantages you get with PGDATA will also exist for PGCONFIG. My previous arguments for removing PGDATA from postmaster can be dealt with by fixing pg_ctl to use explicit command line directives when invoking postmaster -- no changes to postmaster needed. PGCONFIG would be no different in that regard. Sorry if I seem a big gung-ho on the administrator point of view, but as a system administrator myself I understand and feel their pain. :-) -- Kevin Brown [EMAIL PROTECTED] ---(end of broadcast)--- TIP 5: Have you checked our extensive FAQ? http://www.postgresql.org/users-lounge/docs/faq.html
Re: [HACKERS] location of the configuration files
On Thu, Feb 13, 2003 at 11:53:26 -0500, mlw [EMAIL PROTECTED] wrote: Where, specificaly are his arguements against a configuration file methodology? I don't think he is argueing against a configuration methodology, but rather against the methodology being used in Unix distributions. In particular he doesn't file the Linux File Standard because it puts the same software in different places depending on whether the vendor or using installed it. ---(end of broadcast)--- TIP 4: Don't 'kill -9' the postmaster
Re: [HACKERS] location of the configuration files
scott.marlowe [EMAIL PROTECTED] writes:
But this isn't the same thing at all. Apache, when built from a tar ball,
goes into /usr/local/apache/ and ALL it's configuration files are there.
Two comments:
1) Even in that case the config files go into /usr/local/apache/conf and the
other kinds of files like data logs and cache files, all go in other
subdirectories.
2) What you describe is only true if you configure with the default
--with-layout=Apache. The naming should perhaps be a clue that this isn't
a conventional layout. If you configure with --with-layout=GNU you get the
conventional Unix layout in /usr/local, If you use --with-layout=RedHat you
get the conventional layout in /usr directly which is mainly useful for
distribution packagers.
Putting stuff in a subdirectory like /usr/local/apache or /usr/local/pgsql is
unfortunately a widespread practice. It does have some advantages over the
conventional layout in /usr/local/{etc,bin,...} directly. But the major
disadvantage is that users can't run programs without adding dozens of entries
to their paths, can't compile programs without dozens of -L and -I lines, etc.
GNU autoconf script makes it pretty easy to configure packages to work either
though, and /usr/local is the purview of the local admin. As long as it's easy
to configure postgres to install properly with --prefix=/usr/local it won't
be any more of an offender than lots of other packages like apache, kde, etc.
Though I'll mention, please make it $prefix/etc not $prefix/conf. No need to
be gratuitously non-standard on an arbitrary name, and no need to pollute
/usr/local with multiple redundant directories.
--
greg
---(end of broadcast)---
TIP 2: you can get off all lists at once with the unregister command
(send unregister YourEmailAddressHere to [EMAIL PROTECTED])
Re: [HACKERS] location of the configuration files
On Fri, Feb 14, 2003 at 02:58:49 -0800, Kevin Brown [EMAIL PROTECTED] wrote: Lest you think that this is an unlikely scenario, keep in mind that most startup scripts, including pg_ctl, currently start the postmaster without arguments and rely on PGDATA, so a shop that hasn't already been bitten by this *will* be. Right now shops that wish to avoid the trap I described have to go to *extra* lengths: they have to make exactly the same kinds of changes to the scripts that I'm talking about us making (putting an explicit '-D $PGDATA' where none now exists) or they have to resort to tricks like renaming the postmaster executable and creating a shell script in its place that will invoke the (renamed) postmaster with '-D $PGDATA'. On at least some systems ps will dump process' environment and could be easily used to check PGDATA. ---(end of broadcast)--- TIP 4: Don't 'kill -9' the postmaster
Re: [HACKERS] location of the configuration files
Bruce Momjian wrote: I see your point --- pg_ctl does a PGDATA trick when passed -D: -D) shift # pass environment into new postmaster PGDATA=$1 export PGDATA It should pass -D just like it was given. Yes, exactly. Now, the more interesting question in my mind is: if pg_ctl isn't passed -D but inherits PGDATA, should it nonetheless pass -D explicitly to the postmaster? We can make it do that, and it would have the benefit of making transparent what would otherwise be opaque. I'm inclined to answer yes to that question, but only because someone who *really* doesn't want the postmaster to show up with a -D argument in ps can start the postmaster directly without using pg_ctl at all. Tom made a good argument for sometimes wanting to keep the ps output clean, but it's not clear to me that it should necessarily apply to pg_ctl. But you guys might have a different perspective on that. :-) -- Kevin Brown [EMAIL PROTECTED] ---(end of broadcast)--- TIP 4: Don't 'kill -9' the postmaster
Re: [HACKERS] location of the configuration files
On Friday 14 February 2003 6:07 am, Martin Coxall wrote: On Thu, 2003-02-13 at 20:28, Steve Crawford wrote: I don't see why we can't keep everyone happy and let the users choose the setup they want. To wit, make the following, probably simple, changes: 1) Have postgresql default to using /etc/postgresql.conf /etc/postgres/postgresql.conf, if we want to be proper FHS-bitches. 2) Add a setting in postgresql.conf specifying the data directory 3) Change the meaning of -D to mean use this config file 4) In the absence of a specified data directory in postgresql.conf, use the location of the postgresql.conf file as the data directory Shouldn't it in that case default to, say /var/lib/postgres? Idea 4 was just a way to preserve current behaviour for those who desire. Moving postgresql.conf requires adding the data directory info into postgresql.conf or specifying it in some other way. If, in the absence of any specification in postgresql.conf, postgres just looks in the same directory as postgresql.conf then it will be almost identical to the current setup. Cheers, Steve ---(end of broadcast)--- TIP 2: you can get off all lists at once with the unregister command (send unregister YourEmailAddressHere to [EMAIL PROTECTED])
Re: [HACKERS] location of the configuration files
I don't want to over-engineer this. Propogating -D into postmaster makes sense, but grabbing PGDATA doesn't to me. --- Kevin Brown wrote: Bruce Momjian wrote: I see your point --- pg_ctl does a PGDATA trick when passed -D: -D) shift # pass environment into new postmaster PGDATA=$1 export PGDATA It should pass -D just like it was given. Yes, exactly. Now, the more interesting question in my mind is: if pg_ctl isn't passed -D but inherits PGDATA, should it nonetheless pass -D explicitly to the postmaster? We can make it do that, and it would have the benefit of making transparent what would otherwise be opaque. I'm inclined to answer yes to that question, but only because someone who *really* doesn't want the postmaster to show up with a -D argument in ps can start the postmaster directly without using pg_ctl at all. Tom made a good argument for sometimes wanting to keep the ps output clean, but it's not clear to me that it should necessarily apply to pg_ctl. But you guys might have a different perspective on that. :-) -- Kevin Brown [EMAIL PROTECTED] ---(end of broadcast)--- TIP 4: Don't 'kill -9' the postmaster -- Bruce Momjian| http://candle.pha.pa.us [EMAIL PROTECTED] | (610) 359-1001 + If your life is a hard drive, | 13 Roberts Road + Christ can be your backup.| Newtown Square, Pennsylvania 19073 ---(end of broadcast)--- TIP 5: Have you checked our extensive FAQ? http://www.postgresql.org/users-lounge/docs/faq.html
Re: [HACKERS] location of the configuration files
On Fri, 14 Feb 2003, scott.marlowe wrote: Asking for everything in a directory with the name local in it to be shared is kind of counter intuitive to me. Not really. If you install a particular program that doesn't come with the OS on one machine on your site, why would you not want to install it separately on all of the others? Typically, I want my favourite non-OS utilities on all machines, not just one. (Even if I don't use them on all machines.) Thus /usr/local is for site-local stuff. cjs -- Curt Sampson [EMAIL PROTECTED] +81 90 7737 2974 http://www.netbsd.org Don't you know, in this new Dark Age, we're all light. --XTC ---(end of broadcast)--- TIP 6: Have you searched our list archives? http://archives.postgresql.org
Re: [HACKERS] location of the configuration files
Peter Eisentraut [EMAIL PROTECTED] writes: Tom Lane writes: I would favor a setup that allows a -C *directory* (not file) to be specified as a postmaster parameter separately from the -D directory; A directory is not going to satisfy people. Why not? Who won't it satisfy, and what's their objection? AFAICS, you can either set -C to /etc if you want your PG config files loose in /etc, or you can set it to /etc/postgresql/ if you want them in a privately-owned directory. Which other arrangements are needed? regards, tom lane ---(end of broadcast)--- TIP 3: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [HACKERS] location of the configuration files
Tom Lane writes:
I would favor a setup that allows a -C *directory* (not file) to be
specified as a postmaster parameter separately from the -D directory;
A directory is not going to satisfy people.
I don't see any great value in a separate postgresql.conf parameter for
each secondary config file; that just means clutter to me,
Not to other people.
1. No -C switch, no -D switch, no PGDATA found in environment: seek
postgresql.conf in the default -C directory established at configure
time. Use the 'datadir' specified therein as -D. Fail if postgresql.conf
doesn't define a datadir value.
OK.
2. No -C switch, no -D switch, PGDATA found in environment: use $PGDATA
as both -C and -D.
This behavior would be pretty inconsistent. But maybe it's the best we
can do.
3. No -C switch, -D switch on command line: use -D value as both -C and -D,
proceed as in case 2.
Same as above.
4. -C switch, no -D switch on command line: seek postgresql.conf in
-C directory, use the datadir it specifies.
OK.
5. -C and -D on command line: seek postgresql.conf in -C directory,
use -D as datadir overriding what is in postgresql.conf (this is just
the usual rule that command line switches override postgresql.conf).
But that usual rule seems to be in conflict with cases 2 and 3 above.
(The usual rule is that a command-line option overrides a postgresql.conf
parameter. The rule in 3, for example is, that a command-line option (the
same one!) overrides where postgresql.conf is in the first place.)
I would venture that the configure-time-default for -C should be
${prefixdir}/etc if configure is not told differently,
Yeah, we already have that as --sysconfdir.
--
Peter Eisentraut [EMAIL PROTECTED]
---(end of broadcast)---
TIP 3: if posting/reading through Usenet, please send an appropriate
subscribe-nomail command to [EMAIL PROTECTED] so that your
message can get through to the mailing list cleanly
Re: [HACKERS] location of the configuration files
Tom Lane wrote: Bruce Momjian [EMAIL PROTECTED] writes: I think the big question is whether we want the default to install the configs in a separate directory, pgsql/etc, or just allow the specification of a separate location. Advantages of pgsql/etc are initdb-safe, and easier backups. I don't see why we don't just let initdb install suggested config files into the new $PGDATA directory, same as it ever did. Then (as long as we don't use relative paths in the config files) people can move them somewhere else if they like, or not if they prefer not to. Adding more mechanism than that just adds complexity without buying much (except the possibility of initdb overwriting your old config files, which is exactly what I thought we wanted to avoid). The big question is whether PGDATA is still our driving config variable, and PGCONFIG/-C is just an additional option, or whether we are moving in a direction where PGCONFIG/-C is going to be the driving value, and data_dir is going to be read as part of that. I thought the idea was to allow both approaches. We are not moving in the direction of one or the other, we are giving people a choice of how they want to drive it. That's where I am unsure. Is the initdb-safe and backup advantages enough to start to migrate those out to data/? I need to hear comments on that. One new idea is to move the config files into data/etc. That makes it clear which are config files, and makes backup a little easier. It would make -C more logical because you are not moving a clear directory. -- Bruce Momjian| http://candle.pha.pa.us [EMAIL PROTECTED] | (610) 359-1001 + If your life is a hard drive, | 13 Roberts Road + Christ can be your backup.| Newtown Square, Pennsylvania 19073 ---(end of broadcast)--- TIP 4: Don't 'kill -9' the postmaster
Re: [HACKERS] location of the configuration files
Kevin Brown wrote: Tom Lane wrote: Kevin Brown [EMAIL PROTECTED] writes: I agree with your assessment for the most part, except for PGDATA. There's no good reason I can think of for the postmaster to look at it. The other side of that coin is, what's the good reason to remove it? There's a long way between I don't want my setup to depend on PGDATA and I don't think your setup should be allowed to depend on PGDATA. If you don't want to use it, then don't use it. Why do you need to tell me how I'm allowed to run my installation? I'm not talking about getting rid of ALL dependency on PGDATA in our entire distribution, only postmaster's. Recall that the main purpose of making any of these changes at all is to make life easier for the guys who have to manage the systems that will be running PostgreSQL. Agreed? So: imagine you're the newly-hired DBA and your boss points you to the system and says administrate the database on that. You go over to the computer and start looking around. You do a ps and see a postmaster process running. You know that it's the process that is listening for connections. The ps listing only says /usr/bin/postmaster. No arguments to clue you in, nothing. Where do you look to figure out where the data is? How do you figure out what port it's listening on? If you want ps to display the data dir, you should use -D. Remember, it is mostly important for multiple postmaster, so if you are doing that, just use -D, but don't prevent single-postmaster folks from using PGDATA. -- Bruce Momjian| http://candle.pha.pa.us [EMAIL PROTECTED] | (610) 359-1001 + If your life is a hard drive, | 13 Roberts Road + Christ can be your backup.| Newtown Square, Pennsylvania 19073 ---(end of broadcast)--- TIP 6: Have you searched our list archives? http://archives.postgresql.org
Re: [HACKERS] location of the configuration files
OK, here is an updated proposal. I think we have decided: Moving postmaster.pid and postmaster.opts isn't worth it. We don't want per-file GUC variables, but assume it is in the same config directory as postgresql.conf. I don't see any valid reason they would want to put them somewhere different than postgresql.conf. So, we add data_dir to postgresql.conf, and add -C/PGCONFIG to postmaster. Regarding Tom's idea of replacing data_dir with a full path during initdb, I think we are better having it be relative to the config directory, that way if they move pgsql/, the system still works. However, if the config directory is in a different lead directory path, we should replace it with the full path, e.g. /usr/local/pgsql/data and /usr/local/pgsql/etc use relative paths, ../data, while /etc/postgresql and /usr/local/pgsql/data get an absolute path. My idea is to introduce the above capabilities in 7.4, and keep the config files in /data. This will allow package folks to move the config files in 7.4. I also think we should start telling people to use PGCONFIG rather than PGDATA. Then, in 7.5, we move the default config file location to pgsql/etc, and tell folks to point there rather than /data. I think there is major value to getting those config files out of the initdb creation area and for backups. I am now wondering if we should add PGCONFIG and move them out of data all in the same release. Not sure if delaying the split is valuable. --- Tom Lane wrote: Lamar Owen [EMAIL PROTECTED] writes: Ok, if we're going this far already, tell me exactly why we have three config files. Why not really Unify things and fulfil the full promise of Grand Unified Configuration by rolling hba and ident into postgresql.conf. Is there a compelling reason not to do so? Lack of backwards compatibility; unnecessary complexity. Unifying those files would create a big headache in terms of having to unify their syntax. And there are some basic semantic differences too. For instance, order matters in pg_hba.conf, but not in postgresql.conf. Another reason not to do it is that there are differences in the security requirements of these files. postgresql.conf probably doesn't contain anything that needs to be hidden from prying eyes, but I'd be inclined to want to keep the other two mode 600. --- Okay, I've been laying low all day, but here are my thoughts on the discussion: I do see the value in being able to (as opposed to being forced to, please) keep hand-edited config files in a separate location from the machine-processed data files. We have already gone some distance in that direction over the past few releases --- there's much less in the top $PGDATA directory than there once was. It makes sense to let people keep hand-edited files away from what initdb will overwrite. I would favor a setup that allows a -C *directory* (not file) to be specified as a postmaster parameter separately from the -D directory; then the hand-editable config files would be sought in -C not -D. In the absence of -C the config files should be sought in -D, same as they ever were (thus simplifying life for people like me who run many postmasters and don't give a darn about FHS ;-)). I don't see any great value in a separate postgresql.conf parameter for each secondary config file; that just means clutter to me, especially if we add more such files in future. I am also distinctly not in favor of eliminating the PGDATA environment variable; that reads to me as we are going to force you to do it our way rather than the way you've always done it, even if you like the old way. To make the RPM packagers happy, I guess that the default -C directory has to be settable via configure. We do not currently have a default -D directory, and I didn't hear anyone arguing in favor of adding one. So that leaves the following possible combinations that the postmaster might see at startup, for which I propose the following behaviors: 1. No -C switch, no -D switch, no PGDATA found in environment: seek postgresql.conf in the default -C directory established at configure time. Use the 'datadir' specified therein as -D. Fail if postgresql.conf doesn't define a datadir value. 2. No -C switch, no -D switch, PGDATA found in environment: use $PGDATA as both -C and -D. (Minor detail: if the postgresql.conf in the $PGDATA directory specifies a different directory as datadir, do we follow that or raise an error? I'd be inclined to say follow it but maybe there is an argument for erroring out.) (In all the following cases, any environment PGDATA value is ignored.) 3. No -C switch, -D switch on command line: use -D value as both -C and -D, proceed as in case 2. 4. -C switch, no -D switch on command line: seek postgresql.conf in -C directory, use the datadir it
Re: [HACKERS] location of the configuration files
On Fri, 2003-02-14 at 12:17, Bruce Momjian wrote: If you want ps to display the data dir, you should use -D. Remember, it is mostly important for multiple postmaster, so if you are doing that, just use -D, but don't prevent single-postmaster folks from using PGDATA. Could not the ps line be rewritten to show this, as the backend's ps lines are rewritten? -- Oliver Elphick[EMAIL PROTECTED] Isle of Wight, UK http://www.lfix.co.uk/oliver GPG: 1024D/3E1D0C1C: CA12 09E0 E8D5 8870 5839 932A 614D 4C34 3E1D 0C1C God be merciful unto us, and bless us; and cause his face to shine upon us. Psalms 67:1 ---(end of broadcast)--- TIP 3: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [HACKERS] location of the configuration files
I am not sure if it is a good idea to be mucking with it. For backend, we do the entire thing, so it is clear we modified something. --- Oliver Elphick wrote: On Fri, 2003-02-14 at 12:17, Bruce Momjian wrote: If you want ps to display the data dir, you should use -D. Remember, it is mostly important for multiple postmaster, so if you are doing that, just use -D, but don't prevent single-postmaster folks from using PGDATA. Could not the ps line be rewritten to show this, as the backend's ps lines are rewritten? -- Oliver Elphick[EMAIL PROTECTED] Isle of Wight, UK http://www.lfix.co.uk/oliver GPG: 1024D/3E1D0C1C: CA12 09E0 E8D5 8870 5839 932A 614D 4C34 3E1D 0C1C God be merciful unto us, and bless us; and cause his face to shine upon us. Psalms 67:1 -- Bruce Momjian| http://candle.pha.pa.us [EMAIL PROTECTED] | (610) 359-1001 + If your life is a hard drive, | 13 Roberts Road + Christ can be your backup.| Newtown Square, Pennsylvania 19073 ---(end of broadcast)--- TIP 5: Have you checked our extensive FAQ? http://www.postgresql.org/users-lounge/docs/faq.html
Re: [HACKERS] location of the configuration files
Kevin Brown wrote: Bruce Momjian wrote: If you want ps to display the data dir, you should use -D. Remember, it is mostly important for multiple postmaster, so if you are doing that, just use -D, but don't prevent single-postmaster folks from using PGDATA. Perhaps the best compromise would be to change pg_ctl so that it uses -D explicitly when invoking postmaster. That's an easy change. Could you describe how you and other developers use PGDATA? I'm quite interested in knowing why there seems to be so much resistance to removing the potential_DataDir = getenv(PGDATA); line from postmaster.c. I just set PGDATA in my login and I don't have to deal with it again. -- Bruce Momjian| http://candle.pha.pa.us [EMAIL PROTECTED] | (610) 359-1001 + If your life is a hard drive, | 13 Roberts Road + Christ can be your backup.| Newtown Square, Pennsylvania 19073 ---(end of broadcast)--- TIP 6: Have you searched our list archives? http://archives.postgresql.org
Re: [HACKERS] location of the configuration files
Kevin Brown wrote: Bruce Momjian wrote: I just set PGDATA in my login and I don't have to deal with it again. Hmm...you don't use pg_ctl to start/stop/whatever the database? You invoke the postmaster directly (I can easily see that you would, just asking if you do)? I can use either to start/stop it. -- Bruce Momjian| http://candle.pha.pa.us [EMAIL PROTECTED] | (610) 359-1001 + If your life is a hard drive, | 13 Roberts Road + Christ can be your backup.| Newtown Square, Pennsylvania 19073 ---(end of broadcast)--- TIP 5: Have you checked our extensive FAQ? http://www.postgresql.org/users-lounge/docs/faq.html
Re: [HACKERS] location of the configuration files
Give it up. As long as we have -D, we will allow PGDATA. If you don't want to use it, don't use it. --- Kevin Brown wrote: Bruce Momjian wrote: I just set PGDATA in my login and I don't have to deal with it again. DuhI just realized a reason you guys might care about this so much. It's because you want to be able to start the postmaster from within a debugger (or profiler, or whatever), and you don't want to have to mess with command line options from there, right? Sounds like fixing pg_ctl to use -D explicitly when invoking the postmaster is the right change to make here, since that's probably how the majority of the production shops are going to be starting the database anyway. Takes care of the majority of the visibility problem and leaves PGDATA intact. Thoughts? -- Kevin Brown [EMAIL PROTECTED] ---(end of broadcast)--- TIP 2: you can get off all lists at once with the unregister command (send unregister YourEmailAddressHere to [EMAIL PROTECTED]) -- Bruce Momjian| http://candle.pha.pa.us [EMAIL PROTECTED] | (610) 359-1001 + If your life is a hard drive, | 13 Roberts Road + Christ can be your backup.| Newtown Square, Pennsylvania 19073 ---(end of broadcast)--- TIP 5: Have you checked our extensive FAQ? http://www.postgresql.org/users-lounge/docs/faq.html
Re: [HACKERS] location of the configuration files
My point is that folks with multiple postmasters may not want to use PGDATA, but for folks who have single postmasters, it makes things easier and less error-prone. --- Bruce Momjian wrote: Give it up. As long as we have -D, we will allow PGDATA. If you don't want to use it, don't use it. --- Kevin Brown wrote: Bruce Momjian wrote: I just set PGDATA in my login and I don't have to deal with it again. DuhI just realized a reason you guys might care about this so much. It's because you want to be able to start the postmaster from within a debugger (or profiler, or whatever), and you don't want to have to mess with command line options from there, right? Sounds like fixing pg_ctl to use -D explicitly when invoking the postmaster is the right change to make here, since that's probably how the majority of the production shops are going to be starting the database anyway. Takes care of the majority of the visibility problem and leaves PGDATA intact. Thoughts? -- Kevin Brown [EMAIL PROTECTED] ---(end of broadcast)--- TIP 2: you can get off all lists at once with the unregister command (send unregister YourEmailAddressHere to [EMAIL PROTECTED]) -- Bruce Momjian| http://candle.pha.pa.us [EMAIL PROTECTED] | (610) 359-1001 + If your life is a hard drive, | 13 Roberts Road + Christ can be your backup.| Newtown Square, Pennsylvania 19073 ---(end of broadcast)--- TIP 5: Have you checked our extensive FAQ? http://www.postgresql.org/users-lounge/docs/faq.html -- Bruce Momjian| http://candle.pha.pa.us [EMAIL PROTECTED] | (610) 359-1001 + If your life is a hard drive, | 13 Roberts Road + Christ can be your backup.| Newtown Square, Pennsylvania 19073 ---(end of broadcast)--- TIP 2: you can get off all lists at once with the unregister command (send unregister YourEmailAddressHere to [EMAIL PROTECTED])
Re: [HACKERS] location of the configuration files
Kevin Brown wrote: Bruce Momjian wrote: OK, here is an updated proposal. I think we have decided: Moving postmaster.pid and postmaster.opts isn't worth it. We don't want per-file GUC variables, but assume it is in the same config directory as postgresql.conf. I don't see any valid reason they would want to put them somewhere different than postgresql.conf. So, we add data_dir to postgresql.conf, and add -C/PGCONFIG to postmaster. Agreed. One additional thing: when pg_ctl invokes the postmaster, it should explicitly specify -C on the postmaster command line, and if it doesn't find a data_dir in $PGCONFIG/postgresql.conf then it should explicitly specify a -D as well. Pg_ctl is going to have to be modified to take a -C argument anyway, so we may as well go all the way to do the right thing here. This way, people who start the database using the standard tools we supply will know exactly what's going on when they get a ps listing. No. If you want ps to display, don't use environment variables. Many don't care --- especially those with only one postmaster. -- Bruce Momjian| http://candle.pha.pa.us [EMAIL PROTECTED] | (610) 359-1001 + If your life is a hard drive, | 13 Roberts Road + Christ can be your backup.| Newtown Square, Pennsylvania 19073 ---(end of broadcast)--- TIP 2: you can get off all lists at once with the unregister command (send unregister YourEmailAddressHere to [EMAIL PROTECTED])
Re: [HACKERS] location of the configuration files
Bruce Momjian [EMAIL PROTECTED] writes: So, we add data_dir to postgresql.conf, and add -C/PGCONFIG to postmaster. Wait one second. You are blithely throwing in a PGCONFIG variable without any detailed proposal of exactly how it will work. Does that override a PGDATA environment variable? How do they interact? Also, please note Kevin Brown's nearby arguments against using PGDATA at all, which surely apply with equal force to a PGCONFIG variable. Now, I don't buy that Kevin's arguments are enough reason to break backwards compatibility by removing PGDATA --- but I think they are enough reason not to introduce a new environment variable. PGCONFIG wouldn't offer any backwards-compatibility value, and that tilts the scales against it. Regarding Tom's idea of replacing data_dir with a full path during initdb, I think we are better having it be relative to the config directory, that way if they move pgsql/, the system still works. Good thought, but you're assuming that initdb knows where the config files will eventually live. If we do that, then moving the config files breaks the installation. I think it will be fairly common to let initdb drop its proposed config files into $PGDATA, and then manually place them where they should go (or even more likely, manually merge them with a prior version). Probably better to force datadir to be an absolute path in the config file. (In fact, on safety grounds I'd argue in favor of rejecting a datadir value taken from the config file that wasn't absolute.) I also think we should start telling people to use PGCONFIG rather than PGDATA. Then, in 7.5, we move the default config file location to pgsql/etc, and tell folks to point there rather than /data. I agree with none of this. This is not improvement, this is only change for the sake of change. The packagers will do what they want to do (and are already doing, mostly) regardless. regards, tom lane ---(end of broadcast)--- TIP 4: Don't 'kill -9' the postmaster
Re: [HACKERS] location of the configuration files
Oliver Elphick [EMAIL PROTECTED] writes: On Fri, 2003-02-14 at 12:17, Bruce Momjian wrote: If you want ps to display the data dir, you should use -D. Remember, it is mostly important for multiple postmaster, so if you are doing that, just use -D, but don't prevent single-postmaster folks from using PGDATA. Could not the ps line be rewritten to show this, as the backend's ps lines are rewritten? I for one would rather it didn't do that. I already set my postmaster command lines the way I want 'em, and I don't want the code overriding that. (I prefer to use explicit -p arguments to distinguish the various postmasters I have running --- shorter and easier to read than explicit -D would be. At least for me.) regards, tom lane ---(end of broadcast)--- TIP 2: you can get off all lists at once with the unregister command (send unregister YourEmailAddressHere to [EMAIL PROTECTED])
Re: [HACKERS] location of the configuration files
Tom Lane wrote: Bruce Momjian [EMAIL PROTECTED] writes: So, we add data_dir to postgresql.conf, and add -C/PGCONFIG to postmaster. Wait one second. You are blithely throwing in a PGCONFIG variable without any detailed proposal of exactly how it will work. Does that override a PGDATA environment variable? How do they interact? I am just throwing out ideas. I don't think we are near interaction issues yet. I think the big question is whether we want the default to install the configs in a separate directory, pgsql/etc, or just allow the specification of a separate location. Advantages of pgsql/etc are initdb-safe, and easier backups. I do think PGCONFIG would be helpful for the same reason that PGDATA is. However, there is clearly a problem of how does data_dir interact with PGDATA. The big question is whether PGDATA is still our driving config variable, and PGCONFIG/-C is just an additional option, or whether we are moving in a direction where PGCONFIG/-C is going to be the driving value, and data_dir is going to be read as part of that. Also, please note Kevin Brown's nearby arguments against using PGDATA at all, which surely apply with equal force to a PGCONFIG variable. Now, I don't buy that Kevin's arguments are enough reason to break backwards compatibility by removing PGDATA --- but I think they are enough reason not to introduce a new environment variable. PGCONFIG wouldn't offer any backwards-compatibility value, and that tilts the scales against it. Weren't you just showing how you set environment variables to easily configure stuff. If you use a separate configure dir, isn't PGCONFIG part of that? Regarding Tom's idea of replacing data_dir with a full path during initdb, I think we are better having it be relative to the config directory, that way if they move pgsql/, the system still works. Good thought, but you're assuming that initdb knows where the config files will eventually live. If we do that, then moving the config files breaks the installation. I think it will be fairly common to let initdb drop its proposed config files into $PGDATA, and then manually place them where they should go (or even more likely, manually merge them with a prior version). Probably better to force datadir to be an absolute path in the config file. (In fact, on safety grounds I'd argue in favor of rejecting a datadir value taken from the config file that wasn't absolute.) Maybe. Not sure. I also think we should start telling people to use PGCONFIG rather than PGDATA. Then, in 7.5, we move the default config file location to pgsql/etc, and tell folks to point there rather than /data. I agree with none of this. This is not improvement, this is only change for the sake of change. The packagers will do what they want to do (and are already doing, mostly) regardless. Well, it is a step forward in terms of initdb-safe and easier backups. Several people said they liked that. I thought you were one of them. This is back to the big question, who drives things in the default install, config file or pgdata. -- Bruce Momjian| http://candle.pha.pa.us [EMAIL PROTECTED] | (610) 359-1001 + If your life is a hard drive, | 13 Roberts Road + Christ can be your backup.| Newtown Square, Pennsylvania 19073 ---(end of broadcast)--- TIP 1: subscribe and unsubscribe commands go to [EMAIL PROTECTED]
Re: [HACKERS] location of the configuration files
Kevin Brown wrote: Bruce Momjian wrote: The big question is whether PGDATA is still our driving config variable, and PGCONFIG/-C is just an additional option, or whether we are moving in a direction where PGCONFIG/-C is going to be the driving value, and data_dir is going to be read as part of that. I'm actually leaning towards PGCONFIG + PGDATA. Yeah, it may be a surprise given my previous arguments, but I can't help but think that the advantages you get with PGDATA will also exist for PGCONFIG. My previous arguments for removing PGDATA from postmaster can be dealt with by fixing pg_ctl to use explicit command line directives when invoking postmaster -- no changes to postmaster needed. PGCONFIG would be no different in that regard. I see your point --- pg_ctl does a PGDATA trick when passed -D: -D) shift # pass environment into new postmaster PGDATA=$1 export PGDATA It should pass -D just like it was given. Sorry if I seem a big gung-ho on the administrator point of view, but as a system administrator myself I understand and feel their pain. Making things easy for sysadmins is an important feature of PostgreSQL. -- Bruce Momjian| http://candle.pha.pa.us [EMAIL PROTECTED] | (610) 359-1001 + If your life is a hard drive, | 13 Roberts Road + Christ can be your backup.| Newtown Square, Pennsylvania 19073 ---(end of broadcast)--- TIP 2: you can get off all lists at once with the unregister command (send unregister YourEmailAddressHere to [EMAIL PROTECTED])
Re: [HACKERS] location of the configuration files
Bruce Momjian [EMAIL PROTECTED] writes: I think the big question is whether we want the default to install the configs in a separate directory, pgsql/etc, or just allow the specification of a separate location. Advantages of pgsql/etc are initdb-safe, and easier backups. I don't see why we don't just let initdb install suggested config files into the new $PGDATA directory, same as it ever did. Then (as long as we don't use relative paths in the config files) people can move them somewhere else if they like, or not if they prefer not to. Adding more mechanism than that just adds complexity without buying much (except the possibility of initdb overwriting your old config files, which is exactly what I thought we wanted to avoid). The big question is whether PGDATA is still our driving config variable, and PGCONFIG/-C is just an additional option, or whether we are moving in a direction where PGCONFIG/-C is going to be the driving value, and data_dir is going to be read as part of that. I thought the idea was to allow both approaches. We are not moving in the direction of one or the other, we are giving people a choice of how they want to drive it. Weren't you just showing how you set environment variables to easily configure stuff. If you use a separate configure dir, isn't PGCONFIG part of that? I'm just pointing out that there's no backward-compatibility argument for PGCONFIG. It should only be put in if the people who want to use the -C-is-driver approach want it. Kevin clearly doesn't ;-). regards, tom lane ---(end of broadcast)--- TIP 4: Don't 'kill -9' the postmaster
Re: [HACKERS] location of the configuration files
Well, to the extent that you're serious, you understand that a lot of people feel that /usr/local should be reserved for stuff that's installed by the local sysadmin, and your vendor/distro isn't supposed to be messing with it. Which means if the the vendor installed Postgresql (say, the Red Hat Database) you'd expect config files to be in /etc. If the postgresql is compiled from source by local admin, you might look somewhere in /usr/local. Indeed. For better or worse, there is a Filesystem Hierarcy Standard, and most of the important Linux distros, BSDs and some legacy Unixen stick to it, so so should we. Configuration files should be in /etc/postgresql/, or at the very least symlinked from there. Martin ---(end of broadcast)--- TIP 2: you can get off all lists at once with the unregister command (send unregister YourEmailAddressHere to [EMAIL PROTECTED])
Re: [HACKERS] location of the configuration files
On Wed, 12 Feb 2003, Peter Bierman wrote: What do you gain by having the postmaster config and the database data live in different locations? You can then standardize a location for the configuration files. Everybody has room in /etc for another 10K of data. Where you have room for something that might potentially be a half terrabyte of data, and is not infrequently several gigabytes or more, is pretty system-depenendent. cjs -- Curt Sampson [EMAIL PROTECTED] +81 90 7737 2974 http://www.netbsd.org Don't you know, in this new Dark Age, we're all light. --XTC ---(end of broadcast)--- TIP 2: you can get off all lists at once with the unregister command (send unregister YourEmailAddressHere to [EMAIL PROTECTED])
Re: [HACKERS] location of the configuration files
In the last exciting episode, [EMAIL PROTECTED] (Curt Sampson) wrote: On Wed, 12 Feb 2003, Peter Bierman wrote: What do you gain by having the postmaster config and the database data live in different locations? You can then standardize a location for the configuration files. Everybody has room in /etc for another 10K of data. Where you have room for something that might potentially be a half terrabyte of data, and is not infrequently several gigabytes or more, is pretty system-depenendent. Ah, but this has two notable problems: 1. It assumes that there is a location for the configuration files for /the single database instance./ If I have a second database instance, that may conflict. 2. It assumes I have write access to /etc If I'm a Plain Old User, as opposed to root, I may only have read-only access to /etc. These conditions have both been known to occur... -- If this was helpful, http://svcs.affero.net/rm.php?r=cbbrowne rate me http://www.ntlug.org/~cbbrowne/rdbms.html Some people, when confronted with a Unix problem, think I know, I'll use sed. Now they have two problems. -- Jamie Zawinski ---(end of broadcast)--- TIP 6: Have you searched our list archives? http://archives.postgresql.org
Re: [HACKERS] location of the configuration files
Christopher Browne wrote: In the last exciting episode, [EMAIL PROTECTED] (Curt Sampson) wrote: On Wed, 12 Feb 2003, Peter Bierman wrote: What do you gain by having the postmaster config and the database data live in different locations? You can then standardize a location for the configuration files. Everybody has room in /etc for another 10K of data. Where you have room for something that might potentially be a half terrabyte of data, and is not infrequently several gigabytes or more, is pretty system-depenendent. Ah, but this has two notable problems: 1. It assumes that there is "a location" for "the configuration files for /the single database instance./" If I have a second database instance, that may conflict. 2. It assumes I have write access to /etc If I'm a Plain Old User, as opposed to root, I may only have read-only access to /etc. These conditions have both been known to occur... These are not issues at all. You could put the configuration file anywhere, just as you can for any UNIX service. postmaster --config=/home/myhome/mydb.conf I deal with a number of PG databases on a number of sites, and it is a real pain in the ass to get to a PG box and hunt around for data directory so as to be able to administer the system. What's really annoying is when you have to find the data directory when someone else set up the system. Configuring postgresql via a configuration file which specifies all the data, i.e. data directory, name of other configuration files, etc. is the right way to do it. Even if you have reasons against it, even if you think it is a bad idea, a bad standard is almost always a better solution than an arcane work of perfection. Personally, however, I think the configuration issue is a no-brainer and I am amazed that people are balking. EVERY other service on a UNIX box is configured in this way, why not do it this way in PostgreSQL? The patch I submitted allowed the configuration to work as it currently does, but allowed for the more standard configuration file methodology. I just don't understand what the resistance is, it makes no sense.
Re: [HACKERS] location of the configuration files
On Thu, 13 Feb 2003, Christopher Browne wrote: 1. It assumes that there is a location for the configuration files for /the single database instance./ No; it assumes that there's a location for the default instance. If you have more than one, you could have one default and one elsewhere, or just do what I often do, which is put in an empty config file except for a comment saying we have several instances of xxx on this machine; look in yyy for them. 2. It assumes I have write access to /etc If I'm a Plain Old User, as opposed to root, I may only have read-only access to /etc. Right. It's dependent on the sysadmin to create /etc/postgres/ and make it writeable, or set up proper symlinks, or whatever. Fortunately, the files in /etc are only the defaults, to be used if they're not overridden on the command line. If you're in a situation like #2, you're basically stuck where we are now all the time: you have to just put it somewhere and hope that, if someone else needs to find it, they can. cjs -- Curt Sampson [EMAIL PROTECTED] +81 90 7737 2974 http://www.netbsd.org Don't you know, in this new Dark Age, we're all light. --XTC ---(end of broadcast)--- TIP 1: subscribe and unsubscribe commands go to [EMAIL PROTECTED]
Re: [HACKERS] location of the configuration files
On Thursday 13 February 2003 08:32, Christopher Browne wrote: In the last exciting episode, [EMAIL PROTECTED] (Curt Sampson) wrote: Everybody has room in /etc for another 10K of data. Where you have room for something that might potentially be a half terrabyte of data, and is not infrequently several gigabytes or more, is pretty system-depenendent. 1. It assumes that there is a location for the configuration files for /the single database instance./ If I have a second database instance, that may conflict. If you run multiple servers of any kind, the second and subsequent servers must have a command line switch specifying the location of the config file. This is the way named, sendmail, et al do it. I have run multiple nameds on a single box, by using alternate config file locations. 2. It assumes I have write access to /etc If I'm a Plain Old User, as opposed to root, I may only have read-only access to /etc. So you start postmaster with a config file switch pointing to your config file in your tree. Or you specify the default location with a configure switch at compile time. Or you do it the same way you would run any other typical daemon as a regular user. How does Apache, AOLserver (my favorite), sendmail, jabberd, named, or any other typical daemon do it? For example, AOLserver can easily be installed and run as a plain user (just not on port 80). The command line switch '-t' specifies the tcl configuration script's location. There is no default. The configuration script then specifies pageroot and the like -- and a webserver is very much like running PostgreSQL -- you have configuration, you have logs, and you have the spool (database or pageroot). All can be in different locations at the discretion of the admin. And hardcoding dependencies like this stifles the discretion of the admin. These conditions have both been known to occur... Just because the situation is known to occur doesn't mean the whole direction of a project should hinge on those corner cases. They should be allowed but not forced. For better or for worse, thanks to Karl DeBisschop, the latest RPMs have the ability to start multiple postmasters with their data trees and configurations in different places. This is all done in the startup script, and required no new functionality from postmaster. I personally think it is for the better; YMMV. -- Lamar Owen WGCR Internet Radio 1 Peter 4:11 ---(end of broadcast)--- TIP 1: subscribe and unsubscribe commands go to [EMAIL PROTECTED]
Re: [HACKERS] location of the configuration files
On Thu, 13 Feb 2003, Curt Sampson wrote: On Thu, 13 Feb 2003, Christopher Browne wrote: 1. It assumes that there is a location for the configuration files for /the single database instance./ No; it assumes that there's a location for the default instance. If you have more than one, you could have one default and one elsewhere, or just do what I often do, which is put in an empty config file except for a comment saying we have several instances of xxx on this machine; look in yyy for them. 2. It assumes I have write access to /etc If I'm a Plain Old User, as opposed to root, I may only have read-only access to /etc. Right. It's dependent on the sysadmin to create /etc/postgres/ and make it writeable, or set up proper symlinks, or whatever. Fortunately, the files in /etc are only the defaults, to be used if they're not overridden on the command line. If you're in a situation like #2, you're basically stuck where we are now all the time: you have to just put it somewhere and hope that, if someone else needs to find it, they can. It doesn't follow this line of argument directly but it's to do with this thread... Is everyone forgetting that wherever the configuration file is stored and whether or not it needs a command line argument to specify it the database is not going to start up automatically unless at least part of the installation is done as root anyway? As I like to install software as a non root user normally anyway I am happy that the config file lives somewhere not requiring write access by the installer. However, I think having it in an etc directory is a good thing (tm). So, colour me an uncommited, fence sitter :) I'm not talking distribution/package installation here but just plain system administration. Being an untrusting soul I do _not_ want to type make install as root and find things installed outside of where I say I want things placed. That includes configuration files. Doing this as a normal user protects the system from bad software which assumes things about the host system. It also simplifies switching between versions of software, try doing that if your config is /etc/postgresql/postgres.conf. -- Nigel J. Andrews ---(end of broadcast)--- TIP 6: Have you searched our list archives? http://archives.postgresql.org
Re: [HACKERS] location of the configuration files
On Thu, Feb 13, 2003 at 15:03:09 +, Nigel J. Andrews [EMAIL PROTECTED] wrote: Is everyone forgetting that wherever the configuration file is stored and whether or not it needs a command line argument to specify it the database is not going to start up automatically unless at least part of the installation is done as root anyway? Users can use cron to start there own instance. Your cron script can check if the server is running every (say) 15 minutes and start the server if it isn't. ---(end of broadcast)--- TIP 2: you can get off all lists at once with the unregister command (send unregister YourEmailAddressHere to [EMAIL PROTECTED])
Re: [HACKERS] location of the configuration files
On Thu, Feb 13, 2003 at 09:23:20 -0500, mlw [EMAIL PROTECTED] wrote: Personally, however, I think the configuration issue is a no-brainer and I am amazed that people are balking. EVERY other service on a UNIX box is configured in this way, why not do it this way in PostgreSQL? The patch I submitted allowed the configuration to work as it currently does, but allowed for the more standard configuration file methodology. If you are interested in reading a contrary position, you can read Berstein's arguments for his recommended way to install services at: http://cr.yp.to/unix.html ---(end of broadcast)--- TIP 3: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [HACKERS] location of the configuration files
On Thu, 2003-02-13 at 09:23, mlw wrote: I deal with a number of PG databases on a number of sites, and it is a real pain in the ass to get to a PG box and hunt around for data directory so as to be able to administer the system. What's really annoying is when you have to find the data directory when someone else set up the system. find / -name postgresql.conf -print you now know where all of your configuration files are and where the data for each of those servers is as well. (Not I'm not against the idea...) Robert Treat ---(end of broadcast)--- TIP 2: you can get off all lists at once with the unregister command (send unregister YourEmailAddressHere to [EMAIL PROTECTED])
Re: [HACKERS] location of the configuration files
Robert Treat wrote: On Thu, 2003-02-13 at 09:23, mlw wrote: I deal with a number of PG databases on a number of sites, and it is a real pain in the ass to get to a PG box and hunt around for data directory so as to be able to administer the system. What's really annoying is when you have to find the data directory when someone else set up the system. find / -name postgresql.conf -print LOL, That is NOT an option. It can take hours on some systems. Specifically, one of the systems is freedb server, it has over 300,000 files in a directory tree.
Re: [HACKERS] location of the configuration files
Bruno Wolff III wrote: On Thu, Feb 13, 2003 at 09:23:20 -0500, mlw [EMAIL PROTECTED] wrote: Personally, however, I think the configuration issue is a no-brainer and I am amazed that people are balking. EVERY other service on a UNIX box is configured in this way, why not do it this way in PostgreSQL? The patch I submitted allowed the configuration to work as it currently does, but allowed for the more standard configuration file methodology. If you are interested in reading a contrary position, you can read Berstein's arguments for his recommended way to install services at: http://cr.yp.to/unix.html Where, specificaly are his arguements against a configuration file methodology?
Re: [HACKERS] location of the configuration files
On Thu, 13 Feb 2003, mlw wrote: Robert Treat wrote: On Thu, 2003-02-13 at 09:23, mlw wrote: I deal with a number of PG databases on a number of sites, and it is a real pain in the ass to get to a PG box and hunt around for data directory so as to be able to administer the system. What's really annoying is when you have to find the data directory when someone else set up the system. You realize that the actual code feature doesn't necessarily help this case, right? Putting configuration in /etc and having a configuration file option on the command line are separate concepts. I think the feature is worthwhile, but I have some initial condition functionality questions that may have been answered in the previous patch, but I don't remember at this point. Mostly these have to deal with initial creation. Does the user specify an output location to initdb, do they just specify a data dir as now where the configuration goes but then they need to move it somewhere, does initdb now do nothing relating to configuration file and the user should make one on his own. Related, is the admin expected to have already made (say) /etc/postgresql to stick the config in and set the permissions correctly (since initdb doesn't run as root)? ---(end of broadcast)--- TIP 5: Have you checked our extensive FAQ? http://www.postgresql.org/users-lounge/docs/faq.html
Re: [HACKERS] location of the configuration files
Stephan Szabo wrote: On Thu, 13 Feb 2003, mlw wrote: Robert Treat wrote: On Thu, 2003-02-13 at 09:23, mlw wrote: I deal with a number of PG databases on a number of sites, and it is a real pain in the ass to get to a PG box and hunt around for data directory so as to be able to administer the system. What's really annoying is when you have to find the data directory when someone else set up the system. You realize that the actual code feature doesn't necessarily help this case, right? Putting configuration in /etc and having a configuration file option on the command line are separate concepts. I think the feature is worthwhile, but I have some initial condition functionality questions that may have been answered in the previous patch, but I don't remember at this point. Mostly these have to deal with initial creation. Does the user specify an output location to initdb, do they just specify a data dir as now where the configuration goes but then they need to move it somewhere, does initdb now do nothing relating to configuration file and the user should make one on his own. Related, is the admin expected to have already made (say) /etc/postgresql to stick the config in and set the permissions correctly (since initdb doesn't run as root)? My patch only works on the PostgreSQL server code. No changes have been made to the initialization scripts. The patch declares three extra configuration file parameters: hbafile= '/etc/postgres/pg_hba.conf' identfile='/etc/postgres/pg_ident.conf' datadir='/RAID0/postgres' The command line option is a capital 'C,' as in: postmaster -C /etc/postgresql.conf I have no problem leaving the default configuration files remaining in the data directory as sort of a maintenance / boot strap sort of thing, so I don't see any reason to alter the installation. As for this feature helping or not, I think it will. I think it accomplishes two things: (1) Separates configuration from data. (2) Allows an administrator to create a convention across multiple systems regardless of the location and mount points of the database storage. (3) Lastly, it is a familiar methodology to DBAs not familiar with PostgreSQL. Again, I don't see a valid reason for not including the patch. Yes, if you don't want to configure PostgreSQL that way, then so be it, but why not add the functionality for those who do? I can envision the configuration file methodology of managing a database becoming the "preferred" approach over time as it is a more familiar and standard way of configuring servers on UNIX.
Re: [HACKERS] location of the configuration files
On Thu, 13 Feb 2003, mlw wrote: Stephan Szabo wrote: On Thu, 2003-02-13 at 09:23, mlw wrote: I deal with a number of PG databases on a number of sites, and it is a real pain in the ass to get to a PG box and hunt around for data directory so as to be able to administer the system. What's really annoying is when you have to find the data directory when someone else set up the system. You realize that the actual code feature doesn't necessarily help this case, right? Putting configuration in /etc and having a configuration file option on the command line are separate concepts. Re-read my statement and yours about the case you were mentioning. ;) Sure, putting the files in /etc lets you find them easily. However, if you're doing things like finding configuration made by someone else and said configuration isn't in /etc (which if they wanted to they could do now with symlinks I believe - yes symlinks aren't a complete solution, but I think they're reasonable on most of our current ports) then you still have to search the system for the configuration file, except now it might not even be postgresql.conf. That's why I said the two issues aren't the same. I think the feature is worthwhile, but I have some initial condition functionality questions that may have been answered in the previous patch, but I don't remember at this point. Mostly these have to deal with initial creation. Does the user specify an output location to initdb, do they just specify a data dir as now where the configuration goes but then they need to move it somewhere, does initdb now do nothing relating to configuration file and the user should make one on his own. Related, is the admin expected to have already made (say) /etc/postgresql to stick the config in and set the permissions correctly (since initdb doesn't run as root)? My patch only works on the PostgreSQL server code. No changes have been made to the initialization scripts. The patch declares three extra configuration file parameters: hbafile= '/etc/postgres/pg_hba.conf' identfile='/etc/postgres/pg_ident.conf' datadir='/RAID0/postgres' The command line option is a capital 'C,' as in: postmaster -C /etc/postgresql.conf I have no problem leaving the default configuration files remaining in the data directory as sort of a maintenance / boot strap sort of thing, so I don't see any reason to alter the installation. As for this feature helping or not, I think it will. I think it accomplishes two things: (1) Separates configuration from data. (2) Allows an administrator to create a convention across multiple systems regardless of the location and mount points of the database storage. (3) Lastly, it is a familiar methodology to DBAs not familiar with PostgreSQL. I agree on all these points (I think the feature is worthwhile, but...). I just wonder if we were going to do this, we might as well look at all of the various things people want and decide what we want to do, for example, people commenting on default configuration locations through configure, how does this interact with what we have now, etc. I'd rather have a month spent arguing out a behavior rather than just adding a new behavior that we'll need to possibly revisit again in the future. :) ---(end of broadcast)--- TIP 5: Have you checked our extensive FAQ? http://www.postgresql.org/users-lounge/docs/faq.html
Re: [HACKERS] location of the configuration files
On 13 Feb 2003, Martin Coxall wrote: Well, to the extent that you're serious, you understand that a lot of people feel that /usr/local should be reserved for stuff that's installed by the local sysadmin, and your vendor/distro isn't supposed to be messing with it. Which means if the the vendor installed Postgresql (say, the Red Hat Database) you'd expect config files to be in /etc. If the postgresql is compiled from source by local admin, you might look somewhere in /usr/local. Indeed. For better or worse, there is a Filesystem Hierarcy Standard, and most of the important Linux distros, BSDs and some legacy Unixen stick to it, so so should we. Configuration files should be in /etc/postgresql/, or at the very least symlinked from there. So, how do we handle things like installing three or four versions at the same time. This isn't the same thing as /etc/fstab. While we only would likely need to have one fstab or whatever, with postgresql, it's not unreasonable to want to intall more than one copy or version for various reason. Generally things that live in /etc are owned and operated by the OS. Postgresql, by it's definition is a userspace program, not an OS owned one. I've found having a $PGDATA var where EVERYTHING lives to be a huge advantage when you need to run a half dozen instances of pgsql under different accounts or for different versions on the same box. Now, if we could do it like X, where the base stuff is all in the /etc/X11R6 directory, but your own personal config lives in your home directory, then we're right as rain. but what parts of postgresql would always be common to all flavors that might need to be run at the same time? Not much. ---(end of broadcast)--- TIP 5: Have you checked our extensive FAQ? http://www.postgresql.org/users-lounge/docs/faq.html
Re: [HACKERS] location of the configuration files
I don't see why we can't keep everyone happy and let the users choose the setup they want. To wit, make the following, probably simple, changes: 1) Have postgresql default to using /etc/postgresql.conf 2) Add a setting in postgresql.conf specifying the data directory 3) Change the meaning of -D to mean use this config file 4) In the absence of a specified data directory in postgresql.conf, use the location of the postgresql.conf file as the data directory I see several advantages: 1) Anyone who doesn't want to change doesn't have to - leaving the data directory spec out of postgresql.conf and starting with -D will be essentially identical to how things are now (except it would be -D /foo/bar/postgresql.conf instead of -D /foo/bar/ - even this could be overcome with a bit of bailing wire saying if -D specifies a directory, look for postgresql.conf in that directory). 2) Postgresql will be more familiar to those who expect or desire configs to be in /etc. 3) Adding a postgresql.conf line for data location sets the stage for being able to specify directories for all sorts of files (WAL, index, etc.) without the need for symlinks. 4) Multiple config files could be more easily managed for testing/benchmarking/etc. Cheers, Steve On Wednesday 12 February 2003 10:14 pm, Peter Bierman wrote: At 12:31 AM -0500 2/13/03, mlw wrote: The idea that a, more or less, arbitrary data location determines the database configuration is wrong. It should be obvious to any administrator that a configuration file location which controls the server is the right way to do it. Isn't the database data itself a rather significant portion of the 'configuration' of the database? What do you gain by having the postmaster config and the database data live in different locations? -pmb ---(end of broadcast)--- TIP 2: you can get off all lists at once with the unregister command (send unregister YourEmailAddressHere to [EMAIL PROTECTED]) ---(end of broadcast)--- TIP 3: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [HACKERS] location of the configuration files
On 13 Feb 2003, Oliver Elphick wrote: On Thu, 2003-02-13 at 18:45, Bruce Momjian wrote: Oliver Elphick wrote: On Thu, 2003-02-13 at 17:52, Vince Vielhaber wrote: Seems to me that if FHS allows such a mess, it's reason enough to avoid compliance. Either that or those of you who build for distributions are making an ill advised change. Simply because the distribution makes the decision to add PostgreSQL, or some other package, to it's distribution doesn't make it a requirement to change the location of the config files. ... I really don't see why there is such a not-invented-here mentality about this issue. I say again, standards-compliance is the best way. It makes life easier for everyone if standards are followed. Don't we pride ourselves on being closer to the SQL spec than other databases? Any way, if PostgreSQL stays as it is, I will continue to have to ensure that initdb creates symlinks to /etc/postgresql/, as happens now. It doesn't have anything to do with not-invented-here, which is a common refrain by people who don't like our decisions, like Why don't you use mmap()? Oh, it's because I thought of it and you didn't. Does anyone seriously believe that is the motiviation of anyone in this project! I certainly don't. My apologies. I withdraw the comment, which was provoked mostly by Vince's response, quoted above. I agree that it is not characteristic of the project. I certainly wasn't trying to provoke anything. It just seems odd to me that when the distribution installs a package and places it's config files in /etc and later the admin happens to upgrade by the instructions with the package, it's acceptable for the config files to now be in two places and you don't find it confusing. What happens when a new admin comes on and tries to figure out which config file is which? Ever try to figure out where the hell Pine's config really is? Vince. -- Fast, inexpensive internet service 56k and beyond! http://www.pop4.net/ http://www.meanstreamradio.com http://www.unknown-artists.com Internet radio: It's not file sharing, it's just radio. ---(end of broadcast)--- TIP 3: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [HACKERS] location of the configuration files
On Thu, 13 Feb 2003, mlw wrote: Christopher Browne wrote: In the last exciting episode, [EMAIL PROTECTED] (Curt Sampson) wrote: On Wed, 12 Feb 2003, Peter Bierman wrote: What do you gain by having the postmaster config and the database data live in different locations? You can then standardize a location for the configuration files. Everybody has room in /etc for another 10K of data. Where you have room for something that might potentially be a half terrabyte of data, and is not infrequently several gigabytes or more, is pretty system-depenendent. Ah, but this has two notable problems: 1. It assumes that there is a location for the configuration files for /the single database instance./ If I have a second database instance, that may conflict. 2. It assumes I have write access to /etc If I'm a Plain Old User, as opposed to root, I may only have read-only access to /etc. These conditions have both been known to occur... These are not issues at all. You could put the configuration file anywhere, just as you can for any UNIX service. postmaster --config=/home/myhome/mydb.conf I deal with a number of PG databases on a number of sites, and it is a real pain in the ass to get to a PG box and hunt around for data directory so as to be able to administer the system. What's really annoying is when you have to find the data directory when someone else set up the system. Really? I would think it's easier to do this: su - pgsuper cd $PGDATA pwd Than to try to figure out what someone entered when they ran ./configure --config=... Configuring postgresql via a configuration file which specifies all the data, i.e. data directory, name of other configuration files, etc. is the right way to do it. Even if you have reasons against it, even if you think it is a bad idea, a bad standard is almost always a better solution than an arcane work of perfection. Wrong, I strongly disagree with this sentament. Conformity to standards for simple conformity's sake is as wrong as sticking to the old way because it's what we're all comfy with. Personally, however, I think the configuration issue is a no-brainer and I am amazed that people are balking. EVERY other service on a UNIX box is configured in this way, why not do it this way in PostgreSQL? The patch I submitted allowed the configuration to work as it currently does, but allowed for the more standard configuration file methodology. If I do a .tar.gz install of apache, I get /usr/local/apache/conf, which is not the standard way you're listing. If I install openldap from .tar.gz, I get a /usr/local/etc/openldap directory, close, but still not the same. The fact is, it's the packagers that put things into /etc and whatnot, and I can see the postgresql RPMs or debs or whatever having that as the default, but for custom built software, NOTHING that I know of builds from source and uses /etc without a switch to tell it to, just like postgresql can do now. I just don't understand what the resistance is, it makes no sense. I agree, but from the other side of the fence. ---(end of broadcast)--- TIP 5: Have you checked our extensive FAQ? http://www.postgresql.org/users-lounge/docs/faq.html
Re: [HACKERS] location of the configuration files
scott.marlowe wrote: These are not issues at all. You could put the configuration file anywhere, just as you can for any UNIX service. postmaster --config=/home/myhome/mydb.conf I deal with a number of PG databases on a number of sites, and it is a real pain in the ass to get to a PG box and hunt around for data directory so as to be able to administer the system. What's really annoying is when you have to find the data directory when someone else set up the system. Really? I would think it's easier to do this: su - pgsuper cd $PGDATA pwd Than to try to figure out what someone entered when they ran ./configure --config=... Why do you think PGDATA would be set for root? Configuring postgresql via a configuration file which specifies all the data, i.e. data directory, name of other configuration files, etc. is the right way to do it. Even if you have reasons against it, even if you think it is a bad idea, a bad standard is almost always a better solution than an arcane work of perfection. Wrong, I strongly disagree with this sentament. Conformity to standards for simple conformity's sake is as wrong as sticking to the old way because it's what we're all comfy with. It isn't conformity for conformitys sake. It is following an established practice, like driving on the same side of the road or stopping at red lights. Personally, however, I think the configuration issue is a no-brainer and I am amazed that people are balking. EVERY other service on a UNIX box is configured in this way, why not do it this way in PostgreSQL? The patch I submitted allowed the configuration to work as it currently does, but allowed for the more standard configuration file methodology. If I do a .tar.gz install of apache, I get /usr/local/apache/conf, which is not the standard way you're listing. If I install openldap from .tar.gz, I get a /usr/local/etc/openldap directory, close, but still not the same. The fact is, it's the packagers that put things into /etc and whatnot, and I can see the postgresql RPMs or debs or whatever having that as the default, but for custom built software, NOTHING that I know of builds from source and uses /etc without a switch to tell it to, just like postgresql can do now. You are confusing the default location of a file with the ability to use the file. The default I have proposed all along was to use the existing practice of keeping everything in the $PGDATA directory. The change I wish to make to the code allows this to be changed. Most admins want configuration and data separate. Most admins do not want to use symlinks because they are dangerous in a production environment. I would rather have a simpler solution sooner than a perfect solution never.
Re: [HACKERS] location of the configuration files
On Thu, 2003-02-13 at 15:08, mlw wrote: Stephan Szabo wrote: Re-read my statement and yours about the case you were mentioning. ;) Sure, putting the files in /etc lets you find them easily. However, if you're doing things like finding configuration made by someone else and said configuration isn't in /etc (which if they wanted to they could do now with symlinks I believe - yes symlinks aren't a complete solution, but I think they're reasonable on most of our current ports) then you still have to search the system for the configuration file, except now it might not even be postgresql.conf. That's why I said the two issues aren't the same. Actually, I'd almost go so far as to say it will make it worse. In the current system, if you can figure out where $PGDATA is, you've found everything you need for that installation. In the new system, there's no telling where people will put things, and it certainly won't be any easier to find it. THinking on the above Stephan, you'd almost have to require that the config file be called postgresql.conf in order to run, anything else leads to real scary scenario's. On Thu, 13 Feb 2003, mlw wrote: I have absolutely no problem debating and augmenting the feature. None what so ever, I am more pushing to get momentum to actually do it. Stick with it, I think most of us here can see the value in the option, but there are valid concerns that it be implemented correctly. Personally I think a postgresql installation is much more like an apache installation, which generally contains all of the files (data and config) under /usr/local/apache. Maybe someone can dig more to see if that system is more appropriate a comparison than something like bind. Robert Treat ---(end of broadcast)--- TIP 1: subscribe and unsubscribe commands go to [EMAIL PROTECTED]
Re: [HACKERS] location of the configuration files
On Thu, Feb 13, 2003 at 05:59:17PM -0500, Robert Treat wrote:
On Thu, 2003-02-13 at 15:08, mlw wrote:
Stephan Szabo wrote:
On Thu, 13 Feb 2003, mlw wrote:
I have absolutely no problem debating and augmenting the feature. None
what so ever, I am more pushing to get momentum to actually do it.
Stick with it, I think most of us here can see the value in the option,
but there are valid concerns that it be implemented correctly.
Personally I think a postgresql installation is much more like an apache
installation, which generally contains all of the files (data and
config) under /usr/local/apache. Maybe someone can dig more to see if
that system is more appropriate a comparison than something like bind.
I think you are making a pretty uninformed, if not just plain wrong
generalization. I've run exactly one system with apache configuration
files in /usr/local/apache, and even then, the data was not there.
A quick straw poll of the people I know who actually do run real systems
also mentioned that they use packaging systems like encap or rpm to manage
upgrades, and would almost never put datafiles into /usr/local.
RedHat (7.3 at least)'s default httpd datafiles go in /var/www/html and
config goes in /etc/httpd
One OpenBSD user I talked to puts his in /home/www and config files in
/etc/httpd. The defaults are /var/www and /var/www/conf
Another user reports:
On systems that I set up I have /web/{apache|httpd}/ and put all
the config info there.
And /web/sites/name/ holds site data.
What does this mean?
People will put things in different places, and there are typically
very good reasons for this. This is ESPECIALLY true when one wants to
have configuration files, at least the base ones in a common place such
as /etc or /usr/local/etc in order to make backup of configuration easy
and clean, while leaving data somewhere else for performance or magnitude
of partition reasons. It just makes sense to ME to have postgresql.conf
reside in /etc, yet put my data in /var/data/postgresql, yet retain the
option to put my data in /raid/data/postgresql at a later date, when the
new hardware comes in.
Yes, symlinks are an option on most systems. No, they are not a good
one on most systems.
What _I_ would like to see:
o. a default postgresql.conf location of $PREFIX/data/postgresql.conf
o. a default PGDATA location of whatever directory postgresql.conf is in
(this should maintain backward compatibility)
o. a ./configure - time option to override the location of the postgresql.conf
o. a run-time option to override the location of the postgresql.conf
o. options in postgresql.conf to specify the location of PGDATA and PID files.
($PREFIX is already settable at ./configure - time)
This would allow:
o. Config files in /usr/local/pgsql/data, /etc, /usr/local/etc, ~postgresql,
or /dev/.hidden-node, whichever you prefer, so long as you either know
the compile-time default, or are willing to specify it at startup.
o. Datafiles to be in /usr/local/pgsql/data, /var/data, /raid0, /nfs/bigmount
or whichever you prefer, so long as you either know the compile-time default,
or are willing to specify it in a config file that you specify at startup.
Does it add complexity to the system? Sure -- a very little bit, IMHO, especially
compared to the BTREE-folding that I see being bantered about.
Is it some work? Sure -- a very little bit, and it seems that it has already
been done.
However, this seems, to me, to be a very small addition that has some real-world
(and yes, we need to start paying attention to the real world) advantages.
And finally, don't go telling me that I'm wrong to put my data and config files
where I am. You can offer advice, but I'm probably going to ignore it because
I like where they are and don't need to explain why.
--
Adam Haberlach | Because manholes are round.
[EMAIL PROTECTED] |
http://mediariffic.com |
---(end of broadcast)---
TIP 3: if posting/reading through Usenet, please send an appropriate
subscribe-nomail command to [EMAIL PROTECTED] so that your
message can get through to the mailing list cleanly
