Re: [HACKERS] SQL command to edit postgresql.conf, with comments
Dimitri Fontaine dimi...@2ndquadrant.fr writes: cat postgresql.conf.d/work_mem 16MB This database needs at least such a value. Note it's overridden in some ROLEs setup. With such a format (name is filename, value is first line content's, rest is comments), it's easy to preserve comments and have them machine editable. What do I miss? Allow me to insist on this some more, because there's something important going on here. The other proposal (.auto) have a major failure mode that I don't think is acceptable. SET PERMANENT work_mem TO '8 MB'; select pg_reload_conf(); There's simply no way after this sequence to guess the current active value of work_mem, because it could be overridden in the non-automatic file. How do you work with such a system? So, back to my funny proposal. It does not have the problem above because it's either postgresql.conf or postgresql.conf.d, not both. A single source of authority. I'm being told that we're talking about something over 200 files and that's too many. That's a problem size we already know how to handle, I would say: dim ~/dev/PostgreSQL/postgresql-extension ls -C1 doc/src/sgml/ref/ |wc -l 166 Ok, now, some people want to still be able to edit the files by hand because it's the way they do it now. Well, first, that's still possible and easy to script if so you care, then again, open an editor a prepare a SQL script: set permanent foo to bar with comment 'blah'; set … Then you psql -f the script and you're done. So you use the remote edit feature to be able to edit a single file rather than a bunch of them if so you want. Another option in the same spirit would be to prepare a file in the CSV format (name;value) and have a script that COPY it into some table, then SELECT set_config(name, value, 'permanent') FROM import_setup_table; With a CSV foreign data wrapper at the door, you don't even have to use an intermediate table where to direct your COPY. Executive Summary: you can still edit some file the old way, you just need a tool to then issue the SQL for you rather than reload. Now, there are some comments that are not tied to any setting in particular. We could extend my idea to include a .comments file in the directory, and a command to edit it. We could make it so that COMMENT ON SETTING would just amend the given GUC's file, and that COMMENT ON CONFIG allow you to edit the main .comment file. Then, there's another feature here, which is having a single comment entry for more than one parameter. Again, we could support it rather easily by having directories to group files, and the common comment would be the groupname/.comment file entry. Add some new commands to manage that, and refuse to consider the setup at all as soon as there's file name duplication in there. CREATE SETTING GROUP name WITH foo, bar, ... AND COMMENT 'foobar'; CREATE SETTING GROUP other WITH baz, foo; NOTICE: foo was in the group name, it's been moved to other. Regards, -- Dimitri Fontaine http://2ndQuadrant.fr PostgreSQL : Expertise, Formation et Support -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] SQL command to edit postgresql.conf, with comments
Dimitri Fontaine dimi...@2ndquadrant.fr writes:
Ok, now, some people want to still be able to edit the files by hand
I wonder if providing a system function to load some configuration
option from a file, using the current parser, would do it:
SELECT pg_load_conf('path/to/filename.conf', permament = true);
Of course comments are ignored. Settings in the directory would be
changed according to what's in your file, so that it'd be roughtly
equivalent as current steps:
a. edit the file
b. use pg_load_conf() rather than pg_reload_conf()
Comments?
--
Dimitri Fontaine
http://2ndQuadrant.fr PostgreSQL : Expertise, Formation et Support
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] SQL command to edit postgresql.conf, with comments
Dimitri Fontaine dimi...@2ndquadrant.fr writes: Allow me to insist on this some more, because there's something important going on here. The other proposal (.auto) have a major failure mode that I don't think is acceptable. SET PERMANENT work_mem TO '8 MB'; select pg_reload_conf(); There's simply no way after this sequence to guess the current active value of work_mem, Um, other than show work_mem or select from pg_settings? The fact is that you cannot know the active value anyway without checking, because what you did with SET PERMANENT might be overridden in various session-local ways. The proposal for hand-edited versus machine-edited files just adds one more layer of possible overrides to the existing half-dozen layers, all of which are widely considered features not bugs. So I see no merit in your argument. I'm being told that we're talking about something over 200 files and that's too many. Yup, you're dead right about that. Backup/restore of configurations would become a real mess. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] SQL command to edit postgresql.conf, with comments
Dimitri Fontaine dimi...@2ndquadrant.fr writes:
I wonder if providing a system function to load some configuration
option from a file, using the current parser, would do it:
SELECT pg_load_conf('path/to/filename.conf', permament = true);
That seems like a pretty bizarre idea. The settings wouldn't be
persistent would they? Or are you proposing this as a substitute
way of providing input for SET PERMANENT? If so what's the value?
It seems to me that it just introduces unnecessary failure modes
(ie, server can't read file because of permissions) without any
really useful new functionality.
regards, tom lane
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] SQL command to edit postgresql.conf, with comments
Tom Lane t...@sss.pgh.pa.us writes: Um, other than show work_mem or select from pg_settings? The fact is that you cannot know the active value anyway without checking, because what you did with SET PERMANENT might be overridden in various session-local ways. The proposal for hand-edited versus machine-edited files just adds one more layer of possible overrides to the existing half-dozen layers, all of which are widely considered features not bugs. So I see no merit in your argument. I understand that. I just think there are already too many sources for GUCs and would welcome that there's a single possible source file with a complete remote editing feature. -- Dimitri Fontaine http://2ndQuadrant.fr PostgreSQL : Expertise, Formation et Support -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] SQL command to edit postgresql.conf, with comments
Tom Lane t...@sss.pgh.pa.us writes:
SELECT pg_load_conf('path/to/filename.conf', permament = true);
That seems like a pretty bizarre idea. The settings wouldn't be
persistent would they? Or are you proposing this as a substitute
way of providing input for SET PERMANENT? If so what's the value?
Yeah, it only has value if those two conditions are met:
1. there's a single source file possible for the configuration
2. it's not in the format you'd like it to be for easy hand editing
As the premise aren't reached, I agree such a function would have no value.
--
Dimitri Fontaine
http://2ndQuadrant.fr PostgreSQL : Expertise, Formation et Support
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] SQL command to edit postgresql.conf, with comments
Dimitri Fontaine dimi...@2ndquadrant.fr writes: I understand that. I just think there are already too many sources for GUCs and would welcome that there's a single possible source file with a complete remote editing feature. [ shrug... ] So don't use the option of hand-editing postgresql.conf. You're not being forced to do that, and on the other side of the coin, you shouldn't think that you get to force people who'd rather hand-edit to change their habits. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] SQL command to edit postgresql.conf, with comments
Tom Lane t...@sss.pgh.pa.us writes: [ shrug... ] So don't use the option of hand-editing postgresql.conf. You're not being forced to do that, and on the other side of the coin, you shouldn't think that you get to force people who'd rather hand-edit to change their habits. Sure. I just lose comments. I'll live with that. -- Dimitri Fontaine http://2ndQuadrant.fr PostgreSQL : Expertise, Formation et Support -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] SQL command to edit postgresql.conf, with comments
Sure. I just lose comments. I'll live with that. Actually, as part of this scheme, it would be nice if pg_settings had a comment column, which would be optionally set with SET PERMANENT. Not required, but nice to have. If we had that, I suspect that a lot fewer people would want a hand-edited file. -- -- Josh Berkus PostgreSQL Experts Inc. http://www.pgexperts.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] SQL command to edit postgresql.conf, with comments
On Thu, Oct 14, 2010 at 12:40 PM, Josh Berkus j...@agliodbs.com wrote: Sure. I just lose comments. I'll live with that. Actually, as part of this scheme, it would be nice if pg_settings had a comment column, which would be optionally set with SET PERMANENT. Not required, but nice to have. If we had that, I suspect that a lot fewer people would want a hand-edited file. I have to say that I'm woefully unimpressed by the idea of trying to do anything with comments other than ignore them, even something this simple. There's no obvious way to know which comments go with which settings. You can make up a rule, such as on the same line, but it's not necessarily going to be what people want. I think it's better to sidestep the issue entirely. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] SQL command to edit postgresql.conf, with comments
On Thu, Oct 14, 2010 at 9:50 AM, Tom Lane t...@sss.pgh.pa.us wrote:
The fact is that you cannot know the active value anyway without
checking, because what you did with SET PERMANENT might be overridden
in various session-local ways. The proposal for hand-edited versus
machine-edited files just adds one more layer of possible overrides
to the existing half-dozen layers, all of which are widely considered
features not bugs. So I see no merit in your argument.
You know, this is a good point. I was really hoping for a single
file, but maybe two files is not so bad as I think it is. However, I
kind of dislike SET PERMANENT as a command name, partly because I
think it sounds more certain than it really is, and partly because
it's asymmetric with the other, similar GUC-setting commands, which
are:
ALTER ROLE name [ IN DATABASE database_name ] SET
configuration_parameter { TO | = } { value | DEFAULT }
ALTER DATABASE name SET configuration_parameter { TO | = } { value | DEFAULT }
Perhaps ALTER SYSTEM SET configuration_parameter { TO | = } { value |
DEFAULT } ?
A similar syntax exists in Oracle:
http://stanford.edu/dept/itss/docs/oracle/10g/server.101/b10759/statements_2013.htm#i2282157
From what I gather this works out to:
ALTER SYSTEM SCOPE = { MEMORY | SPFILE | BOTH } SET
configuration_paramater = value
(I don't think we can the SCOPE clause because I believe the only way
we have of propagating a GUC through the system is to have all the
backends reread the file.)
--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] SQL command to edit postgresql.conf, with comments
On 10/14/2010 01:10 PM, Robert Haas wrote: On Thu, Oct 14, 2010 at 12:40 PM, Josh Berkusj...@agliodbs.com wrote: Sure. I just lose comments. I'll live with that. Actually, as part of this scheme, it would be nice if pg_settings had a comment column, which would be optionally set with SET PERMANENT. Not required, but nice to have. If we had that, I suspect that a lot fewer people would want a hand-edited file. I have to say that I'm woefully unimpressed by the idea of trying to do anything with comments other than ignore them, even something this simple. There's no obvious way to know which comments go with which settings. You can make up a rule, such as on the same line, but it's not necessarily going to be what people want. I think it's better to sidestep the issue entirely. If the file is completely machine-maintained, including the comments, it seems eminently doable. It's only if the file gets mangled by humans that there's a problem. cheers andrew -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] SQL command to edit postgresql.conf, with comments
Andrew Dunstan and...@dunslane.net writes: If the file is completely machine-maintained, including the comments, it seems eminently doable. It's only if the file gets mangled by humans that there's a problem. Yeah. We could have comments that were injected by some sort of COMMENT ON command, stored in the file in some suitably strict format, and then maintained mechanically. What we don't want to do is promise that hand editing of the file can coexist with machine updates. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] SQL command to edit postgresql.conf, with comments
Robert Haas robertmh...@gmail.com writes:
I kind of dislike SET PERMANENT as a command name, partly because I
think it sounds more certain than it really is, and partly because
it's asymmetric with the other, similar GUC-setting commands, which
are:
ALTER ROLE name [ IN DATABASE database_name ] SET
configuration_parameter { TO | = } { value | DEFAULT }
ALTER DATABASE name SET configuration_parameter { TO | = } { value | DEFAULT }
Perhaps ALTER SYSTEM SET configuration_parameter { TO | = } { value |
DEFAULT } ?
That might be a good idea. One argument for it is that you could
extend this syntax to include an optional comment, which would avoid
having to wedge an inherently non-transactional operation into
COMMENT ON. Say
ALTER SYSTEM SET foo = 'bar' COMMENT 'hacked by tgl 10/13/2010';
regards, tom lane
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] SQL command to edit postgresql.conf, with comments
Tom Lane t...@sss.pgh.pa.us writes: I agree with Josh's proposal: keep mechanically-generated settings in a separate file, and don't even pretend to allow comments to be kept there. And then, when you SET PERMANENT knob TO value (or whatever the syntax is), you never know what value is selected at next startup or SIGHUP. I know I'm alone on this, but I much prefer the all-machine-friendly proposal that still makes it possible to hand-edit the files. You get remote editing, comments, and the code is *very easy* to write. The only drawback is that we're not used to it so it might look odd, or outright ugly. I mean, the directory containing the files that you're not supposed to edit manually at all any more looks strange. How big a problem is that, when it allows for implementing the feature easily? Regards, -- Dimitri Fontaine http://2ndQuadrant.fr PostgreSQL : Expertise, Formation et Support -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] SQL command to edit postgresql.conf, with comments
On Wed, Oct 13, 2010 at 3:41 AM, Dimitri Fontaine dimi...@2ndquadrant.fr wrote: Tom Lane t...@sss.pgh.pa.us writes: I agree with Josh's proposal: keep mechanically-generated settings in a separate file, and don't even pretend to allow comments to be kept there. And then, when you SET PERMANENT knob TO value (or whatever the syntax is), you never know what value is selected at next startup or SIGHUP. I know I'm alone on this, but I much prefer the all-machine-friendly proposal that still makes it possible to hand-edit the files. You're not alone on this at all: I agree 100%. I don't like your proposed syntax, but I completely agree with your concern. I don't see what's wrong with having the initial contents of postgresql.conf look like this (these are the settings that are uncommented by default on my machine): # type man postgresql.conf for help on editing this file max_connections = 100 shared_buffers = 32MB datestyle = 'iso, mdy' lc_messages = 'en_US.UTF-8' lc_monetary = 'en_US.UTF-8' lc_numeric = 'en_US.UTF-8' lc_time = 'en_US.UTF-8' default_text_search_config = 'pg_catalog.english' When you type 'man postgresql.conf' it can tell you all of the things that are currently in the file as comments. It's just that they'll be in a man page rather in the file itself. I don't see what's bad about that. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] SQL command to edit postgresql.conf, with comments
Robert Haas robertmh...@gmail.com writes: You're not alone on this at all: I agree 100%. I don't like your proposed syntax, but I completely agree with your concern. I don't see what's wrong with having the initial contents of postgresql.conf look like this (these are the settings that are uncommented by default on my machine): # type man postgresql.conf for help on editing this file max_connections = 100 shared_buffers = 32MB datestyle = 'iso, mdy' lc_messages = 'en_US.UTF-8' lc_monetary = 'en_US.UTF-8' lc_numeric = 'en_US.UTF-8' lc_time = 'en_US.UTF-8' default_text_search_config = 'pg_catalog.english' I'm not sure if anybody is particularly against the initial contents looking like that. The big problem, which both you and Dimitri are conveniently ignoring, is that if people are allowed to hand-edit the file they are going to introduce comments that no mechanical parser will do a nice job of preserving. And they're not going to be happy when SET PERMANENT has a side-effect of throwing away their comments. I don't see anything particularly wrong with Josh's proposal of keeping machine-generated and person-generated text in separate files. Dimitri complains that the behavior will be confusing if there are conflicting settings, but I think that's hogwash. We already have the ability for pg_settings to tell you which file, and even which line, set the active value of the setting. It's not going to be hard for people to figure that out. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] SQL command to edit postgresql.conf, with comments
On 10/13/2010 10:25 AM, Tom Lane wrote: Robert Haasrobertmh...@gmail.com writes: You're not alone on this at all: I agree 100%. I don't like your proposed syntax, but I completely agree with your concern. I don't see what's wrong with having the initial contents of postgresql.conf look like this (these are the settings that are uncommented by default on my machine): # type man postgresql.conf for help on editing this file max_connections = 100 shared_buffers = 32MB datestyle = 'iso, mdy' lc_messages = 'en_US.UTF-8' lc_monetary = 'en_US.UTF-8' lc_numeric = 'en_US.UTF-8' lc_time = 'en_US.UTF-8' default_text_search_config = 'pg_catalog.english' I'm not sure if anybody is particularly against the initial contents looking like that. The big problem, which both you and Dimitri are conveniently ignoring, is that if people are allowed to hand-edit the file they are going to introduce comments that no mechanical parser will do a nice job of preserving. And they're not going to be happy when SET PERMANENT has a side-effect of throwing away their comments. I don't see anything particularly wrong with Josh's proposal of keeping machine-generated and person-generated text in separate files. Dimitri complains that the behavior will be confusing if there are conflicting settings, but I think that's hogwash. We already have the ability for pg_settings to tell you which file, and even which line, set the active value of the setting. It's not going to be hard for people to figure that out. +1. Preserving the comments when you change the value could make the comments totally bogus. Separating machine-generated values into a separate file makes plenty of sense to me. Which one wins, though? I can see cases being made for both. cheers andrew -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] SQL command to edit postgresql.conf, with comments
Andrew Dunstan and...@dunslane.net writes: +1. Preserving the comments when you change the value could make the comments totally bogus. Separating machine-generated values into a separate file makes plenty of sense to me. Which one wins, though? I can see cases being made for both. IIRC the proposal was that postgresql.conf (the people-editable file) would have include postgresql.auto in it. You could put that at the top, bottom, or even middle to control how the priority works. So it's user-configurable. I think the factory default ought to be to have it at the top, which would result in manually edited settings (if any) overriding SET PERMANENT. Basically the way I'd like to see this go is that SET PERMANENT gets attached on the side of what's there now, and people who are used to the old way don't have to change their habits at all. If the new way is as much better as its advocates claim, use of manual editing of postgresql.conf will soon die off; then at some future date we could consider whether to remove that file or at least delete all the comments it contains out-of-the-box. About the only change I want to make immediately is that initdb ought to shove its settings into postgresql.auto instead of mucking with postgresql.conf. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] SQL command to edit postgresql.conf, with comments
On Oct 13, 2010, at 10:24 AM, Tom Lane wrote: Andrew Dunstan and...@dunslane.net writes: +1. Preserving the comments when you change the value could make the comments totally bogus. Separating machine-generated values into a separate file makes plenty of sense to me. Which one wins, though? I can see cases being made for both. IIRC the proposal was that postgresql.conf (the people-editable file) would have include postgresql.auto in it. You could put that at the top, bottom, or even middle to control how the priority works. So it's user-configurable. I think the factory default ought to be to have it at the top, which would result in manually edited settings (if any) overriding SET PERMANENT. Since this is just touching the local servers' postgresql.conf.auto (or whatever), any reason why SET PERMANENT couldn't be used on a read-only standby? Could this be to manage some of the failover scenarios (i.e., setting any relevant config from a central clusterware|whatever)? Regards, David -- David Christensen End Point Corporation da...@endpoint.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] SQL command to edit postgresql.conf, with comments
Tom Lane t...@sss.pgh.pa.us writes: I'm not sure if anybody is particularly against the initial contents looking like that. The big problem, which both you and Dimitri are conveniently ignoring, is that if people are allowed to hand-edit the file they are going to introduce comments that no mechanical parser will do a nice job of preserving. IMO the only reason why my proposal is sound is that is address the point. Consider: cat postgresql.conf.d/work_mem 16MB This database needs at least such a value. Note it's overridden in some ROLEs setup. With such a format (name is filename, value is first line content's, rest is comments), it's easy to preserve comments and have them machine editable. What do I miss? Regards, -- Dimitri Fontaine http://2ndQuadrant.fr PostgreSQL : Expertise, Formation et Support -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] SQL command to edit postgresql.conf, with comments
On Wed, Oct 13, 2010 at 10:25 AM, Tom Lane t...@sss.pgh.pa.us wrote: Robert Haas robertmh...@gmail.com writes: You're not alone on this at all: I agree 100%. I don't like your proposed syntax, but I completely agree with your concern. I don't see what's wrong with having the initial contents of postgresql.conf look like this (these are the settings that are uncommented by default on my machine): # type man postgresql.conf for help on editing this file max_connections = 100 shared_buffers = 32MB datestyle = 'iso, mdy' lc_messages = 'en_US.UTF-8' lc_monetary = 'en_US.UTF-8' lc_numeric = 'en_US.UTF-8' lc_time = 'en_US.UTF-8' default_text_search_config = 'pg_catalog.english' I'm not sure if anybody is particularly against the initial contents looking like that. The big problem, which both you and Dimitri are conveniently ignoring, is that if people are allowed to hand-edit the file they are going to introduce comments that no mechanical parser will do a nice job of preserving. And they're not going to be happy when SET PERMANENT has a side-effect of throwing away their comments. But creating a separate file doesn't fix that problem. It just moves it around. If people will expect comments in postgresql.conf to get preserved, then why won't they also expect comments in postgresql.conf.auto to get preserved? If the answer is because postgresql.conf has always worked that way before, then add one more line to the proposed initial contents saying it's not true any more. It can be the same line you were going to put in postgresql.conf.auto explaining the same fact. # The SQL command SET PERMANENT rewrites this file and may move or remove comment lines. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] SQL command to edit postgresql.conf, with comments
Robert Haas robertmh...@gmail.com writes: But creating a separate file doesn't fix that problem. It just moves it around. If people will expect comments in postgresql.conf to get preserved, then why won't they also expect comments in postgresql.conf.auto to get preserved? Because postgresql.conf.auto will have a nice leading comment telling people (1) not to hand-edit the file, (2) if they do so anyway, not to expect comments to be preserved, and (3) the place to do manual editing of settings is postgresql.conf. If the answer is because postgresql.conf has always worked that way before, then add one more line to the proposed initial contents saying it's not true any more. Sorry, wrong answer. The objection to this is not whether you tell people that you're taking away the ability to keep useful comments in postgresql.conf, it's that you're taking away the ability. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] SQL command to edit postgresql.conf, with comments
IIRC the proposal was that postgresql.conf (the people-editable file) would have include postgresql.auto in it. You could put that at the top, bottom, or even middle to control how the priority works. So it's user-configurable. I think the factory default ought to be to have it at the top, which would result in manually edited settings (if any) overriding SET PERMANENT. Right, I think that's the behavior which will result in the least newbie confusion. So what we'd add to postgresql.conf would look something like this: # Link to auto-generated configuration file. # Do not edit the auto-generated file or remove this link; # instead, edit settings below in this file and they will # override the auto-generated file. include 'postgresql.conf.auto' the only change I want to make immediately is that initdb ought to shove its settings into postgresql.auto instead of mucking with postgresql.conf. That would be good. -- -- Josh Berkus PostgreSQL Experts Inc. http://www.pgexperts.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] SQL command to edit postgresql.conf, with comments
On Wed, Oct 13, 2010 at 12:56:15PM -0400, Tom Lane wrote: Robert Haas robertmh...@gmail.com writes: But creating a separate file doesn't fix that problem. It just moves it around. If people will expect comments in postgresql.conf to get preserved, then why won't they also expect comments in postgresql.conf.auto to get preserved? Because postgresql.conf.auto will have a nice leading comment telling people (1) not to hand-edit the file, (2) if they do so anyway, not to expect comments to be preserved, and (3) the place to do manual editing of settings is postgresql.conf. If the answer is because postgresql.conf has always worked that way before, then add one more line to the proposed initial contents saying it's not true any more. Sorry, wrong answer. The objection to this is not whether you tell people that you're taking away the ability to keep useful comments in postgresql.conf, it's that you're taking away the ability. regards, tom lane I like this approach. I was just wondering if there is a simple tweak to this schema to make it work more easily with standbys. If there was a GUC that controlled the 'auto filename' and it could expand something like %h to hostname (or name if we had something like standby registration). This would allow each standby to store its local settings in a different location and have something like a unified set of config files. I suppose something like symlinking postgresql.auto hostname.auto on each machine might achieve a similar effect... Garick -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] SQL command to edit postgresql.conf, with comments
On 10/12/2010 05:02 PM, Dimitri Fontaine wrote: So, what you do is have a file per GUC, file name is the GUC name, first line contains *ONLY* current value, the rest of the file is comments. You're joking, right? cheers andrew -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] SQL command to edit postgresql.conf, with comments
Andrew Dunstan and...@dunslane.net writes: On 10/12/2010 05:02 PM, Dimitri Fontaine wrote: So, what you do is have a file per GUC, file name is the GUC name, first line contains *ONLY* current value, the rest of the file is comments. You're joking, right? No. I just want both comments and SQL commands. If you refuse this simple file scheme, keep your postgresql.conf and don't remote edit it. That's my proposal, I'm happy that it comes with laughter :) -- Dimitri Fontaine http://2ndQuadrant.fr PostgreSQL : Expertise, Formation et Support -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] SQL command to edit postgresql.conf, with comments
Dimitri Fontaine wrote: Andrew Dunstan and...@dunslane.net writes: On 10/12/2010 05:02 PM, Dimitri Fontaine wrote: So, what you do is have a file per GUC, file name is the GUC name, first line contains *ONLY* current value, the rest of the file is comments. You're joking, right? No. I just want both comments and SQL commands. If you refuse this simple file scheme, keep your postgresql.conf and don't remote edit it. That's my proposal, I'm happy that it comes with laughter :) Maybe I missed something important, but why is it not possible to retain the single existing postgres.conf file format (human writable) *and* have it machine/SQL-editable *and* maintain the comments? I should think that it would be possible to do all of these without too much trouble. All you would need is for the file parser to retain the comments as metadata, include them in the relations that the SQL commands see where the latter can also edit them as data, and then write out the updated file with comments. The fact that Postgres already explicitly supports comment metadata in its system catalog means it must already know something about this. If something is missing, then expand the catalog so it represents all the details you want to preserve. -- Darren Duncan -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] SQL command to edit postgresql.conf, with comments
Darren Duncan dar...@darrenduncan.net wrote: why is it not possible to retain the single existing postgres.conf file format (human writable) *and* have it machine/SQL-editable *and* maintain the comments? My recollection of the previous discussion wasn't that there was no way to do it, but rather that each person could thing of a somewhat different way to do it and no two people agreed on which was best, so no consensus was reached. -Kevin -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] SQL command to edit postgresql.conf, with comments
Darren, All, All you would need is for the file parser to retain the comments as metadata, include them in the relations that the SQL commands see where the latter can also edit them as data, and then write out the updated file with comments. All you need to do in order to trisect and angle is to divide it into three equal parts using just a compass and straightedge. Given the number of different ways people can comment up pg.conf files, and the fact that they can have the same settings set multiple times in the file, both commented and uncommented, and that both the settings and the comments can be in any order, with user-created spacing, you've added a *huge* burden of parsing code to any .conf updater. One which would, in terms of LOC, exceed the updating code by more than 300%. The fact that Postgres already explicitly supports comment metadata in its system catalog means it must already know something about this. We support what? The solution to this is simple, and was previously discussed on this list: (a) have a postgresql.conf.auto (b) add a default include for postgresql.conf.auto at the beginning of PostgreSQL.conf (c) SQL updates go to postgresql.conf.auto, which consists only of setting = value in alphabetical order. (d) We document that settings which are changed manually in postgresql.conf will override postgresql.conf.auto. (e) $$profit$$!! --Josh Berkus -- -- Josh Berkus PostgreSQL Experts Inc. http://www.pgexperts.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] SQL command to edit postgresql.conf, with comments
Josh Berkus wrote: The fact that Postgres already explicitly supports comment metadata in its system catalog means it must already know something about this. We support what? Postgres has COMMENT ON ... SQL for various database objects and I assumed that said comments would be stored in the system catalog. http://www.postgresql.org/docs/9.0/interactive/sql-comment.html The solution to this is simple, and was previously discussed on this list: (a) have a postgresql.conf.auto (b) add a default include for postgresql.conf.auto at the beginning of PostgreSQL.conf (c) SQL updates go to postgresql.conf.auto, which consists only of setting = value in alphabetical order. (d) We document that settings which are changed manually in postgresql.conf will override postgresql.conf.auto. (e) $$profit$$!! I agree that this looks like an effective solution. -- Darren Duncan -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] SQL command to edit postgresql.conf, with comments
Postgres has COMMENT ON ... SQL for various database objects and I assumed that said comments would be stored in the system catalog. Oh. Now that's an interesting perspective ... you're suggesting that we take the comments and apply them as COMMENTS on the specific pg_settings? That wouldn't solve the issues of ordering, or of comments in the file not associated with a setting, but might be a good 80% solution. -- -- Josh Berkus PostgreSQL Experts Inc. http://www.pgexperts.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] SQL command to edit postgresql.conf, with comments
On Tue, Oct 12, 2010 at 3:54 PM, Josh Berkus j...@agliodbs.com wrote: Oh. Now that's an interesting perspective ... you're suggesting that we take the comments and apply them as COMMENTS on the specific pg_settings? On a side note regarding comments, I'd like to make a request for a more comprehensive commenting mechanism. The first though that comes to my mind would allow for comments to be stored and annotated using XML or sgml. It'd be nice to be able to generate user documentation from selected comments taken from application derived database objects. I don't know, maybe this is already possible. -- Regards, Richard Broersma Jr. Visit the Los Angeles PostgreSQL Users Group (LAPUG) http://pugs.postgresql.org/lapug -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] SQL command to edit postgresql.conf, with comments
Josh Berkus j...@agliodbs.com writes: Postgres has COMMENT ON ... SQL for various database objects and I assumed that said comments would be stored in the system catalog. Oh. Now that's an interesting perspective ... you're suggesting that we take the comments and apply them as COMMENTS on the specific pg_settings? Well, if the settings were stored in a catalog ... which they are not ... that might be a useful idea. The reason that COMMENT ON isn't a terribly helpful analogy is that it puts the burden on the user to associate specific comment texts with specific database objects. The problem with parsing and modifying freestyle comments as seen in postgresql.conf is exactly (or mostly) that there's no reasonable way of mechanically determining which comments are associated with which setting. We could invent some arbitrary rule or other, but it'd have little to do with the way people perceive what's in such a file. I agree with Josh's proposal: keep mechanically-generated settings in a separate file, and don't even pretend to allow comments to be kept there. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] SQL command to edit postgresql.conf, with comments
Josh Berkus wrote first: Postgres has COMMENT ON ... SQL for various database objects and I assumed that said comments would be stored in the system catalog. Oh. Now that's an interesting perspective ... you're suggesting that we take the comments and apply them as COMMENTS on the specific pg_settings? Yes, something along those lines. I should point out that in general I consider a COMMENT ON to just be a less ambiguous form of a -- or /* */ and that it would be a good idea for a code or config parser to preserve comments given in the latter formats in the same way it preserves the former when it can reasonably infer what to associate the latter comments with. That wouldn't solve the issues of ordering, or of comments in the file not associated with a setting, but might be a good 80% solution. Yes, and for something like this an 80% solution is good. As for ordering, that can be handled with more metadata, like a visual order number. Tom Lane wrote second: Well, if the settings were stored in a catalog ... which they are not ... that might be a useful idea. The catalog is where they *should* be stored, at least from the user's perspective. The reason that COMMENT ON isn't a terribly helpful analogy is that it puts the burden on the user to associate specific comment texts with specific database objects. The problem with parsing and modifying freestyle comments as seen in postgresql.conf is exactly (or mostly) that there's no reasonable way of mechanically determining which comments are associated with which setting. We could invent some arbitrary rule or other, but it'd have little to do with the way people perceive what's in such a file. There's no reason that you can't have both kinds of comments. You could have comments that are specially formatted such that the parser will then recognize they are associated with something specific and so put them in the system catalog. And then you can have other comments not formatted that way and the parser will then pass over them like whitespace. I agree with Josh's proposal: keep mechanically-generated settings in a separate file, and don't even pretend to allow comments to be kept there. I agree with the separate files approach as being the most practical in at least the short term. However, I think that this file could support comments too, but they would just be limited to end-of-line comments on the same line as the setting, and it would be only these comments appearing in the system catalog by default. Comments in the file for user editing would only appear in the catalog if specially formatted, which for now could just mean taking end-of-line comments. -- Darren Duncan -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] SQL command to edit postgresql.conf, with comments
Richard Broersma wrote: On Tue, Oct 12, 2010 at 3:54 PM, Josh Berkus j...@agliodbs.com wrote: Oh. Now that's an interesting perspective ... you're suggesting that we take the comments and apply them as COMMENTS on the specific pg_settings? On a side note regarding comments, I'd like to make a request for a more comprehensive commenting mechanism. The first though that comes to my mind would allow for comments to be stored and annotated using XML or sgml. It'd be nice to be able to generate user documentation from selected comments taken from application derived database objects. I don't know, maybe this is already possible. When you start going there, you have new issues to consider. (For the record I also prefer plain text for comments.) I should point out that the group making the Perl 6 language has already been looking into such matters extensively of essentially unifying code comments and code documentation into a common metadata both accessible in the source code and programmatically at runtime. I think this approach goes beyond comments as we normally know them, which I just think of plain text strings associated with some code element. But if you want to pursue bringing documentation into this, I suggest looking at what Perl 6, and other languages, have done. While some of the results of the Perl 6 discussion may have just been in the forums, these urls at least are related to it: - http://perlcabal.org/syn/S02.html#Whitespace_and_Comments - http://github.com/perl6/specs/raw/master/S26-documentation.pod for I'm not proposing adopting their syntax, but some features or design may be useful to learn from. I also want to point out that the FoxPro language constitutes some prior art about including comments as data fields in their runtime-accessible objects. -- Darren Duncan -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
