Re: [HACKERS] proposal: a validator for configuration files

2011-07-25 Thread Alexey Klyukin
On Jul 16, 2011, at 9:55 PM, Tom Lane wrote: I wrote: I think that it might be sensible to have the following behavior: 1. Parse the file, where parse means collect all the name = value pairs. Bail out if we find any syntax errors at that level of detail. (With this patch, we could

Re: [HACKERS] proposal: a validator for configuration files

2011-07-23 Thread Dimitri Fontaine
Florian Pflug f...@phlo.org writes: A variant of this would be to allow extensions (in the CREATE EXTENSION sense) to declare custom GUCs in their control file. Then we'd only need to load those files, which seems better than loading a shared library. The original patch for the extensions had

Re: [HACKERS] proposal: a validator for configuration files

2011-07-19 Thread Tom Lane
Peter Eisentraut pete...@gmx.net writes: On sön, 2011-07-17 at 00:59 -0400, Tom Lane wrote: Well, we *do* have a C API for that, of a sort. The problem is, what do you do in processes that have not loaded the relevant extension? Those processes that have the extension loaded check the

Re: [HACKERS] proposal: a validator for configuration files

2011-07-19 Thread Tom Lane
Josh Berkus j...@agliodbs.com writes: Hmmm. As someone who often deploys pg.conf changes as part of a production code rollout, I actually like the atomic nature of updating postgresql.conf -- that is, all your changes succeed, or they all fail. If we actually *had* that, I'd agree with you.

Re: [HACKERS] proposal: a validator for configuration files

2011-07-18 Thread Tom Lane
Robert Haas robertmh...@gmail.com writes: On Sun, Jul 17, 2011 at 12:59 AM, Tom Lane t...@sss.pgh.pa.us wrote: I agree custom_variable_classes is conceptually messy, but it's a reasonably lightweight compromise that gives some error checking without requiring a lot of possibly-irrelevant

Re: [HACKERS] proposal: a validator for configuration files

2011-07-18 Thread Josh Berkus
Tom, Florian, On the downside, the current behaviour prevents problems if someone changes two interrelated GUCs, but makes a mistake at one of them. For example, someone might drastically lower bgwriter_delay but might botch the matching adjustment of bgwriter_lru_maxpages. That's a fair

Re: [HACKERS] proposal: a validator for configuration files

2011-07-18 Thread Peter Eisentraut
On sön, 2011-07-17 at 00:59 -0400, Tom Lane wrote: Well, we *do* have a C API for that, of a sort. The problem is, what do you do in processes that have not loaded the relevant extension? Those processes that have the extension loaded check the parameter settings in their namespace, those that

Re: [HACKERS] proposal: a validator for configuration files

2011-07-17 Thread Robert Haas
On Sun, Jul 17, 2011 at 12:59 AM, Tom Lane t...@sss.pgh.pa.us wrote: Robert Haas robertmh...@gmail.com writes: On Sat, Jul 16, 2011 at 10:04 PM, Tom Lane t...@sss.pgh.pa.us wrote: Robert Haas robertmh...@gmail.com writes: Is there any way that we could get *rid* of custom_variable_classes?

Re: [HACKERS] proposal: a validator for configuration files

2011-07-17 Thread Florian Pflug
On Jul18, 2011, at 01:29 , Robert Haas wrote: Hmm. Maybe what we need is a mechanism that allows the configuration to be associated a loadable module, and whenever that module is loaded, we also load the associated configuration settings. This is probably terribly syntax, but something like:

Re: [HACKERS] proposal: a validator for configuration files

2011-07-17 Thread Pavel Stehule
2011/7/18 Florian Pflug f...@phlo.org: On Jul18, 2011, at 01:29 , Robert Haas wrote: Hmm.  Maybe what we need is a mechanism that allows the configuration to be associated a loadable module, and whenever that module is loaded, we also load the associated configuration settings.  This is

Re: [HACKERS] proposal: a validator for configuration files

2011-07-16 Thread Tom Lane
I wrote: I think that it might be sensible to have the following behavior: 1. Parse the file, where parse means collect all the name = value pairs. Bail out if we find any syntax errors at that level of detail. (With this patch, we could report some or all of the syntax errors first.) 2.

Re: [HACKERS] proposal: a validator for configuration files

2011-07-16 Thread Florian Pflug
On Jul16, 2011, at 20:55 , Tom Lane wrote: The original argument for the current behavior was to avoid applying settings from a thoroughly munged config file, but I think that the checks involved in steps 1-3 would be sufficient to reject files that had major problems. It's possible that step

Re: [HACKERS] proposal: a validator for configuration files

2011-07-16 Thread Tom Lane
Florian Pflug f...@phlo.org writes: On the downside, the current behaviour prevents problems if someone changes two interrelated GUCs, but makes a mistake at one of them. For example, someone might drastically lower bgwriter_delay but might botch the matching adjustment of

Re: [HACKERS] proposal: a validator for configuration files

2011-07-16 Thread Florian Pflug
On Jul16, 2011, at 21:23 , Tom Lane wrote: Florian Pflug f...@phlo.org writes: On the downside, the current behaviour prevents problems if someone changes two interrelated GUCs, but makes a mistake at one of them. For example, someone might drastically lower bgwriter_delay but might botch the

Re: [HACKERS] proposal: a validator for configuration files

2011-07-16 Thread Tom Lane
Florian Pflug f...@phlo.org writes: Btw, if we touch that, I think we should think about providing some way to detect when a backend fails to apply a value. Hm, maybe, but keep in mind that there are valid reasons for a backend to ignore a postgresql.conf setting --- in particular, it might

Re: [HACKERS] proposal: a validator for configuration files

2011-07-16 Thread Florian Pflug
On Jul16, 2011, at 22:55 , Tom Lane wrote: Florian Pflug f...@phlo.org writes: Btw, if we touch that, I think we should think about providing some way to detect when a backend fails to apply a value. Hm, maybe, but keep in mind that there are valid reasons for a backend to ignore a

Re: [HACKERS] proposal: a validator for configuration files

2011-07-16 Thread Robert Haas
On Fri, Jul 15, 2011 at 8:02 PM, Tom Lane t...@sss.pgh.pa.us wrote: 2. Tentatively apply the new custom_variable_classes setting if any. Is there any way that we could get *rid* of custom_variable_classes? The idea of using a GUC to define the set of valid GUCs seems intrinsically problematic.

Re: [HACKERS] proposal: a validator for configuration files

2011-07-16 Thread Tom Lane
Robert Haas robertmh...@gmail.com writes: On Fri, Jul 15, 2011 at 8:02 PM, Tom Lane t...@sss.pgh.pa.us wrote: 2. Tentatively apply the new custom_variable_classes setting if any. Is there any way that we could get *rid* of custom_variable_classes? The idea of using a GUC to define the set of

Re: [HACKERS] proposal: a validator for configuration files

2011-07-16 Thread Robert Haas
On Sat, Jul 16, 2011 at 10:04 PM, Tom Lane t...@sss.pgh.pa.us wrote: Robert Haas robertmh...@gmail.com writes: On Fri, Jul 15, 2011 at 8:02 PM, Tom Lane t...@sss.pgh.pa.us wrote: 2. Tentatively apply the new custom_variable_classes setting if any. Is there any way that we could get *rid* of

Re: [HACKERS] proposal: a validator for configuration files

2011-07-16 Thread Tom Lane
Robert Haas robertmh...@gmail.com writes: On Sat, Jul 16, 2011 at 10:04 PM, Tom Lane t...@sss.pgh.pa.us wrote: Robert Haas robertmh...@gmail.com writes: Is there any way that we could get *rid* of custom_variable_classes? Well, we could just drop it and say you can set any dotted-name GUC

Re: [HACKERS] proposal: a validator for configuration files

2011-07-15 Thread Alvaro Herrera
Excerpts from Alexey Kluykin's message of jue jul 14 09:18:15 -0400 2011: On Jul 14, 2011, at 4:38 AM, Alvaro Herrera wrote: On Jul14, 2011, at 01:38 , Alvaro Herrera wrote: This is happening because a check for total number of errors so far is happening only after coming across at

Re: [HACKERS] proposal: a validator for configuration files

2011-07-15 Thread Tom Lane
Alvaro Herrera alvhe...@commandprompt.com writes: Excerpts from Alexey Kluykin's message of jue jul 14 09:18:15 -0400 2011: This is happening because a check for total number of errors so far is happening only after coming across at least one non-recognized configuration option. What about

Re: [HACKERS] proposal: a validator for configuration files

2011-07-14 Thread Alexey Klyukin
On Jul 14, 2011, at 4:38 AM, Alvaro Herrera wrote: Excerpts from Florian Pflug's message of mié jul 13 20:12:28 -0400 2011: On Jul14, 2011, at 01:38 , Alvaro Herrera wrote: One strange thing here is that you could get two such messages; say if a file has 100 parse errors and there are also

Re: [HACKERS] proposal: a validator for configuration files

2011-07-13 Thread Alvaro Herrera
Excerpts from Alexey Kluykin's message of mar jun 21 07:43:02 -0400 2011: Another benefit of removing the check is that it reduces code complexity. Maybe not when measured in line counts, but it's one less outside factor that changes ProcessConfigFiles()'s behaviour and thus one thing

Re: [HACKERS] proposal: a validator for configuration files

2011-07-13 Thread Florian Pflug
On Jul14, 2011, at 01:38 , Alvaro Herrera wrote: One strange thing here is that you could get two such messages; say if a file has 100 parse errors and there are also valid lines that contain bogus settings (foo = bar). I don't find this to be too problematic, and I think fixing it would be

Re: [HACKERS] proposal: a validator for configuration files

2011-07-13 Thread Alvaro Herrera
Excerpts from Florian Pflug's message of mié jul 13 20:12:28 -0400 2011: On Jul14, 2011, at 01:38 , Alvaro Herrera wrote: One strange thing here is that you could get two such messages; say if a file has 100 parse errors and there are also valid lines that contain bogus settings (foo =

Re: [HACKERS] proposal: a validator for configuration files

2011-06-21 Thread Alexey Klyukin
On Jun 20, 2011, at 6:22 PM, Florian Pflug wrote: On Jun20, 2011, at 17:02 , Alexey Klyukin wrote: I don't think it has changed at all. Previously, we did goto cleanup_list (or cleanup_exit in ParseConfigFp) right after the first error, no matter whether that was a postmaster or its child.

Re: [HACKERS] proposal: a validator for configuration files

2011-06-20 Thread Alexey Klyukin
Florian, On Jun 18, 2011, at 5:40 PM, Florian Pflug wrote: On Jun16, 2011, at 22:34 , Alexey Klyukin wrote: Attached is the v2 of the patch to show all parse errors at postgresql.conf. Changes (per review and suggestions from Florian): - do not stop on the first error during postmaster's

Re: [HACKERS] proposal: a validator for configuration files

2011-06-20 Thread Florian Pflug
On Jun20, 2011, at 17:02 , Alexey Klyukin wrote: On Jun 18, 2011, at 5:40 PM, Florian Pflug wrote: On Jun16, 2011, at 22:34 , Alexey Klyukin wrote: Attached is the v2 of the patch to show all parse errors at postgresql.conf. Changes (per review and suggestions from Florian): - do not stop

Re: [HACKERS] proposal: a validator for configuration files

2011-06-20 Thread Tom Lane
Florian Pflug f...@phlo.org writes: The code the actually implements the check settings first, apply later logic isn't easy to read. Now, assume that this code has a bug. Then, with your patch applied, we might end up with the postmaster applying a setting (because it didn't abort early) but

Re: [HACKERS] proposal: a validator for configuration files

2011-06-20 Thread Florian Pflug
On Jun20, 2011, at 18:16 , Tom Lane wrote: Florian Pflug f...@phlo.org writes: The code the actually implements the check settings first, apply later logic isn't easy to read. Now, assume that this code has a bug. Then, with your patch applied, we might end up with the postmaster applying a

Re: [HACKERS] proposal: a validator for configuration files

2011-06-20 Thread Tom Lane
Florian Pflug f...@phlo.org writes: On Jun20, 2011, at 18:16 , Tom Lane wrote: This is already known to happen: there are cases where the postmaster and a backend can come to different conclusions about whether a setting is valid (eg, because it depends on database encoding). Whether that's

Re: [HACKERS] proposal: a validator for configuration files

2011-06-18 Thread Florian Pflug
On Jun16, 2011, at 22:34 , Alexey Klyukin wrote: Attached is the v2 of the patch to show all parse errors at postgresql.conf. Changes (per review and suggestions from Florian): - do not stop on the first error during postmaster's start. - fix errors in processing files from include

Re: [HACKERS] proposal: a validator for configuration files

2011-06-16 Thread Florian Pflug
Hi On May14, 2011, at 00:49 , Alexey Klyukin wrote: The patch forces the parser to report all errors (max 100) from the ProcessConfigFile/ParseConfigFp. Currently, only the first parse error or an invalid directive is reported. Reporting all of them is crucial to automatic validation of

Re: [HACKERS] proposal: a validator for configuration files

2011-06-16 Thread Alexey Klyukin
Florian, On Jun 16, 2011, at 2:34 PM, Florian Pflug wrote: Hi On May14, 2011, at 00:49 , Alexey Klyukin wrote: The patch forces the parser to report all errors (max 100) from the ProcessConfigFile/ParseConfigFp. Currently, only the first parse error or an invalid directive is reported.

Re: [HACKERS] proposal: a validator for configuration files

2011-06-16 Thread Florian Pflug
On Jun16, 2011, at 17:23 , Alexey Klyukin wrote: On Jun 16, 2011, at 2:34 PM, Florian Pflug wrote: The first problem I ran into when I tried to test this is that it *only* reports multiple errors during config file reload on SIHUP, not during postmaster startup. I guess it's been done that way

Re: [HACKERS] proposal: a validator for configuration files

2011-06-16 Thread Alexey Klyukin
On Jun 16, 2011, at 6:49 PM, Florian Pflug wrote: On Jun16, 2011, at 17:23 , Alexey Klyukin wrote: On Jun 16, 2011, at 2:34 PM, Florian Pflug wrote: The first problem I ran into when I tried to test this is that it *only* reports multiple errors during config file reload on SIHUP, not during

Re: [HACKERS] proposal: a validator for configuration files

2011-06-16 Thread Florian Pflug
On Jun16, 2011, at 18:46 , Alexey Klyukin wrote: On Jun 16, 2011, at 6:49 PM, Florian Pflug wrote: Hm, wouldn't a test for context == PGC_POSTMASTER be more appropriate? In such a case the errors caused by command-line arguments won't stop the postmaster. PGC_S_FILE seems to handle this

Re: [HACKERS] proposal: a validator for configuration files

2011-06-16 Thread Alexey Klyukin
On Jun 16, 2011, at 8:01 PM, Florian Pflug wrote: On Jun16, 2011, at 18:46 , Alexey Klyukin wrote: On Jun 16, 2011, at 6:49 PM, Florian Pflug wrote: Hm, wouldn't a test for context == PGC_POSTMASTER be more appropriate? In such a case the errors caused by command-line arguments won't stop

Re: [HACKERS] proposal: a validator for configuration files

2011-06-16 Thread Florian Pflug
On Jun16, 2011, at 20:14 , Alexey Klyukin wrote: On Jun 16, 2011, at 8:01 PM, Florian Pflug wrote: On Jun16, 2011, at 18:46 , Alexey Klyukin wrote: I just recalled a reason for counting the total number of errors. There is a condition that checks that the total number of errors is less than

Re: [HACKERS] proposal: a validator for configuration files

2011-06-16 Thread Alexey Klyukin
Hi, On Jun 16, 2011, at 9:18 PM, Florian Pflug wrote: On Jun16, 2011, at 20:14 , Alexey Klyukin wrote: Well, while thinking about this I decided to leave the counter for the ParseConfigFp, but drop it in ProcessConfigFile. The case we are protecting against is a single file full of

Re: [HACKERS] proposal: a validator for configuration files

2011-05-13 Thread Alexey Klyukin
Hi, On Apr 14, 2011, at 9:50 PM, Robert Haas wrote: On Mon, Apr 4, 2011 at 2:03 PM, Alexey Klyukin al...@commandprompt.com wrote: Here's the update of Selena's patch, which also shows all errors in configuration parameters (as well as parser errors) during reload. You should add this

Re: [HACKERS] proposal: a validator for configuration files

2011-04-14 Thread Robert Haas
On Mon, Apr 4, 2011 at 2:03 PM, Alexey Klyukin al...@commandprompt.com wrote: Here's the update of Selena's patch, which also shows all errors in configuration parameters (as well as parser errors) during reload. You should add this here:

Re: [HACKERS] proposal: a validator for configuration files

2011-04-04 Thread Alexey Klyukin
On Apr 1, 2011, at 12:08 AM, Alexey Klyukin wrote: Hi Selena, On Mar 30, 2011, at 11:42 PM, Selena Deckelmann wrote: Hi! On Wed, Mar 30, 2011 at 8:40 AM, Alexey Klyukin al...@commandprompt.com wrote: I did a little bit of work on this, and we discussed it here:

Re: [HACKERS] proposal: a validator for configuration files

2011-03-31 Thread Alexey Klyukin
Hi Selena, On Mar 30, 2011, at 11:42 PM, Selena Deckelmann wrote: Hi! On Wed, Mar 30, 2011 at 8:40 AM, Alexey Klyukin al...@commandprompt.com wrote: I did a little bit of work on this, and we discussed it here: http://archives.postgresql.org/pgsql-hackers/2009-03/msg00345.php

Re: [HACKERS] proposal: a validator for configuration files

2011-03-30 Thread Selena Deckelmann
Hi! On Wed, Mar 30, 2011 at 8:40 AM, Alexey Klyukin al...@commandprompt.com wrote: I'd like to add an ability to validate the corectness of PostgreSQL configuration files, i.e. postgresql.conf, pg_hba.conf, pg_ident.conf without actually applying them. The idea is that it would be a