Re: [HACKERS] Generating config stuff from single source
Martijn van Oosterhout wrote: > The input file could be simply line based. Attached is a simple > script that takes the input below and produces something resembling > what you suggested. It wouldn't be too hard to get it to produce > multiple output formats and dump the output to different files... > > > Group: Query Tuning > Subgroup: Planner Method Configuration > > Name: enable_hashagg > Context: userset > > ... etc ... > I'm trying out this input format now, processing it with Perl, and it seems to be quite reasonable. There are a number of special cases, however, that need to be worked out. I'm trying to generate guc.c now and once that works I'll report here. -- Peter Eisentraut http://developer.postgresql.org/~petere/ ---(end of broadcast)--- TIP 5: don't forget to increase your free space map settings
Re: [HACKERS] Generating config stuff from single source
On Thu, Feb 16, 2006 at 10:52:19AM -0500, Tom Lane wrote: > Peter Eisentraut <[EMAIL PROTECTED]> writes: > > Am Donnerstag, 16. Februar 2006 02:50 schrieb Tom Lane: > >> That's fine for users, but what new demands are you about to place on > >> developers? Does this require tools not already needed in order to > >> build from a CVS pull? (There's sure no xsltproc on this machine...) > > > It is to be expected that sooner or later we'll move from SGML to XML > > documentation builds, at which point xsltproc will become a > > semi-requirement > > anyway. I don't think this requirement is too onerous; libxslt is portable > > and easy to install. > > Forgot to mention, but: I don't find the above argument very convincing. > The buildfarm machines are not expected to build documentation, and many > developers seem not to have installed doc tools either. So I think this > would be raising the bar another notch in terms of what's required to do > development or testing, even if it does overlap with docs-build needs. >From what I've seen it's not terribly difficult to install some sort of XSLT processor now-a-days. It's certainly less involved than installing docbook in any case. -- Jim C. Nasby, Sr. Engineering Consultant [EMAIL PROTECTED] Pervasive Software http://pervasive.comwork: 512-231-6117 vcard: http://jim.nasby.net/pervasive.vcf cell: 512-569-9461 ---(end of broadcast)--- TIP 4: Have you searched our list archives? http://archives.postgresql.org
Re: [HACKERS] Generating config stuff from single source
Peter Eisentraut <[EMAIL PROTECTED]> writes: > Am Donnerstag, 16. Februar 2006 02:50 schrieb Tom Lane: >> The m4 idea seems attractive to me because that's already effectively >> required as part of the autoconf infrastructure (and I think bison >> uses it too these days). > That is true, but I'm afraid that this will lead to code that only a few > people will be able to maintain. (Try programming a loop in m4 to start.) >> A similar issue that's been bothering me for awhile is that it'd be a >> lot less error prone if keywords.c and the keyword list productions in >> gram.y were generated off a common declarative source (which might as >> well produce the keyword documentation appendix too). > That could be a job for m4, I think. Hmm ... well, if we are going to do this other thing with xsltproc, the keyword problem should probably be solved with the same tool. I hesitate to mention this, but: we have several other derived files in the tarball (postgres.bki, fmgroids.h, etc) and those are all built with shell/awk/sed/perl scripts. How out of the question is it to solve the GUC problem with that technology? regards, tom lane ---(end of broadcast)--- TIP 6: explain analyze is your friend
Re: [HACKERS] Generating config stuff from single source
Peter Eisentraut <[EMAIL PROTECTED]> writes: > Am Donnerstag, 16. Februar 2006 02:50 schrieb Tom Lane: >> That's fine for users, but what new demands are you about to place on >> developers? Does this require tools not already needed in order to >> build from a CVS pull? (There's sure no xsltproc on this machine...) > It is to be expected that sooner or later we'll move from SGML to XML > documentation builds, at which point xsltproc will become a semi-requirement > anyway. I don't think this requirement is too onerous; libxslt is portable > and easy to install. Forgot to mention, but: I don't find the above argument very convincing. The buildfarm machines are not expected to build documentation, and many developers seem not to have installed doc tools either. So I think this would be raising the bar another notch in terms of what's required to do development or testing, even if it does overlap with docs-build needs. regards, tom lane ---(end of broadcast)--- TIP 3: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faq
Re: [HACKERS] Generating config stuff from single source
On Thu, Feb 16, 2006 at 02:36:01AM +0100, Peter Eisentraut wrote: > We are currently maintaining information about configuration parameters > in at least three places: the documentation, guc.c, and > postgresql.conf.sample. I would like to generate these from a single > source. Computationally, this is not very challenging, it's just a bit > of work. I imagine as the source an XML file with a custom schema; see > below for an example. I think this is the best source format because > it allows integrating the DocBook-formatted descriptions without too > much trouble and it allows for file format validation. An alternative > might be m4 but that would not offer these features. To process this > we'd use XSLT stylesheets run through xsltproc. We'd run this part > during the tarball building phase, so users would not need it. > Obviously, all of this will need some fine-tuning, but can we agree on > this general direction? Is there any reason why we can't just use something like awk? It's installed almost everywhere (it may be required, not sure) and a lot more people know how to use it. I havn't managed to wrap my brain around xslt yet. The input file could be simply line based. Attached is a simple script that takes the input below and produces something resembling what you suggested. It wouldn't be too hard to get it to produce multiple output formats and dump the output to different files... Group: Query Tuning Subgroup: Planner Method Configuration Name: enable_hashagg Context: userset ... etc ... -- Martijn van Oosterhout http://svana.org/kleptog/ > Patent. n. Genius is 5% inspiration and 95% perspiration. A patent is a > tool for doing 5% of the work and then sitting around waiting for someone > else to do the other 95% so you can sue them. BEGIN { havedata = 0; group = subgroup = ""; print "\n"; } /^[A-Za-z]+: *(.*)/ { keyword = tolower(substr( $0, 1, index( $0, ":" ) - 1 ) ); data = substr( $0, index( $0, ":" ) + 1 ); sub( /^ +/, "", data ); sub( / +$/, "", data ); if( keyword == "group" ) { if( group != "" ) { print "\n" } printf "%s\n", data; group = data; subgroup = ""; next; } if( keyword == "subgroup" ) { if( subgroup != "" ) { printf "\n", subgroup } printf "%s\n", data; subgroup = data; next; } havedata = 1; fields[keyword] = data; next; } /^ *$/ { if( havedata == 0 ) { next } print "\n"; for( keyword in fields ) { printf "<%s>%s<%s>\n", keyword, fields[keyword], keyword } print "\n"; havedata = 0; delete fields; next; } { printf "Invalid input: %s\n", $0; exit; } END { if( havedata == 1 ) { print "\n"; for( keyword in fields ) { printf "<%s>%s<%s>\n", keyword, fields[keyword], keyword } print "\n"; } if( subgroup != "" ) { print "\n" } if( group != "" ) { print "\n" } print "\n"; } signature.asc Description: Digital signature
Re: [HACKERS] Generating config stuff from single source
Am Donnerstag, 16. Februar 2006 02:50 schrieb Tom Lane: > That's fine for users, but what new demands are you about to place on > developers? Does this require tools not already needed in order to > build from a CVS pull? (There's sure no xsltproc on this machine...) It is to be expected that sooner or later we'll move from SGML to XML documentation builds, at which point xsltproc will become a semi-requirement anyway. I don't think this requirement is too onerous; libxslt is portable and easy to install. > The m4 idea seems attractive to me because that's already effectively > required as part of the autoconf infrastructure (and I think bison > uses it too these days). That is true, but I'm afraid that this will lead to code that only a few people will be able to maintain. (Try programming a loop in m4 to start.) > A similar issue that's been bothering me for awhile is that it'd be a > lot less error prone if keywords.c and the keyword list productions in > gram.y were generated off a common declarative source (which might as > well produce the keyword documentation appendix too). That could be a job for m4, I think. -- Peter Eisentraut http://developer.postgresql.org/~petere/ ---(end of broadcast)--- TIP 5: don't forget to increase your free space map settings
Re: [HACKERS] Generating config stuff from single source
Peter Eisentraut <[EMAIL PROTECTED]> writes: > We are currently maintaining information about configuration parameters > in at least three places: the documentation, guc.c, and > postgresql.conf.sample. I would like to generate these from a single > source. Computationally, this is not very challenging, it's just a bit > of work. I imagine as the source an XML file with a custom schema; see > below for an example. I think this is the best source format because > it allows integrating the DocBook-formatted descriptions without too > much trouble and it allows for file format validation. An alternative > might be m4 but that would not offer these features. To process this > we'd use XSLT stylesheets run through xsltproc. We'd run this part > during the tarball building phase, so users would not need it. That's fine for users, but what new demands are you about to place on developers? Does this require tools not already needed in order to build from a CVS pull? (There's sure no xsltproc on this machine...) ISTM the last thing we need at the moment is to put more obstacles in the way of new developers, or for that matter new buildfarm entrants. The m4 idea seems attractive to me because that's already effectively required as part of the autoconf infrastructure (and I think bison uses it too these days). A similar issue that's been bothering me for awhile is that it'd be a lot less error prone if keywords.c and the keyword list productions in gram.y were generated off a common declarative source (which might as well produce the keyword documentation appendix too). But again, I'm not sure what kind of price I want to pay in infrastructure for that. regards, tom lane ---(end of broadcast)--- TIP 9: In versions below 8.0, the planner will ignore your desire to choose an index scan if your joining column's datatypes do not match
[HACKERS] Generating config stuff from single source
We are currently maintaining information about configuration parameters in at least three places: the documentation, guc.c, and postgresql.conf.sample. I would like to generate these from a single source. Computationally, this is not very challenging, it's just a bit of work. I imagine as the source an XML file with a custom schema; see below for an example. I think this is the best source format because it allows integrating the DocBook-formatted descriptions without too much trouble and it allows for file format validation. An alternative might be m4 but that would not offer these features. To process this we'd use XSLT stylesheets run through xsltproc. We'd run this part during the tarball building phase, so users would not need it. Obviously, all of this will need some fine-tuning, but can we agree on this general direction? Query Tuning Planer Method Configuration enable_hashagg userset Enables the planner's use of hashed aggregation plans. blah bool enable_hashagg true ... ... ... ... -- Peter Eisentraut http://developer.postgresql.org/~petere/ ---(end of broadcast)--- TIP 6: explain analyze is your friend