Re: [HACKERS] Generating config stuff from single source
Martijn van Oosterhout wrote: The input file could be simply line based. Attached is a simple script that takes the input below and produces something resembling what you suggested. It wouldn't be too hard to get it to produce multiple output formats and dump the output to different files... Group: Query Tuning Subgroup: Planner Method Configuration Name: enable_hashagg Context: userset ... etc ... I'm trying out this input format now, processing it with Perl, and it seems to be quite reasonable. There are a number of special cases, however, that need to be worked out. I'm trying to generate guc.c now and once that works I'll report here. -- Peter Eisentraut http://developer.postgresql.org/~petere/ ---(end of broadcast)--- TIP 5: don't forget to increase your free space map settings
Re: [HACKERS] Generating config stuff from single source
On Thu, Feb 16, 2006 at 10:52:19AM -0500, Tom Lane wrote: Peter Eisentraut [EMAIL PROTECTED] writes: Am Donnerstag, 16. Februar 2006 02:50 schrieb Tom Lane: That's fine for users, but what new demands are you about to place on developers? Does this require tools not already needed in order to build from a CVS pull? (There's sure no xsltproc on this machine...) It is to be expected that sooner or later we'll move from SGML to XML documentation builds, at which point xsltproc will become a semi-requirement anyway. I don't think this requirement is too onerous; libxslt is portable and easy to install. Forgot to mention, but: I don't find the above argument very convincing. The buildfarm machines are not expected to build documentation, and many developers seem not to have installed doc tools either. So I think this would be raising the bar another notch in terms of what's required to do development or testing, even if it does overlap with docs-build needs. From what I've seen it's not terribly difficult to install some sort of XSLT processor now-a-days. It's certainly less involved than installing docbook in any case. -- Jim C. Nasby, Sr. Engineering Consultant [EMAIL PROTECTED] Pervasive Software http://pervasive.comwork: 512-231-6117 vcard: http://jim.nasby.net/pervasive.vcf cell: 512-569-9461 ---(end of broadcast)--- TIP 4: Have you searched our list archives? http://archives.postgresql.org
Re: [HACKERS] Generating config stuff from single source
Am Donnerstag, 16. Februar 2006 02:50 schrieb Tom Lane: That's fine for users, but what new demands are you about to place on developers? Does this require tools not already needed in order to build from a CVS pull? (There's sure no xsltproc on this machine...) It is to be expected that sooner or later we'll move from SGML to XML documentation builds, at which point xsltproc will become a semi-requirement anyway. I don't think this requirement is too onerous; libxslt is portable and easy to install. The m4 idea seems attractive to me because that's already effectively required as part of the autoconf infrastructure (and I think bison uses it too these days). That is true, but I'm afraid that this will lead to code that only a few people will be able to maintain. (Try programming a loop in m4 to start.) A similar issue that's been bothering me for awhile is that it'd be a lot less error prone if keywords.c and the keyword list productions in gram.y were generated off a common declarative source (which might as well produce the keyword documentation appendix too). That could be a job for m4, I think. -- Peter Eisentraut http://developer.postgresql.org/~petere/ ---(end of broadcast)--- TIP 5: don't forget to increase your free space map settings
Re: [HACKERS] Generating config stuff from single source
On Thu, Feb 16, 2006 at 02:36:01AM +0100, Peter Eisentraut wrote:
We are currently maintaining information about configuration parameters
in at least three places: the documentation, guc.c, and
postgresql.conf.sample. I would like to generate these from a single
source. Computationally, this is not very challenging, it's just a bit
of work. I imagine as the source an XML file with a custom schema; see
below for an example. I think this is the best source format because
it allows integrating the DocBook-formatted descriptions without too
much trouble and it allows for file format validation. An alternative
might be m4 but that would not offer these features. To process this
we'd use XSLT stylesheets run through xsltproc. We'd run this part
during the tarball building phase, so users would not need it.
Obviously, all of this will need some fine-tuning, but can we agree on
this general direction?
Is there any reason why we can't just use something like awk? It's
installed almost everywhere (it may be required, not sure) and a lot
more people know how to use it. I havn't managed to wrap my brain
around xslt yet.
The input file could be simply line based. Attached is a simple script
that takes the input below and produces something resembling what you
suggested. It wouldn't be too hard to get it to produce multiple output
formats and dump the output to different files...
Group: Query Tuning
Subgroup: Planner Method Configuration
Name: enable_hashagg
Context: userset
... etc ...
--
Martijn van Oosterhout kleptog@svana.org http://svana.org/kleptog/
Patent. n. Genius is 5% inspiration and 95% perspiration. A patent is a
tool for doing 5% of the work and then sitting around waiting for someone
else to do the other 95% so you can sue them.
BEGIN {
havedata = 0;
group = subgroup = ;
print parameters\n;
}
/^[A-Za-z]+: *(.*)/ {
keyword = tolower(substr( $0, 1, index( $0, : ) - 1 ) );
data = substr( $0, index( $0, : ) + 1 );
sub( /^ +/, , data );
sub( / +$/, , data );
if( keyword == group )
{
if( group != ) { print /group\n }
printf grouptitle%s/title\n, data;
group = data;
subgroup = ;
next;
}
if( keyword == subgroup )
{
if( subgroup != ) { printf /subgroup[%s]\n, subgroup }
printf subgrouptitle%s/title\n, data;
subgroup = data;
next;
}
havedata = 1;
fields[keyword] = data;
next;
}
/^ *$/ {
if( havedata == 0 ) { next }
print parameter\n;
for( keyword in fields )
{ printf %s%s%s\n, keyword, fields[keyword], keyword }
print /parameter\n;
havedata = 0;
delete fields;
next;
}
{ printf Invalid input: %s\n, $0; exit; }
END {
if( havedata == 1 )
{
print parameter\n;
for( keyword in fields )
{ printf %s%s%s\n, keyword, fields[keyword], keyword }
print /parameter\n;
}
if( subgroup != )
{ print /subgroup\n }
if( group != )
{ print /group\n }
print /parameters\n;
}
signature.asc
Description: Digital signature
Re: [HACKERS] Generating config stuff from single source
Peter Eisentraut [EMAIL PROTECTED] writes: Am Donnerstag, 16. Februar 2006 02:50 schrieb Tom Lane: That's fine for users, but what new demands are you about to place on developers? Does this require tools not already needed in order to build from a CVS pull? (There's sure no xsltproc on this machine...) It is to be expected that sooner or later we'll move from SGML to XML documentation builds, at which point xsltproc will become a semi-requirement anyway. I don't think this requirement is too onerous; libxslt is portable and easy to install. Forgot to mention, but: I don't find the above argument very convincing. The buildfarm machines are not expected to build documentation, and many developers seem not to have installed doc tools either. So I think this would be raising the bar another notch in terms of what's required to do development or testing, even if it does overlap with docs-build needs. regards, tom lane ---(end of broadcast)--- TIP 3: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faq
Re: [HACKERS] Generating config stuff from single source
Peter Eisentraut [EMAIL PROTECTED] writes: Am Donnerstag, 16. Februar 2006 02:50 schrieb Tom Lane: The m4 idea seems attractive to me because that's already effectively required as part of the autoconf infrastructure (and I think bison uses it too these days). That is true, but I'm afraid that this will lead to code that only a few people will be able to maintain. (Try programming a loop in m4 to start.) A similar issue that's been bothering me for awhile is that it'd be a lot less error prone if keywords.c and the keyword list productions in gram.y were generated off a common declarative source (which might as well produce the keyword documentation appendix too). That could be a job for m4, I think. Hmm ... well, if we are going to do this other thing with xsltproc, the keyword problem should probably be solved with the same tool. I hesitate to mention this, but: we have several other derived files in the tarball (postgres.bki, fmgroids.h, etc) and those are all built with shell/awk/sed/perl scripts. How out of the question is it to solve the GUC problem with that technology? regards, tom lane ---(end of broadcast)--- TIP 6: explain analyze is your friend
[HACKERS] Generating config stuff from single source
We are currently maintaining information about configuration parameters in at least three places: the documentation, guc.c, and postgresql.conf.sample. I would like to generate these from a single source. Computationally, this is not very challenging, it's just a bit of work. I imagine as the source an XML file with a custom schema; see below for an example. I think this is the best source format because it allows integrating the DocBook-formatted descriptions without too much trouble and it allows for file format validation. An alternative might be m4 but that would not offer these features. To process this we'd use XSLT stylesheets run through xsltproc. We'd run this part during the tarball building phase, so users would not need it. Obviously, all of this will need some fine-tuning, but can we agree on this general direction? parameters group titleQuery Tuning/title subgroup titlePlaner Method Configuration/title parameter nameenable_hashagg/name contextuserset/context shortdescEnables the planner's use of hashed aggregation plans./shortdesc longdescblah/longdesc vartypebool/vartype variableenable_hashagg/variable resetvaltrue/resetval min.../min max.../max assignhook.../assignhook showhook.../showhook /parameter /subgroup /group /parameters -- Peter Eisentraut http://developer.postgresql.org/~petere/ ---(end of broadcast)--- TIP 6: explain analyze is your friend
